Device and method for processing and disambiguating touch input

Abstract

The present disclosure generally relates to devices and methods for processing and disambiguating touch inputs. An electronic device detects a first intensity increase of an input on an input followed by a first intensity decrease of the input on the input and determines whether the first decrease satisfies a release click detection criterion. The release click detection criterion requires that the intensity of the input decrease to be below a first release click intensity threshold, which is selected based on the intensity of the input during the first intensity increase. Based on determining that the first intensity decrease of the input satisfies the release click detection criterion, the device provides first feedback, such as performing an operation to change a displayed user interface and / or providing a tactile output, indicating that the first intensity decrease of the input was recognized as a release click.

Description

[0001] This application is a divisional application of the invention patent application with application number 201910352081.4, application date July 12, 2017, and name “Device and method for processing and eliminating ambiguity of touch input”. Technical Field

[0002]

[0013] This document generally relates to electronic devices having one or more intensity-sensitive input elements, including but not limited to electronic devices having a touch-sensitive display and optionally including other input elements for detecting intensity of contacts on a touch-sensitive surface. Background Art

[0003] The use of intensity-sensitive input elements, including but not limited to touch-sensitive surfaces, as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary intensity-sensitive input elements include buttons with contact intensity sensors, and touch pads and touch screen displays with contact intensity sensors. Touch input on such surfaces is used to manipulate user interfaces and user interface objects on the display.

[0004] Exemplary user interface objects include digital images, videos, text, icons, control elements (such as buttons), and other graphics. Exemplary manipulations include: adjusting the position and / or size of one or more user interface objects or activating a button represented by a user interface object or opening a file / application represented by a user interface object; scrolling or changing a user interface within an application or otherwise manipulating a user interface. Certain manipulations of user interface objects or user interfaces are associated with specific types of touch inputs known as gestures.

[0005] Conventional methods and interfaces for processing touch input are ineffective in disambiguating certain touch inputs to determine the intended gesture and intended user interface object manipulation. It is therefore desirable to have an improved processing and disambiguation framework for touch input. Summary of the invention

[0006] Therefore, there is a need for electronic devices with faster and more efficient methods and interfaces for processing and disambiguating touch inputs. Such methods and interfaces optionally supplement or replace conventional methods for processing and disambiguating touch inputs. Such methods and interfaces reduce the amount, degree, and / or nature of inputs from users and produce more efficient human-computer interfaces. For battery-powered devices, such methods and interfaces can save power and increase the time between battery charges.

[0007] The above-mentioned defects and other problems associated with the user interface of an electronic device with a touch-sensitive surface can be reduced or eliminated by means of the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, a tablet computer, or a handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also referred to as a "touch screen" or a "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, a memory, and one or more modules, a program or instruction set stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI mainly through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, these functions optionally include image editing, drawing, display, word processing, spreadsheet creation, playing games, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, note-taking and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0008] According to some embodiments, a method performed on an electronic device having a display and an intensity-sensitive input element (e.g., a hardware button, a touch-sensitive surface, or an area of ​​a device associated with one or more intensity sensors) includes: detecting a first intensity increase of an input on the input element that satisfies a press-click detection criterion, and after detecting the first intensity increase of the input on the input element, detecting a first intensity decrease of a contact, the intensity-sensitive input element being used to detect the intensity of a user input made with the input element. The method also includes determining whether the first intensity decrease of the input satisfies a release-click detection criterion, wherein: for the first intensity decrease, the release-click detection criterion requires that the intensity of the input decrease to be below a first release-click intensity threshold in order for the release-click detection criterion to be satisfied, the first release-click intensity threshold being selected based on the intensity of the input during an intensity increase of the contact detected before detecting the first intensity decrease of the input. The method also includes, based on determining that the first intensity decrease of the input satisfies the release-click detection criterion, providing first feedback indicating that the first intensity decrease of the input is identified as a release-click input, and based on determining that the intensity decrease of the input does not satisfy the release-click detection criterion, forgoing providing the first feedback.

[0009] According to some embodiments, a method performed on an electronic device having a display and an intensity-sensitive input element (e.g., a hardware button, a touch-sensitive surface, or an area of ​​a device associated with one or more intensity sensors) (for detecting intensity of user input made with the input element) includes: detecting a change in intensity of an input on the input element, the intensity change comprising an increase in intensity of the input on the input element followed by a decrease in intensity of the input on the input element; identifying at least a portion of the change in intensity of the input as a first input event associated with a first operation; and after identifying the first input event, delaying performance of the first operation while monitoring subsequent intensity changes of the input for a second input event, wherein the delay is limited by a default delay time period. The method also includes, after delaying the execution of the first operation: based on determining that the second input event has been identified before the default delay time period has passed, executing the second operation and abandoning the execution of the first operation; based on determining that the early confirmation criteria for the first input event have been met before the default delay time period has passed and the second input event has not been identified, executing the first operation before the default delay time period has passed; and based on determining that the default delay time period has passed and the early confirmation criteria for the first input event have not been met and the second input event has not been identified, executing the first operation once the default delay time period has passed.

[0010] According to some embodiments, a method performed on an electronic device having a display and an intensity-sensitive input element (e.g., a hardware button, a touch-sensitive surface, or an area of ​​a device associated with one or more intensity sensors) includes detecting an input sequence including an increase in the intensity of an input corresponding to a first input event, the intensity-sensitive input element being used to detect the intensity of a user input performed with the input element. The method also includes, in response to detecting the input sequence: performing a first operation based on determining that a second input event (including a decrease in the intensity of the input after the first input event) is detected within a first time period after the first input event is detected. The method also includes performing a second operation once the second time period has passed based on determining that the second input event is not detected within a second time period longer than the first time period and the input has a characteristic intensity above a corresponding intensity threshold between when the first input event is detected and when the second time period has passed, wherein the second time period is determined at least in part based on the intensity of the input after the first input event is detected. The method also includes performing the second operation once the third time period has passed based on determining that the second input event is not detected within a third time period longer than the second time period and the input does not have a characteristic intensity above the corresponding intensity threshold between when the first input event is detected and when the second time period has passed.

[0011] According to some embodiments, an electronic device includes a display, a touch-sensitive surface, a memory, one or more processors, one or more programs, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface; the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the execution of the operations of any method described herein. According to some embodiments, a computer-readable storage medium has instructions stored therein, which, when executed by an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, causes the device to perform the operations of any method described herein or causes the operations of any method described herein to be performed. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, a memory, and one or more processors executing one or more programs stored in the memory includes one or more elements displayed in any method described herein, and the one or more elements are updated in response to input, as described in any method described herein. According to some embodiments, an electronic device includes: a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface; and a device for performing or causing the performance of the operations of any of the methods described herein. According to some embodiments, an information processing device for an electronic device having a display and a touch-sensitive surface and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface includes a device for performing or causing the performance of the operations of any of the methods described herein.

[0012] Thus, electronic devices having a display, one or more touch-sensitive surfaces, and one or more sensors for detecting intensity of contact with the touch-sensitive surfaces are provided with faster, more efficient methods and interfaces for processing and disambiguating touch inputs, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may supplement or replace conventional methods for processing and disambiguating touch inputs. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0014] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

[0015] Figure 1Bis a block diagram illustrating example components for event processing according to some embodiments.

[0016] Figure 1C is a block diagram illustrating a tactile output module according to some embodiments.

[0017] Figure 2A A portable multifunction device with a touch screen according to some embodiments is shown.

[0018] Figure 2B-2C An exploded view of an intensity-sensitive input device according to some embodiments is shown.

[0019] Figure 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0020] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some embodiments is shown.

[0021] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0022] FIG. 4C to FIG. 4E Examples of dynamic intensity thresholds according to some embodiments are shown.

[0023] Figure 4F-4G A set of sample tactile output patterns according to some embodiments are shown.

[0024] Figure 5A-5II An exemplary user interface and multiple timeout periods and intensity thresholds for detecting gestures are shown according to some embodiments.

[0025] Figures 6A-6F is a flow chart illustrating a method of processing and disambiguating touch input according to some embodiments.

[0026] Figures 7A-7E is a flow chart illustrating a method of processing and disambiguating touch input according to some embodiments.

[0027] Figures 8A-8C is a flow chart illustrating a method of processing and disambiguating touch input according to some embodiments.

[0028] Figure 9-11 is a functional block diagram of an electronic device according to some embodiments. DETAILED DESCRIPTION

[0029] In an electronic device that displays a graphical user interface and has an intensity-sensitive input element, it is challenging to accurately and quickly detect and distinguish multiple different gestures (such as a deep press gesture, a long press gesture, a single-click gesture, a double-click gesture, and a possible triple-click gesture). Such gestures involve inputs with time-varying intensity. Distinguishing such gestures requires analyzing the intensity of the input on the input element and analyzing the timing aspects or characteristics of the input. In addition, some users have a "heavier touch" than others, applying more pressure than other users on average. Similarly, some users input gestures at a higher speed than other users. Despite these different modes or styles of user input, electronic devices need to accurately discern user intent, and must discern user intent with low latency so that the operation of user requests or commands is quickly performed in response to the user's touch input. In some embodiments, in order to achieve such accuracy and touch input processing performance, one or more intensity thresholds for detecting gestures or gesture portions vary according to the intensity of the user's input during one or more preceding portions of the gesture. Furthermore, in some embodiments, to achieve such accuracy and touch input processing performance, one or more time periods used in the analysis of touch input vary based on the intensity of the touch input, thereby enabling faster recognition of particular gestures when predefined criteria are met.

[0030] On the other hand, tactile feedback (also referred to as tactile output) can be used to facilitate user input, confirm recognition of various user inputs, and prompt the user of the occurrence of various events, various input conditions, etc. As the number and complexity of tactile feedback events in a device increase, it becomes important to ensure that particular tactile outputs are consistently generated, even when the detection criteria for triggering those tactile outputs vary (e.g., the detection criteria for triggering a long press tactile output may be different for different applications, or may be different for different contexts within the same application).

[0031] under, Figure 1A-1B , Figure 2 and Figure 3 A description of an exemplary apparatus is provided. Figure 4A-4B 5A-5II show an exemplary user interface of an electronic device that is configured to monitor input on an intensity-sensitive input element and detect various events (such as a release click, a press click, a single click, a double click, a deep press input, and a long press input) using various input intensity standards and timing standards for quickly and efficiently determining user input. Figures 6A-6F A flow chart illustrating a method of monitoring input on an intensity-sensitive input element and detecting a release click and / or a press click in the monitored input using one or more intensity thresholds based on previous input intensities of the input. Figures 7A-7E A flow chart illustrating a method of monitoring changes in intensity of an input and applying early confirmation criteria for accelerating identification of single-click inputs as distinct from double-click inputs. Figures 8A-8C A flow chart showing a method of monitoring intensity changes of an input and applying intensity-sensitive criteria for faster identification of a long press input. Figures 5A-5N The user interface in Figures 6A-6F process. Figure 5O-5Y The user interface in Figures 7A-7E process. Figure 5Z-5II The user interface in Figures 8A-8C process.

[0032] Exemplary Devices

[0033] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail, so as not to unnecessarily obscure various aspects of the embodiments.

[0034] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact, unless the context clearly indicates otherwise.

[0035] The terms used in the description of various described embodiments in this article are only for the purpose of describing specific embodiments, and are not intended to be limited. As used in the description of various described embodiments and in the appended claims, the singular forms "one" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more items in the items listed in association. It will also be understood that the term "includes" "including" "comprises" and / or "comprising" specifies the presence of stated features, integers, steps, operations, elements, and / or parts when used in this specification, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or its grouping.

[0036] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that ..." or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that ..." or "in response to determining that ..." or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0037] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple iPod from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads) are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0038] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse and / or joystick.

[0039] The device typically supports a variety of applications, such as one or more of the following applications: a note-taking application, a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital camcorder application, a web browsing application, a digital music player application, and / or a digital video player application.

[0040] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and clear to the user.

[0041] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A 1 is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes simply referred to as a touch-sensitive display. The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input, or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0042] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the touch-sensitive surface does not change, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. Although such interpretation of touch by the user will be limited by the user's individualized sensory perceptions, there are many sensory perceptions of touch that are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "release click," "press click," "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that would generate the described sensory perception of a typical (or ordinary) user. Providing tactile feedback to the user using tactile output enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0043] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the motion waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.

[0044] When a device (e.g., one or more tactile output generators that generate tactile output via moving a movable mass) generates tactile output with different tactile output patterns, the tactile output can produce different tactile sensations in a user holding or touching the device. Although the user's senses are based on the user's perception of the tactile output, most users will be able to recognize changes in the waveform, frequency, and amplitude of the tactile output generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different operations have been performed. In this way, a tactile output with a tactile output pattern designed, selected, and / or arranged to simulate the characteristics (e.g., size, material, weight, stiffness, smoothness, etc.) of an object in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or a combination of any of the above); behavior (e.g., oscillation, displacement, acceleration, rotation, extension, etc.); and / or interaction (e.g., collision, adhesion, repulsion, attraction, friction, etc.) will provide helpful feedback to the user in some cases, which reduces input errors and improves the efficiency of the user's operation of the device. In addition, tactile output is optionally generated to correspond to feedback that is independent of simulated physical properties (such as input threshold or object selection). Such tactile output will provide helpful feedback to the user in some cases, which reduces input errors and improves the efficiency of the user's operation of the device.

[0045] In some embodiments, the tactile output with a suitable tactile output mode serves as a prompt for an event of interest in the user interface or behind the screen in the device. Examples of events of interest include activation of affordances (such as real or virtual buttons or toggle switches) provided on the device or in the user interface, the success or failure of the requested operation, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or identifying a type of input or gesture, etc. In some embodiments, a tactile output is provided to serve as a warning or prompt for an upcoming event or result that will occur unless a change in direction or interruption input is detected in time. Tactile output is also used in other contexts to enrich user experience, improve the accessibility of users with visual or motor difficulties or other accessibility needs to the device, and / or improve the efficiency and functionality of the user interface and / or device. Optionally, the tactile output is compared with audio input and / or visual user interface changes, which further enhances the user's experience when the user interacts with the user interface and / or device, and is conducive to better transmission of information about the state of the user interface and / or device, and this reduces input errors and improves the efficiency of the user's operation on the device.

[0046] Figure 4FA set of sample tactile output patterns are provided that can be used, individually or in combination, as is or through one or more transformations (e.g., modulation, amplification, truncation, etc.) to generate suitable tactile feedback in various scenarios for various purposes (such as those described above and those described for the user interfaces and methods discussed herein). This example of a control panel for tactile output shows how a set of three waveforms and eight frequencies can be used to generate an array of tactile output patterns. In addition to the tactile output patterns shown in the figure, each of these tactile output patterns is optionally adjusted in amplitude by changing the gain value of the tactile output pattern, as shown in the figure, for example, for Figure 4G 80Hz, FullTap 200Hz, MiniTap 80Hz, MiniTap 200Hz, MicroTap 80Hz, and MicroTap 200Hz, each of which is illustrated with a gain of 1.0, 0.75, 0.5, and 0.25. Figure 4G As shown, changing the gain of a tactile output pattern changes the amplitude of the pattern without changing the frequency of the pattern or changing the shape of the waveform. In some embodiments, changing the frequency of the tactile output pattern also results in a lower amplitude because some tactile output generators are limited by how much force can be applied to the movable mass, so the higher frequency movement of the mass is constrained to a lower amplitude to ensure that the acceleration required to generate the waveform does not require forces outside the operating force range of the tactile output generator (e.g., the peak amplitude of FullTap at 230 Hz, 270 Hz, and 300 Hz is lower than the amplitude of FullTap at 80 Hz, 100 Hz, 125 Hz, and 200 Hz).

[0047] exist Figure 4F In FIG. 1 , each column shows a tactile output pattern having a specific waveform. The waveform of the tactile output pattern represents the position of the tactile output pattern relative to the neutral position (e.g., x zero ) versus time, through which the movable mass passes to generate a tactile output having the tactile output pattern. For example, Figure 4F The first set of tactile output patterns shown in the left column (eg, the “FullTap” tactile output pattern) each have a waveform that includes oscillations having two complete cycles (eg, oscillations that start and end at a neutral position and pass through the neutral position three times). Figure 4F The second set of tactile output patterns shown in the middle column (eg, the “MiniTap” tactile output pattern) each have a waveform that includes an oscillation having one complete cycle (eg, an oscillation that starts and ends at a neutral position and passes through the neutral position once). Figure 4FThe third set of tactile output patterns shown in the right column (e.g., the tactile output patterns of "MicroTap") each have a waveform that includes an oscillation with half a complete cycle (e.g., an oscillation that starts and ends at a neutral position and does not pass through the neutral position). The waveforms of the tactile output patterns also include a start buffer and an end buffer that represent the gradual acceleration and deceleration of the movable mass at the beginning and end of the tactile output. Figure 4F-4G The exemplary waveforms shown include x and y representing the maximum and minimum extents of movement of the movable mass. min and x max For larger electronic devices with larger movable masses, the minimum and maximum degrees of movement of the mass may be larger or smaller. Figure 4F-4G The examples shown describe movement of a mass in 1 dimension, but similar principles may also apply to movement of a movable mass in two or three dimensions.

[0048] like Figure 4F As shown, each tactile output pattern also has a corresponding characteristic frequency, which affects the "pitch" of the tactile sensation felt by the user from the tactile output having the characteristic frequency. For continuous tactile output, the characteristic frequency represents the number of cycles (e.g., cycles per second) completed by the movable mass of the tactile output generator in a given time period. For discrete tactile output, a discrete output signal is generated (e.g., having 0.5, 1, or 2 cycles), and the characteristic frequency value specifies how fast the movable mass needs to move to generate a tactile output having the characteristic frequency. Figure 4F As shown, for each type of tactile output (e.g., defined by a corresponding waveform, such as FullTap, MiniTap, or MicroTap), higher frequency values ​​correspond to faster movement of the movable mass and, therefore, generally speaking, to shorter tactile output completion times (e.g., including the time required to complete the number of cycles of the discrete tactile output plus the start and end buffer times). For example, a FullTap with a characteristic frequency of 80 Hz takes longer to complete (e.g., in 100 Hz) than a FullTap with a characteristic frequency of 100 Hz. Figure 4F, 35.4ms vs. 28.3ms). In addition, for a given frequency, a tactile output having more cycles in its waveform at the corresponding frequency takes longer to complete than a tactile output having fewer cycles in its waveform at the same corresponding frequency. For example, a FullTap at 150Hz takes longer to complete than a MiniTap at 150Hz (e.g., 19.4ms vs. 12.8ms), and a MiniTap at 150Hz takes longer to complete than a MicroTap at 150Hz (e.g., 12.8ms vs. 9.4ms). However, for tactile output modes with different frequencies, this rule may not apply (e.g., a tactile output with more cycles but a higher frequency may take a shorter amount of time to complete than a tactile output with fewer cycles but a lower frequency, and vice versa). For example, at 300Hz, FullTap takes the same amount of time as MiniTap (e.g., 9.9ms).

[0049] like Figure 4F As shown, the tactile output pattern also has a characteristic amplitude, which affects the amount of energy contained in the tactile signal, or the "intensity" of the tactile sensation that the user can feel through the tactile output having the characteristic amplitude. In some embodiments, the characteristic amplitude of the tactile output pattern refers to an absolute or normalized value representing the maximum displacement of the movable mass relative to the neutral position when generating the tactile output. In some embodiments, the characteristic amplitude of the tactile output pattern can be adjusted according to various conditions (e.g., customized based on user interface context and behavior) and / or preconfigured metrics (e.g., input-based metrics, and / or user interface-based metrics), such as by a fixed or dynamically determined gain factor (e.g., a value between 0 and 1). In some embodiments, an input-based metric (e.g., an intensity change metric or an input speed metric) measures the characteristics of the input during the input that triggers the generation of the tactile output (e.g., the rate of change of the characteristic intensity of the contact in the press input or the rate of movement of the contact on the touch-sensitive surface). In some embodiments, a user interface-based metric (e.g., a cross-border speed metric) measures a characteristic of a user interface element during a user interface change that triggers generation of a tactile output (e.g., how fast the element moves across a hidden or visible boundary in the user interface). In some embodiments, a characteristic amplitude of a tactile output pattern may be "envelope" modulated, and peaks of adjacent cycles may have different amplitudes, wherein one of the waveforms shown above is further modified by multiplying an envelope parameter that changes over time (e.g., from 0 to 1) to gradually adjust the amplitude of a portion of the tactile output over time when generating the tactile output.

[0050] Although in Figure 4FSpecific frequencies, amplitudes, and waveforms are shown in the sample tactile output patterns for illustrative purposes, but tactile output patterns with other frequencies, amplitudes, and waveforms can also be used for similar purposes. For example, a waveform with between 0.5 and 4 cycles can be used. Other frequencies in the range of 60 Hz-400 Hz can also be used. Table 1 provides examples of specific tactile feedback behaviors, configurations, and examples of their use.

[0051] Table 1

[0052]

[0053]

[0054]

[0055] It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof (including one or more signal processing circuits and / or application specific integrated circuits).

[0056] The memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory 102 by other components of the device 100 (such as one or more CPUs 120 and peripheral device interface 118) is optionally controlled by a memory controller 122.

[0057] Peripheral interface 118 may be used to couple the device's input and output peripherals to memory 102 and one or more CPUs 120. The one or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0058] In some embodiments, peripherals interface 118, one or more CPUs 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0059] RF (radio frequency) circuit 108 receives and transmits RF signals, also called electromagnetic signals. RF circuit 108 converts electrical signals into / from electromagnetic signals and communicates with a communication network and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. RF circuit 108 optionally communicates with a network and other devices via wireless communications, such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). The wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution-Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilizing Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols that have not been developed on the date of filing of this document.

[0060] The audio circuit 110, the speaker 111, and the microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headphone jack (e.g., Figure 2A 212 in). The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single-ear headset or a dual-ear headset) and input (e.g., a microphone).

[0061] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from other input control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., Figure 2A 208) optionally includes an up / down button for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2A 206).

[0062] The touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch-sensitive display system 112 and / or sends electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some visual outputs or all of the visual outputs correspond to user interface objects. As used herein, the term "indicator" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to input directed to a graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0063] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from the user based on tactile and / or haptic contact. The touch-sensitive display system 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0064] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112 to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as the . iPod and Technology found in.

[0065] The touch-sensitive display system 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi, or greater). The user optionally uses any suitable object or appendage such as a stylus, finger, etc. to contact the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as accurate as stylus-based input due to the larger contact area of ​​the finger on the touch screen. In some embodiments, the device translates the rough finger-based input into a precise pointer / cursor position or command for performing the action desired by the user.

[0066] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touch screen.

[0067] The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0068] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. One or more optical sensors 164 optionally include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. One or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), one or more optical sensors 164 optionally capture static images and / or videos. In some embodiments, an optical sensor is located on the rear portion of the device 100 opposite to the touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to obtain an image of the user (e.g., for self-portraits, for video conferencing when the user is watching other video conference participants on the touch screen, etc.).

[0069] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor coupled to a force sensor controller 159 in the I / O subsystem 106 is shown. One or more contact force sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact force sensors 165 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the rear of the device 100 opposite to the touch-sensitive display system 112 located on the front of the device 100.

[0070] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0071] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1A A tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106 is shown. One or more tactile output generators 167 optionally include one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components for converting electrical signals into tactile outputs on the device). The tactile output generator 167 receives tactile feedback generation instructions from the tactile feedback module 133 and generates tactile outputs on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is arranged in juxtaposition or proximity to a touch-sensitive surface (e.g., a touch-sensitive display system 112), and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on a rear portion of the device 100 opposite the touch-sensitive display system 112 located on the front of the device 100.

[0072] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on an analysis of data received from the one or more accelerometers. The device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to the accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0073] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a tactile feedback module (or instruction set) 133, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, as shown in FIG. 1A and FIG. Figure 3 . The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state, including information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information about the device's position and / or posture.

[0074] The operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.

[0075] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices, or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a device compatible with some of Apple Inc. (Cupertino, California). iPod A multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used in iPod devices. In some embodiments, the external port is compatible with some of Apple Inc. (Cupertino, California) iPod A Lightning connector that is the same as, similar to, and / or compatible with the Lightning connector used in iPod devices.

[0076] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of the contact point being represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact or stylus contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, contact / motion module 130 and display controller 156 detect contact on a touch pad.

[0077] The contact / motion module 130 optionally detects gesture input from a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0078] In some embodiments, detecting a finger tap gesture (e.g., on a touch-sensitive display system 112) depends on the length of time between detecting a finger press event and a finger lift event, but is independent of the strength of the finger contact between detecting the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between the finger press event and the finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the strength of the finger contact during the tap reaches a given strength threshold (greater than a nominal contact detection strength threshold), such as a light press or deep press strength threshold. Thus, a finger tap gesture may satisfy an input criterion that is configured to be satisfied even when the characteristic strength of the contact does not satisfy a given strength threshold. For clarity, the finger contact in a tap gesture is typically required to satisfy a nominal contact detection strength threshold to detect a finger press event, below which the contact is not detected. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In the case where the device is configured to detect contact with a finger or stylus hovering over a touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0079] The same concepts apply in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, a zoom gesture, and / or a long press gesture are optionally detected (e.g., on a touch-sensitive display system 112) based on satisfying criteria that are independent of the intensity of the contacts included in the gesture. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on the movement of two or more contacts toward each other; a zoom gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of a contact on the touch-sensitive surface having less than a threshold amount of movement. Therefore, the statement that the gesture recognition criteria are configured such that the criteria are satisfied when the contacts in the gesture have an intensity below a corresponding intensity threshold means that the gesture recognition criteria can be satisfied even if the contacts in the gesture do not reach the corresponding intensity threshold. However, it should be understood that this statement does not exclude the gesture recognition criteria from being satisfied when one or more contacts in the gesture reach or exceed the corresponding intensity threshold. For example, a tap gesture is configured to be detected if a finger down event and a finger up event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding strength threshold during the predefined time period, and a swipe gesture is configured to be detected if the contact movement is greater than a predefined amount, even if the contact is above the corresponding strength threshold at the end of the contact movement.

[0080] In some cases, the contact intensity threshold, duration threshold, and movement threshold are combined in various combinations to create a heuristic algorithm to distinguish between two or more different gestures for the same input element or area, so that multiple different interactions with the same input element can provide a richer set of user interactions and responses. The statement that a particular set of gesture recognition criteria is configured such that the criteria are satisfied when the contact in the gesture has an intensity below the corresponding intensity threshold does not exclude the simultaneous evaluation of other intensity-related gesture recognition criteria to identify other gestures that do have criteria that the criteria are satisfied when the gesture includes a contact with an intensity above the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture (which is configured such that the criteria are satisfied when the gesture has an intensity below the corresponding intensity threshold) competes with a second gesture recognition criterion for a second gesture (which depends on the gesture reaching the corresponding intensity threshold). In such a competition, if the second gesture recognition criterion for the second gesture is satisfied first, the gesture is optionally not recognized as satisfying the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount of movement, a deep press gesture is detected instead of a swipe gesture. In contrast, if the contact moves a predefined amount of movement before the contact reaches the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in this case, the first gesture recognition criteria for the first gesture is still configured such that the criteria is satisfied when the contact in the gesture has an intensity below the corresponding intensity threshold, because if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with a contact that does not increase in intensity above the corresponding intensity threshold), the gesture would have been recognized as a swipe gesture by the first gesture recognition criteria. Therefore, a particular gesture recognition criterion that is configured such that the criteria is satisfied when the intensity of the contact remains below the corresponding intensity threshold will (A) in some cases ignore the intensity of the contact relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases still depend on the intensity of the contact relative to the intensity threshold in the following sense: if a competing set of intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-related gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input, then the particular gesture recognition criterion (e.g., for a long press gesture) will fail.

[0081] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other displays, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, without limitation, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0082] In some embodiments, the graphics module 132 stores data to be used to represent graphics. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application program, etc., and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data to be output to the display controller 156.

[0083] Haptic feedback module 133 includes various software components for generating instructions used by tactile output generator 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

[0084] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0085] The GPS module 135 determines the location of the device and provides such information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and maps / navigation widgets).

[0086] The application 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0087] Contacts module 137 (sometimes called address book or contact list);

[0088] Telephone module 138;

[0089] Video conferencing module 139;

[0090] Email client module 140;

[0091] Instant messaging (IM) module 141;

[0092] Fitness support module 142;

[0093] A camera module 143 for still images and / or video images;

[0094] Image management module 144;

[0095] Browser module 147;

[0096] Calendar module 148;

[0097] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, and a user-created widget 149-6;

[0098] A widget creator module 150 for forming a user-created widget 149-6;

[0099] Search module 151;

[0100] Video and music player module 152, optionally composed of a video player module and a music player module;

[0101] Notepad module 153;

[0102] Map module 154; and / or

[0103] Online video module 155.

[0104] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0105] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contacts module 137 includes executable instructions for managing an address book or contact list (e.g., stored in the application internal state 192 of the contacts module 137 in memory 102 or memory 370), including: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers and / or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140, or instant messaging 141; and the like.

[0106] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for entering a character sequence corresponding to a phone number, accessing one or more phone numbers in address book 137, modifying an entered phone number, dialing a corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communications optionally use any of a variety of communication standards, protocols, and technologies.

[0107] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0108] In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0109] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for performing the following operations: inputting a character sequence corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs) or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs or IMPS).

[0110] In combination with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 152, fitness support module 142 includes executable instructions for creating a workout (e.g., with time, distance, and / or calorie burn goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for a workout; and displaying, storing, and transmitting fitness data.

[0111] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0112] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, displaying (e.g., in a digital slide show or album), and storing still images and / or video images.

[0113] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) in accordance with user instructions.

[0114] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0115] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5), or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and, a JavaScript file (e.g., the Yahoo! desktop widget).

[0116] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget creator module 150 includes executable instructions for creating a desktop widget (e.g., transferring a user-specified portion of a web page to a desktop widget).

[0117] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0118] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple Inc.).

[0119] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. according to user instructions.

[0120] In combination with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with the maps (e.g., driving routes; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0121] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a specific online video.

[0122] Each module and application identified above corresponds to a set of executable instructions for performing one or more functions and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0123] In some embodiments, the device 100 is a device on which operation of a set of predefined functions is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

[0124] The set of predefined functions that are performed exclusively through the touch screen and / or touchpad optionally includes navigating between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, a home menu, or a root menu. In such embodiments, a "menu button" is implemented using a touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0125] Figure 1B 1 is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in the operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0126] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event distributor module 174. In some embodiments, the application 136-1 includes an application internal state 192, which indicates one or more current application views displayed on the touch-sensitive display system 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0127] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating information being displayed by the application 136-1 or information ready to be displayed by the application, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0128] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer 168, and / or microphone 113 (through audio circuit 110). The information received by peripherals interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0129] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0130] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0131] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0132] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0133] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0134] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, views higher in the hierarchy will still remain actively participating views.

[0135] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

[0136] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In other embodiments, event classifier 170 is a standalone module, or is part of another module (such as contact / motion module 130) stored in memory 102.

[0137] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, the corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more event recognizers in event recognizers 180 are part of an independent module, such as a user interface toolkit (not shown) or a higher level object from which application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views in application view 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0138] A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0139] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes a rotation of the device from one orientation to another orientation (e.g., from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0140] Event comparator 184 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition for event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift (touch end) of a predetermined duration. In another example, the definition for event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display system 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0141] In some embodiments, event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with the sub-event and the object that triggered the hit test.

[0142] In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays the delivery of the event information until it has been determined whether the sub-event sequence does or does not correspond to the event type of the event identifier.

[0143] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0144] In some embodiments, the corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0145] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag receives the flag and performs a predefined process.

[0146] In some embodiments, the event delivery instructions 188 include a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event series or to an actively participating view. The event handler associated with the sub-event series or with an actively participating view receives the event information and performs a predetermined process.

[0147] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0148] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0149] It should be understood that the above discussion of event processing for user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the multifunction device 100, and not all user input is initiated on the touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events defining an event to be recognized.

[0150] Figure 1C 1 is a block diagram illustrating a tactile output module according to some embodiments. In some embodiments, the I / O subsystem 106 (e.g., the tactile feedback controller 161 ( Figure 1A ) and / or other input controller 160 ( Figure 1A ))include Figure 1C At least some of the exemplary components shown. In some embodiments, the peripheral device interface 118 includes Figure 1C At least some of the example components shown.

[0151] In some embodiments, the tactile output module includes a tactile feedback module 133. In some embodiments, the tactile feedback module 133 aggregates and combines tactile outputs of user interface feedback from software applications on the electronic device (e.g., feedback in response to user input corresponding to the displayed user interface and prompts and other notifications indicating the execution of operations or the occurrence of events in the user interface of the electronic device). The tactile feedback module 133 includes one or more of a waveform module 123 (for providing a waveform for generating tactile output), a mixer 125 (for mixing waveforms, such as waveforms in different channels), a compressor 127 (for reducing or compressing the dynamic range of the waveform), a low-pass filter 129 (for filtering out high-frequency signal components in the waveform), and a thermal controller 131 (for adjusting the waveform according to thermal conditions). In some embodiments, the tactile feedback module 133 is included in a tactile feedback controller 161 ( Figure 1A In some embodiments, a separate unit of the tactile feedback module 133 (or a separate specific implementation of the tactile feedback module 133) is also included in the audio controller (e.g., the audio circuit 110, Figure 1A ) and used to generate an audio signal. In some embodiments, a single tactile feedback module 133 is used to generate an audio signal and to generate a waveform for a tactile output.

[0152] In some embodiments, the tactile feedback module 133 also includes a trigger module 121 (e.g., a software application, an operating system, or other software module that determines that a tactile output is to be generated and triggers a process for generating the corresponding tactile output). In some embodiments, the trigger module 121 generates a trigger signal for triggering (e.g., by the waveform module 123) the generation of a waveform. For example, the trigger module 121 generates a trigger signal based on a preset timing standard. In some embodiments, the trigger module 121 receives a trigger signal from outside the tactile feedback module 133 (e.g., in some embodiments, the tactile feedback module 133 receives a trigger signal from a hardware input processing module 146 located outside the tactile feedback module 133) and relays the trigger signal to other components within the tactile feedback module 133 (e.g., the waveform module 123) or a software application that triggers an operation (with the trigger module 121) based on the activation of a hardware input device (e.g., a main button). In some embodiments, the trigger module 121 also (e.g., from the tactile feedback module 133, Figure 1A and Figure 3 ) receives a tactile feedback generation instruction. In some embodiments, the trigger module 121 responds to the tactile feedback module 133 (or the trigger module 121 in the tactile feedback module 133) (e.g., from the tactile feedback module 133, Figure 1A and Figure 3 ) receives tactile feedback instructions and generates a trigger signal.

[0153] Waveform module 123 receives the trigger signal as input (e.g., from trigger module 121) and provides a waveform for generating one or more tactile outputs in response to receiving the trigger signal (e.g., a waveform selected from a predefined set of waveforms assigned for use by waveform module 123, such as those described below with reference to Figure 4F-4G waveform described in more detail).

[0154] The mixer 125 receives waveforms as input (e.g., from the waveform module 123) and mixes the waveforms together. For example, when the mixer 125 receives two or more waveforms (e.g., a first waveform in a first channel and a second waveform in a second channel that at least partially overlaps the first waveform), the mixer 125 outputs a combined waveform corresponding to the sum of the two or more waveforms. In some embodiments, the mixer 125 also modifies one or more of the two or more waveforms to emphasize a particular waveform relative to the rest of the two or more waveforms (e.g., by increasing the scale of the particular waveform and / or reducing the scale of the other waveforms in the waveforms). In some cases, the mixer 125 selects one or more waveforms to remove from the combined waveform (e.g., when waveforms from more than three sources have been requested to be output simultaneously by the tactile output generator 167, the waveform from the oldest source is discarded).

[0155] Mixer 127 receives waveforms (e.g., combined waveforms from mixer 125) as input and modifies these waveforms. In some embodiments, compressor 127 reduces these waveforms (e.g., based on tactile output generator 167 ( Figure 1A ) or 357( Figure 3 ) such that the tactile output corresponding to these waveforms is reduced. In some embodiments, compressor 127 limits the waveforms, such as by imposing a predefined maximum amplitude on the waveforms. For example, compressor 127 reduces the amplitude of the waveform portion that exceeds a predefined amplitude threshold, while maintaining the amplitude of the waveform portion that does not exceed the predefined amplitude threshold. In some embodiments, compressor 127 reduces the dynamic range of the waveform. In some embodiments, compressor 127 dynamically reduces the dynamic range of the waveform so that the combined waveform remains within the performance specifications of tactile output generator 167 (e.g., force and / or movable mass displacement limits).

[0156] Low-pass filter 129 receives a waveform (e.g., a compressed waveform from compressor 127) as input and filters (e.g., smoothes) the waveform (e.g., removes or reduces high-frequency signal components in the waveform). For example, in some cases, compressor 127 includes extraneous signals (e.g., high-frequency signal components) in the compressed waveform that interfere with tactile output generation and / or exceed the performance specifications of tactile output generator 167 when generating tactile output based on the compressed waveform. Low-pass filter 129 reduces or removes such extraneous signals in the waveform.

[0157] Thermal controller 131 receives a waveform (e.g., a filtered waveform from low pass filter 129) as input and adjusts the waveform based on a thermal condition of device 100 (e.g., based on an internal temperature detected within device 100, such as the temperature of haptic feedback controller 161, and / or an external temperature detected by device 100). For example, in some cases, the output of haptic feedback controller 161 varies based on temperature (e.g., in response to receiving the same waveform, haptic feedback controller 161 generates a first tactile output when haptic feedback controller 161 is at a first temperature and generates a second tactile output when haptic feedback controller 161 is at a second temperature different from the first temperature). For example, the magnitude (or amplitude) of the tactile output may vary based on temperature. To reduce the effects of temperature changes, the waveform is modified (e.g., the amplitude of the waveform is increased or decreased based on temperature).

[0158] In some embodiments, the tactile feedback module 133 (e.g., trigger module 121) is coupled to the hardware input processing module 146. In some embodiments, Figure 1A The other input controllers 160 in the embodiment include a hardware input processing module 146. In some embodiments, the hardware input processing module 146 receives input from a hardware input device 145 (e.g. Figure 1A In some embodiments, the hardware input device 145 is any input device described herein, such as a touch-sensitive display system 112 ( Figure 1A )、Keyboard / Mouse 350( Figure 3 )、Touchpad 355( Figure 3 ), one of the other input or control devices 116 ( Figure 1A ), or intensity-sensitive primary buttons (e.g. Figure 2B As shown or Figure 2C In some embodiments, the hardware input device 145 consists of a strength-sensitive primary button (e.g., Figure 2B as shown, or Figure 2C ) rather than being comprised of a touch-sensitive display system 112 ( Figure 1A )、Keyboard / Mouse 350( Figure 3 ) or TouchPad 355( Figure 3 ). In some embodiments, in response to input from the hardware input device 145, the hardware input processing module 146 provides one or more trigger signals to the tactile feedback module 133 to indicate that a user input that meets the predefined input criteria has been detected, such as an input corresponding to a primary button "click" (e.g., a "press click" or a "release click"). In some embodiments, the tactile feedback module 133 provides a waveform corresponding to a primary button "click" in response to the input corresponding to the primary button "click", thereby simulating the tactile feedback of pressing a physical primary button.

[0159] In some embodiments, the tactile output module includes a tactile feedback controller 161 (eg, Figure 1A 145 ) which controls the generation of tactile output. In some embodiments, the tactile feedback controller 161 is coupled to a plurality of tactile output generators, and selects one or more tactile output generators from the plurality of tactile output generators and sends a waveform to the selected one or more tactile output generators for generating tactile output. In some embodiments, the tactile feedback controller 161 coordinates a tactile output request corresponding to activation of the hardware input device 145 and a tactile output request corresponding to a software event (e.g., a tactile output request from the tactile feedback module 133), and modifies one or more of the two or more waveforms to emphasize a particular waveform relative to the remaining waveforms of the two or more waveforms (e.g., by increasing the size of the particular waveform and / or reducing the size of the remaining waveforms of these waveforms to prioritize the tactile output corresponding to the activation of the hardware input device 145 over the tactile output corresponding to the software event).

[0160] In some embodiments, such as Figure 1C As shown, the output of the tactile feedback controller 161 is coupled to the audio circuit of the device 100 (eg, the audio circuit 110, Figure 1A ), and provides the audio signal to the audio circuit of device 100. In some embodiments, tactile feedback controller 161 provides both a waveform for generating a tactile output and an audio signal for providing an audio output together with generating the tactile output. In some embodiments, tactile feedback controller 161 modifies the audio signal and / or the waveform (for generating the tactile output) so that the audio output and the tactile output are synchronized (e.g., by delaying the audio signal and / or the waveform). In some embodiments, tactile feedback controller 161 includes a digital-to-analog converter for converting a digital waveform into an analog signal, which is received by amplifier 163 and / or tactile output generator 167.

[0161] In some embodiments, the tactile output module includes an amplifier 163. In some embodiments, the amplifier 163 receives a waveform (e.g., from the tactile feedback controller 161), amplifies the waveform, and then sends the amplified waveform to the tactile output generator 167 (e.g., the tactile output generator 167 ( Figure 1A ) or 357( Figure 3 )). For example, amplifier 163 amplifies the received waveform to a signal level that meets the physical specifications of tactile output generator 167 (e.g., amplifies to the voltage and / or current required by tactile output generator 167 to generate a tactile output so that the signal sent to tactile output generator 167 generates a tactile output corresponding to the waveform received from tactile feedback controller 161) and sends the amplified waveform to tactile output generator 167. In response, tactile output generator 167 generates a tactile output (e.g., by moving the movable mass back and forth in one or more dimensions relative to the neutral position of the movable mass).

[0162] In some embodiments, the tactile output module includes a sensor 169 coupled to the tactile output generator 167. The sensor 169 detects the state or state change (e.g., mechanical position, physical displacement, and / or movement) of the tactile output generator 167 or one or more components of the tactile output generator 167 (e.g., one or more moving components, such as a membrane, for generating a tactile output). In some embodiments, the sensor 169 is a magnetic field sensor (e.g., a Hall effect sensor) or other displacement and / or motion sensor. In some embodiments, the sensor 169 provides information (e.g., the position, displacement, and / or movement of one or more components in the tactile output generator 167) to the tactile feedback controller 161, and, based on the information about the state of the tactile output generator 167 provided by the sensor 169, the tactile feedback controller 161 adjusts the waveform output from the tactile feedback controller 161 (e.g., the waveform optionally sent to the tactile output generator 167 via the amplifier 163).

[0163] Figure 2A A touch screen (e.g., touch-sensitive display system 112, Figure 1A) of a portable multifunction device 100. The touch screen optionally displays one or more graphics within a user interface (UI) 200. In these embodiments and in other embodiments described below, a user can select one or more of the graphics by, for example, making gestures on the graphics using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics will occur when the user breaks contact with the one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of fingers that have made contact with the device 100 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with a graphic will not select the graphic. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0164] The device 100 optionally also includes one or more physical buttons, such as a "home" button, or a menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display.

[0165] In some embodiments, the device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a main button 204), a push button 206 for powering the device on / off and for locking the device, a volume adjustment button 208, a user identity module (SIM) card slot 210, an earphone jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and keeping the button in a pressed state for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In some embodiments, the device 100 also accepts voice input for activating or deactivating certain functions through a microphone 113. The device 100 also optionally includes one or more contact strength sensors 165 for detecting the strength of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0166] Figure 2B-2C Shown suitable for Figure 1A , 2A, 3 and / or 4A are exploded views of a first input device (e.g., as a primary button 204) used in the electronic device shown in FIG. Figure 2B An example of an intensity-sensitive primary button with a capacitive sensor for determining a range of intensity values ​​corresponding to a force applied to the intensity-sensitive primary button is shown. Figure 2C An example of a primary button having a mechanical switch element is shown. Figure 2B , the input device stack 220 includes a cover element 222 and a trim 224. In the illustrated embodiment, the trim 224 completely surrounds the sides of the cover element 222 and the perimeter of the top surface of the cover element 222. Other embodiments are not limited to this configuration. For example, in one embodiment, the sides and / or top surface of the cover element 222 may be partially surrounded by the trim 224. Alternatively, the trim 224 may be omitted in other embodiments.

[0167] Both the cover element 222 and the rim 224 can be made of any suitable opaque, transparent and / or translucent material. For example, the cover element 222 can be formed of glass, plastic, or sapphire, and the rim 224 can be formed of metal or plastic. In some embodiments, one or more additional layers (not shown) can be located below the cover element 222. For example, when the cover element 222 is formed of a transparent material, an opaque ink layer can be disposed below the cover element 222. The opaque ink layer can hide other components in the input device stack 220 so that the other components cannot be seen through the transparent cover element.

[0168] A first circuit layer 226 may be disposed below the cover element 222. Any suitable circuit layer may be used. For example, the first circuit layer 226 may be a circuit board or a flexible circuit. The first circuit layer 226 may include one or more circuits, signal lines, and / or integrated circuits. In one embodiment, the first circuit layer 226 includes a biometric sensor 228. Any suitable type of biometric sensor may be used. For example, in one embodiment, the biometric sensor is a capacitive fingerprint sensor that captures at least one fingerprint when a user's finger approaches and / or contacts the cover element 222.

[0169] The first circuit layer 226 may be attached to the bottom surface of the cover element 222 by an adhesive layer 230. Any suitable adhesive may be used for the adhesive layer. For example, a pressure sensitive adhesive layer may be used as the adhesive layer 230.

[0170] The conformal layer 232 is disposed below the first circuit layer 226. In one embodiment, the conformal layer 232 includes an opening 234 formed in the conformal layer 232. When the device stack 220 is assembled, the opening 234 exposes the top surface of the first circuit layer 226 and / or the biometric sensor 228. In the illustrated embodiment, the conformal layer 232 is positioned around the inner perimeter of the trim 224 and / or around the peripheral edge of the cover element 222. Although illustrated in a circular shape, the conformal layer 232 can have any given shape and / or dimensions, such as a square or an oval. The conformal layer 232 is formed in Figure 2B and 2C 234, but other embodiments are not limited to this configuration. In some embodiments, multiple discrete compliant layers can be used in the device stack 220. In some other embodiments, the compliant layer 232 does not include the opening 234, and the compliant layer 232 extends across at least a portion of the input device stack 220. For example, the compliant layer 232 can extend across the bottom surface of the cover element 222, the bottom surface of the first circuit layer 226, or a portion of the bottom surface of the cover element 222 (e.g., around the peripheral edge of the cover element) and the bottom surface of the first circuit layer 226.

[0171] The second circuit layer 238 is positioned below the first circuit layer 226. Flexible circuits and circuit boards are examples of circuit layers that can be used in the second circuit layer 238. In some embodiments, the second circuit layer 238 can include a first circuit section 240 and a second circuit section 242. The first circuit section 240 and the second circuit section 242 can be electrically connected to each other.

[0172] First circuit section 240 may include a first set of one or more intensity sensor components included in an intensity sensor. In some embodiments, first circuit section 240 may be electrically connected to first circuit layer 226. For example, when first circuit layer 226 includes biometric sensor 228, biometric sensor 228 may be electrically connected to first circuit section 240 of second circuit layer 238.

[0173] The second circuit section 242 may include additional circuits, such as signal lines, circuit components, integrated circuits, and the like. In one embodiment, the second circuit section 242 may include a board-to-board connector 244 for electrically connecting the second circuit layer 238 to other circuits in the electronic device. For example, the second circuit layer 238 may be operably connected to a processing device using the board-to-board connector 244. In addition or alternatively, the second circuit layer 238 may be operably connected to a circuit that transmits a signal (e.g., a sensing signal) received from the intensity sensor component in the first circuit section 240 to a processing device. In addition or alternatively, the second circuit layer 238 may be operably connected to a circuit that provides a signal (e.g., a drive signal, a reference signal) to one or more of the intensity sensor components in the first circuit section 240.

[0174] In some implementations, the first circuit section 240 of the second circuit layer 238 can be attached to the bottom surface of the first circuit layer 226 using the adhesive layer 236. In a non-limiting example, the first circuit section 240 can be attached to the bottom surface of the first circuit layer 226 using a die attach film.

[0175] A third circuit layer 246 is disposed below the first circuit section 240 of the second circuit layer 238. The third circuit layer 246 may include a second set of one or more intensity sensor components included in the intensity sensor. The third circuit layer 246 is supported by and / or attached to a support element 248. In one embodiment, the support element 248 is attached to the trim 224 to create a housing for other components in the device stack 220. The support element 248 may be attached to the trim 224 using any suitable attachment mechanism.

[0176] The first set of one or more intensity sensor components in first circuit section 240 and the second set of one or more intensity sensor components in third circuit layer 246 together form an intensity sensor. The intensity sensor may use any suitable intensity sensing technology. Exemplary sensing technologies include, but are not limited to, capacitive, piezoelectric, piezoresistive, ultrasonic, and magnetic.

[0177] exist Figure 2B and 2C In the example shown, the force sensor is a capacitive force sensor. For a capacitive force sensor, the first set of one or more force sensor components may include a first set of one or more electrodes 250, and the second set of one or more force sensor components may include a second set of one or more electrodes 252. Figure 2B and 2CAlthough illustrated in a square shape, each electrode in the first group and the second group of one or more electrodes 250, 252 may have any given shape (e.g., rectangle, circle). In addition, the one or more electrodes in the first group 250 and the second group 252 may be arranged in any given pattern (e.g., one or more rows and one or more columns).

[0178] Figure 2B and 2C Two electrodes in the first group and the second group of one or more electrodes 250, 252 are shown. However, other embodiments are not limited to this configuration. The first group and the second group of one or more electrodes 250, 252 may each be a single electrode or a plurality of discrete electrodes. For example, if the first group of one or more electrodes is a single electrode, the second group of one or more electrodes includes a plurality of discrete electrodes. In some embodiments, the second group of one or more electrodes may be a single electrode and the first group includes a plurality of discrete electrodes. Alternatively, the first group and the second group of one or more electrodes may each include a plurality of discrete electrodes.

[0179] Each electrode in the first group of one or more electrodes 250 is aligned in at least one direction (e.g., vertically) with a corresponding electrode in the second group of one or more electrodes 252 to generate one or more capacitors. When a force input is applied to the cover element 222 (e.g., an input surface of an input device), at least one electrode in the first group 250 moves closer to a corresponding electrode in the second group 252, which changes the capacitance of the capacitor. The capacitance signal sensed from each capacitor represents a capacitance measurement of the capacitor. A processing device (not shown) is configured to receive the capacitance signal and relate the capacitance signal to the amount of strength applied to the cover element 222. In some embodiments, the force sensor can replace the switch element, and different intensity thresholds can be used to determine an activation event.

[0180] In some embodiments, such as Figure 2C In the illustrated embodiment, the switch element 254 may be located below the support element 248. The switch element 254 registers user input when the force input applied to the cover element 222 exceeds a given amount of force (e.g., a force threshold of the distance between the closed first circuit section 240 and the third circuit layer 246). Any appropriate switch element may be used. For example, the switch element 254 may be a dome switch that collapses when the force input applied to the cover element 222 exceeds the force threshold. When collapsed, the dome switch completes a circuit that is detected by the processing device and is identified as a user input (e.g., selecting an icon, function, or application). In one embodiment, the dome switch is arranged so that the vertex of the dome that can collapse is adjacent to the bottom surface of the support plate 248. In another embodiment, the base of the collapsible dome may be adjacent to the bottom surface of the support plate 248.

[0181] Figure 3 300 is a block diagram of an exemplary multi-function device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (for example, a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuits (sometimes called chipsets) that interconnect system components and control communications between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touch screen display. The I / O interface 330 also optionally includes a keyboard and / or a mouse (or other pointing device) 350 and a touch pad 355, a tactile output generator 357 for generating tactile output on the device 300 (for example, similar to the above reference). Figure 1A The one or more tactile output generators 167 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A The memory 370 may include one or more storage devices located away from the CPU 310. In some embodiments, the memory 370 stores information related to the portable multifunction device 100 ( Figure 1A ) stored in memory 102 of portable multifunction device 100, or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100 ( Figure 1A )'s memory 102 optionally does not store these modules.

[0182] Figure 3Each element in the above-mentioned identified element is optionally stored in one or more memory devices in the previously mentioned memory device. Each module in the above-mentioned identified module corresponds to a group of instructions for performing the above-mentioned functions. The above-mentioned identified module or program (that is, instruction set) need not be implemented as a separate software program, process or module, so the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores the subgroup of the above-mentioned modules and data structure. In addition, memory 370 optionally stores other modules and data structure not described above.

[0183] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

[0184] Figure 4A An exemplary user interface of an application menu on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0185] Signal strength indicators 402 for wireless communications (such as cellular signals and Wi-Fi signals);

[0186] Time 404;

[0187] Bluetooth indicator;

[0188] Battery status indicator 406;

[0189] A tray 408 with common application icons, such as:

[0190] o an icon 416 of the phone module 138 labeled “Phone” which optionally includes an indicator 414 of the number of missed calls or voice messages;

[0191] o an icon 418 of the email client module 140 labeled “Mail”, which optionally includes an indicator 410 of the number of unread emails;

[0192] o An icon 420 of the browser module 147 labeled “Browser”; and

[0193] o an icon 422 labeled “iPod” of a video and music player module 152 (also referred to as an iPod (trademark of Apple Inc.) module 152); and

[0194] Icons of other applications, such as:

[0195] o Icon 424 of IM module 141 labeled “Message”;

[0196] o Icon 426 labeled “Calendar” of calendar module 148;

[0197] o Icon 428 labeled “Photos” of the image management module 144;

[0198] o An icon 430 labeled “Camera” of the camera module 143;

[0199] o Icon 432 labeled “Online Video” of the online video module 155;

[0200] o Icon 434 labeled "Stock Market" of the Stock Market widget 149-2;

[0201] o an icon 436 labeled “MAP” of the map module 154;

[0202] o Icon 438 labeled "Weather" of the Weather widget 149-1;

[0203] o Icon 440 labeled as “Clock” of the alarm clock widget 149-4;

[0204] o an icon 442 labeled “Fitness Support” of the fitness support module 142;

[0205] o An icon 444 of the notepad module 153 labeled "Notepad"; and

[0206] o An icon 446 for a settings application or module that provides access to settings for the device 100 and its various applications 136.

[0207] It should be noted that Figure 4A The icon labels shown in are exemplary only. For example, in some embodiments, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0208] Figure 4B A touch-sensitive surface 451 (eg, Figure 3 A device (e.g., a tablet or touch pad 355) Figure 3Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357) for detecting intensity of contact on touch-sensitive surface 451 and / or one or more tactile output generators 359 for generating tactile output for a user of device 300.

[0209] Although many of the examples that follow will be given with reference to input on a touch screen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with touch-sensitive surface 451 at a location corresponding to a corresponding location on the display (e.g., Figure 4B 460 and 462) (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g., Figure 4B 450 in ) is separated, the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0210] In addition, although the following examples are given primarily with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures, etc.), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., stylus input). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple finger contacts, or a combination of finger contacts and stylus inputs, are used simultaneously.

[0211] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface adjacent to the contact and / or its change, and / or the resistance of the touch-sensitive surface adjacent to the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is measured in pressure units). The intensity of the contact is used as an attribute of the user input, thereby allowing the user to access additional device functions that the user cannot easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical controls / mechanical controls such as knobs or buttons).

[0212] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, the user of the device is provided with a software setting for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0213] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples sampled during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact intensity increases and / or before or after contact intensity decreases). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half-maximum value of the contact intensity, the 90% maximum value of the contact intensity, the value generated by low-pass filtering the contact intensity over a predefined time period or starting from a predefined time, etc. In some embodiments, the duration of the contact is used when determining the feature strength (e.g., when the feature strength is the average of the strength of the contact over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this example, a contact whose feature strength does not exceed the first threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the second threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more strength thresholds is used to determine whether to perform one or more operations (e.g., whether to execute the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0214] In some embodiments, a portion of a gesture is identified for determining a characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an end position (e.g., a drag gesture), where the strength of the contact increases. In this embodiment, the characteristic strength of the contact at the end position may be based only on a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only a portion of the swipe contact at the end position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the strength of the swipe contact for determining the characteristic strength.

[0215] The user interface diagrams described herein optionally include various intensity diagrams that illustrate the intensity of contacts on the touch-sensitive surface relative to one or more intensity thresholds (e.g., contact detection intensity threshold IT0, light press intensity threshold IT1, and light touch intensity threshold IT2). L , Deep compression intensity threshold IT D (For example, it is initially at least higher than I L ) and / or one or more other intensity thresholds (e.g., ratio I L Lower intensity threshold I H )). This intensity graph is not typically part of the displayed user interface, but is provided to help explain the graph. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, when a contact having a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold IT0, contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing operations associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface illustrations.

[0216] In some embodiments, the response of the device to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, the strength of the contact exceeding the first strength threshold during the input triggers a first response. In some embodiments, the response of the device to the input detected by the device depends on a criterion including both the contact strength during the input and a time-based criterion. For example, for some "deep press" inputs, as long as a delay time passes between satisfying the first strength threshold and satisfying the second strength threshold, the strength of the contact exceeding the second strength threshold greater than the first strength threshold of the light press during the input triggers a second response. The duration of the delay time is typically less than 200ms (milliseconds) (e.g., 40ms, 100ms, or 120ms, depending on the magnitude of the second strength threshold, where the delay time increases as the second strength threshold increases). The delay time helps avoid accidentally identifying a deep press input. As another example, for some "deep press" inputs, a period of reduced sensitivity will occur after reaching the first strength threshold. During this period of reduced sensitivity, the second strength threshold increases. This temporary increase in the second intensity threshold also helps avoid accidental deep press inputs.For other deep press inputs, the response to detecting the deep press input does not depend on time-based criteria.

[0217] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Exemplary factors are described in U.S. patent application serial numbers 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety.

[0218] For example, Figure 4C A dynamic intensity threshold 480 is shown that changes over time based in part on the intensity of touch input 476 over time. Dynamic intensity threshold 480 is the sum of two components: a first component 474 that decays over time after a predefined delay time p1 from when touch input 476 is initially detected, and a second component 478 that tracks the intensity of touch input 476 over time. The initial high intensity threshold of first component 474 reduces accidental triggering of a “deep press” response while still allowing an immediate “deep press” response if touch input 476 provides sufficient intensity. Second component 478 reduces unintentional triggering of a “deep press” response through gradual intensity fluctuations of the touch input. In some embodiments, when touch input 476 satisfies dynamic intensity threshold 480 (e.g., at Figure 4C ), triggering a "deep press" response.

[0219] Figure 4D Another dynamic intensity threshold 486 (eg, intensity threshold I D ). Figure 4D Two other intensity thresholds are also shown: a first intensity threshold I H and the second intensity threshold I L .exist Figure 4D Although the touch input 484 satisfies the first intensity threshold I before time p2, H and the second intensity threshold I L , but no response is provided until a delay time p2 has elapsed at time 482. Similarly, Figure 4D , the dynamic intensity threshold 486 decays over time, wherein the decay is from time 482 (triggering the second intensity threshold I L This type of dynamic intensity threshold reduction begins at time 488 after a predefined delay time p1 has elapsed when the trigger is associated with a lower threshold intensity (such as the first intensity threshold I H Or the second intensity threshold I L ) is triggered after or simultaneously with the response associated with the dynamic intensity threshold I D The associated response.

[0220] Figure 4E Another dynamic intensity threshold 492 (eg, intensity threshold I D ).exist Figure 4E In the example, after a delay time p2 has passed since the touch input 490 was initially detected, the trigger is triggered with the intensity threshold I L At the same time, the dynamic intensity threshold 492 decays after a predefined delay time p1 has passed since the touch input 490 was initially detected. L The associated response then decreases the intensity of the touch input 490, and then increases the intensity of the touch input 490 without releasing the touch input 490, which may trigger the touch input 490 to be in accordance with the intensity threshold I. D The associated response (e.g., at time 494) may be performed even when the intensity of touch input 490 is below another intensity threshold (e.g., intensity threshold I L ) is also the case.

[0221] The contact characteristic strength is below the light press strength threshold IT L The intensity increases to between the light pressure intensity threshold IT L and deep compression intensity threshold IT D The intensity between IT and the deep press intensity threshold is sometimes referred to as a "light press" input. D The intensity of the deep compression increases to above the deep compression intensity threshold ITD The intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. L A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact on the touch surface. A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact lifted off the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some illustrations, shaded circles or ellipses are used to represent the intensity of a contact on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0222] In some embodiments, Figure 4C-4E The dynamic intensity thresholds shown are used for input on the touch-sensitive display system 112. In some embodiments, for an intensity-sensitive input device (such as Figure 2B and 2C The input on the intensity-sensitive primary button shown in the figure uses different standards (such as reference Figure 5A-5II In some embodiments, reference Figure 5A-5II The stated criteria are used for all inputs, rather than Figure 4C-4E Dynamic intensity thresholds shown.

[0223] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the strength of the contact (or multiple contacts) increases to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting that the strength of the corresponding contact increases to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and the strength of the contact subsequently decreases to below the press input strength threshold, and the corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases to below the press input threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0224] In some embodiments, the device employs intensity hysteresis to avoid unexpected input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units less than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the strength of the contact increases from an strength equal to or below a hysteresis strength threshold to an strength equal to or above a press input strength threshold and, optionally, the strength of the contact subsequently decreases to an strength equal to or below the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (for example, depending on the circumstances, the strength of the contact increases or the strength of the contact decreases).

[0225] For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input are optionally triggered in response to detecting that the intensity of the contact increases above the press input intensity threshold, the intensity of the contact increases from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreases below the press input intensity threshold, or the intensity of the contact decreases below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting that the intensity of the contact decreases below the press input intensity threshold, the operations are optionally performed in response to detecting that the intensity of the contact decreases below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, triggering of these operations is also dependent on satisfying a time-based criterion (e.g., a delay time has elapsed between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0226] Although in Figure 4F For illustrative purposes, only specific frequencies, amplitudes, and waveforms are shown in the sample tactile output patterns, but tactile output patterns with other frequencies, amplitudes, and waveforms can also be used for similar purposes. For example, a waveform with between 0.5 and 4 cycles can be used. Other frequencies in the range of 60 Hz-400 Hz can also be used.

[0227] User interface and associated processes

[0228] Attention now turns to embodiments of a user interface ("UI") and associated processes that may be implemented on an electronic device such as portable multifunction device 100 or device 300 having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface.

[0229] Figure 5A-5II An exemplary user interface and multiple timeout periods and intensity thresholds for detecting gestures are shown. According to some embodiments, some of the intensity thresholds and timeout periods are based on previous input intensities (e.g., during a gesture input, some intensity thresholds are based on characteristic or representative intensities of the input during the same gesture). The user interfaces in these figures are used to illustrate the processes described below, including Figures 6A-6F , 7A-7E, and 8A-8C. For ease of explanation, some embodiments in the embodiments will be discussed with reference to operations performed on a device with touch-sensitive display system 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representative point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on touch-sensitive display system 112. However, in response to detecting a contact on touch-sensitive surface 451 when the user interface shown in the figures is displayed on display 450 together with the focus selector, similar operations are optionally performed on a device with display 450 and a separate touch-sensitive surface 451.

[0230] Figures 5A-5N A user interface, user input intensity, and corresponding intensity thresholds are shown for distinguishing multiple gestures, such as single-tap, double-tap, and long-press gestures. Figures 5A-5C A single-click gesture is shown on a primary button 204 of an electronic device that also includes a touch-sensitive display 112. In some embodiments, the primary button 204 is separate from the display and optionally includes a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display. In some embodiments, the primary button 204 is a virtual primary button displayed on the display (e.g., having a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of input using the virtual primary button). The primary button 204 is associated with an intensity sensor for measuring the intensity of user input on the primary button. Figure 5A The electronic device 100, the display 112 and the main button 204 are shown, and the first pressing click intensity threshold I is also shown. D .like Figure 5B As shown, touch input 505 on primary button 204 has an intensity that varies over time, including a first intensity increase 520 (sometimes referred to as a first press click) reaching a peak intensity IPeak, , the peak intensity is higher than the first press click intensity threshold I D (For example, because the user will usually exceed the press-click intensity threshold when performing a press-click operation). Therefore, the electronic device or its module (such as the contact / motion module 130, Figure 1A ) detects that the strength increase satisfies the press-click detection criterion, which requires the input strength to increase to above the first press-click strength threshold I D So that the press-click detection criteria are met. In some embodiments, when the press-click intensity threshold is reached, the device provides feedback (e.g., auditory and / or tactile feedback) indicating that the press-click intensity threshold has been reached.

[0231] like Figure 5C As shown, after the first intensity increase, the intensity of the touch input on the main button 204 decreases, and the electronic device detects a decrease in the first intensity of the contact 522, sometimes referred to as a release click. Figure 5B-5C In the example shown, the first intensity reduction of the input satisfies the release click detection criteria, which requires the intensity of the input to decrease below the first release click intensity threshold I U So that the release click detection criteria are met. As will be explained in more detail below, the first release click intensity threshold I U The selection is based on the intensity of the input during the first intensity increase 520 of the contact. Since the first intensity decrease 522 of the contact meets the release click detection criteria, the electronic device 100 provides the first feedback. Figure 5B-5C In the example shown, the first feedback is or includes switching from displaying a user interface of a first application (e.g., a timer application) to displaying an application launch user interface. In some embodiments, Figure 5B-5C The user interface transition shown is achieved by closing an application (e.g., a timer application) or stopping displaying the user interface of an application in response to a single-click gesture on the main button 204, the single-click gesture being detected when a first intensity reduction of the input satisfies a release click detection criterion. In some embodiments, when the release click intensity threshold is reached, the device provides feedback (e.g., auditory and / or tactile feedback) indicating that the release click intensity threshold has been reached. In some embodiments, the tactile output for the release click is different from the tactile output for the press click (e.g., the tactile output for the release click has a reduced amplitude relative to the press click tactile output, such as a MicroTap (270 Hz) tactile output pattern with a gain of 0.5 relative to a MicroTap (270 Hz) tactile output pattern with a gain of 1.0 for the press click).

[0232] Compared to Figure 5B-5C Represents a touch-based input gesture, Figure 5B and 5FThe gesture based on the touch input corresponds to a first intensity decrease of the input that does not meet the release click detection criteria (e.g., does not decrease below the first release click intensity threshold I U )'s touch input, so the electronic device abandons providing the first feedback (e.g., visual, auditory and / or tactile feedback).

[0233] Figure 5D-5E The second single click input is shown. Figure 5D As shown, in the first touch input 505 ( Figure 5C The second touch input 507 detected on the primary button 204 after the end of the touch input 507 has an intensity that changes over time, including an intensity increase 524 to a peak intensity I Peak, , the peak intensity is higher than the first press click intensity threshold I D Therefore, the electronic device or its module (such as the touch / motion module 130, Figure 1A ) detects that the intensity increase 524 satisfies the press-click detection criteria. In some embodiments, when the press-click intensity threshold is reached, the device provides feedback (e.g., auditory and / or tactile feedback) indicating that the press-click intensity threshold has been reached.

[0234] like Figure 5E As shown, after the intensity of the second input 507 increases 524, the intensity of the touch input on the main button 204 decreases, and the electronic device detects a decrease in the intensity of the input 526. Figure 5D-5E In the example shown, the decrease in input strength satisfies the release click detection criteria, which requires the input strength to decrease below a first release click strength threshold I U So that the release click detection criterion is met. Since the intensity of the input decreases to meet the release click detection criterion, the electronic device 100 provides feedback. Figure 5D-5E In the example shown, the feedback is or includes launching a user interface (such as Figure 5D In some embodiments, when the release click strength threshold is reached, the device provides feedback (e.g., auditory and / or tactile feedback) to indicate that the release click strength threshold has been reached.

[0235] exist Figure 5C , 5D5E, indicator 510 indicates the time point when the press click detection criteria are met, indicator 512 indicates the time point when the release click detection criteria are met, indicator 514 indicates the time point when the press click detection criteria are met for the second time, and indicator 516 indicates the time point when the release click detection criteria are met for the second time. In some embodiments, the electronic device or its module (e.g., an application independent module, such as contact / motion module 130, Figure 1A ) generates an event (e.g., a press-click event) in response to the press-click detection criteria being met, or generates an event (e.g., a release-click event) in response to the press-click detection criteria being met, or both. In some embodiments, the event (e.g., a press-click event or a release-click event) is delivered to one or more targets, such as an application, or a web page for processing by instructions in the web page, or is delivered to a web browser (which is a special case of an application) and / or a tactile feedback module, such as module 133, Figure 1A For example, Figure 5C As shown, in some embodiments, the tactile output 502 (e.g., a tactile output having a MicroTap (270 Hz) tactile output mode, Figure 4F ) is generated in conjunction with the electronic device detecting that the intensity of the input is reduced to meet the release click criteria (sometimes referred to as detecting a release click or detecting a single click input). In addition, reference Figure 5B , 5BB , 5DD, 5FF, 5HH and 5II, in some embodiments, tactile output 502 or 504 is generated in conjunction with the electronic device detecting that the intensity of the input increases to meet the press-click criterion (sometimes referred to as detecting a press-click or long press), for example, as described above with reference to Figures 5Z to 5II discussed.

[0236] Figure 5G-5I Two examples of double-click input and providing corresponding feedback are shown. In particular, Figure 5G The first intensity increase 532 reaches a peak intensity I Peak , the peak intensity of the touch input is higher than the first press click intensity threshold I D When reaching the peak intensity I Peak , after which the touch input includes a first intensity of the input reduced 534 to a low intensity I Valley , the low intensity is lower than the first release click intensity threshold I U So that the release click detection standard is met. As mentioned above, the first release click intensity threshold I U The touch input includes a second intensity increase 536 to a level above the second press click intensity threshold I after the first intensity decrease 534. D2As will be explained in more detail below, the second press click intensity threshold I D2 The second press click intensity threshold I is determined based on the intensity of the input during the first intensity reduction of the contact. D2 Based on the minimum or lowest intensity I input during the first intensity reduction of the contact Valley to be sure.

[0237] In some embodiments, Figure 5G The input shown includes a first intensity increase 532 indicated by indicator 510 that satisfies a down-click detection criterion, followed by a first intensity decrease 534 indicated by indicator 512 that satisfies a release-click detection criterion, followed by a second intensity increase 536 indicated by indicator 518 that satisfies the down-click detection criterion, wherein the down-click detection criterion includes a first down-click intensity threshold for the first intensity increase and includes a second down-click intensity threshold for the second intensity increase.

[0238] In some embodiments, Figure 5G Towards Fig.5I The change is indicated by the following: when the electronic device 101 determines Figure 5G When the first intensity increase 532, the first intensity decrease 534, and the second intensity increase 536 of the input shown have satisfied the pressed click detection criteria, the released click detection criteria, and the pressed click detection criteria, respectively, the electronic device generates second feedback (e.g., visual, auditory, and / or tactile feedback) indicating that the second intensity increase is recognized as part of the double-click input, such as displaying Figure 5H In some embodiments, the second feedback is in Figure 5G and 5I The second feedback is generated or triggered at or immediately after the time indicated by the indicator 518 in the middle, at which time the second intensity increase satisfies the pressed click detection criteria. It should be noted that in some embodiments or in some cases, "one and a half clicks" (e.g., an input including a first pressed click, a first released click, and a second pressed click in sequence) is treated as a double click, which triggers the execution of an action, such as generating a second feedback. In some embodiments, the second feedback is or includes a transition to a multitasking user interface, such as Fig.5I In some embodiments, the second feedback is or includes generating a tactile output 503, such as Figure 5H and 5I In some embodiments, the tactile output 503 is a tactile output mode having a MiniTap (270 Hz) Figure 4F )’s tactile output.

[0239] In some embodiments, by Figure 5G Towards Figure 5H The transformation of Figure 5G The input shown continues to drop below the release click intensity threshold I U The second intensity of 538 is reduced, so that the release click detection criteria is met at the time indicated by indicator 522. Figure 5H The same release click intensity threshold I is shown for the first and second intensity reductions U , but in some embodiments or in some cases, the release click intensity threshold for the second release click (e.g., the second intensity decreases) is different from the release click intensity threshold for the first release click (e.g., the first intensity decreases). It should be noted that in some embodiments or in some cases, "two complete clicks" (e.g., inputs including the first press click 532, the first release click 534, the second press click 536, and the second release click 538 in sequence) are treated as double-clicks, which triggers the execution of an action, such as generating a second feedback (e.g., visual, auditory, and / or tactile feedback). In some embodiments, the second feedback is a transition to a multitasking user interface, such as Figure 5H shown.

[0240] Figure 5J Show the release click intensity threshold I in graphical form D (See Figure 5C , and 5E-5I) and the ratio of the strength value representing the contact strength changes based on the strength value representing the contact strength. As shown in the figure, the ratio is greater than the first press click strength threshold I D Big (or tall) I D1 It has a maximum value a2 when the input intensity is greater than I D1 Big (or tall) I D2 The ratio has a value between 0 and 1. In some embodiments, the maximum value a2 of the ratio is equal to 0.73, and the minimum value a1 of the ratio is equal to 0.6. In one example, when the current strength of the contact after low pass filtering (as described below with reference to Figure 5K Discussion) is 500g, the ratio is 0.6, and when the low-pass filtered current intensity of the contact is 300g, the ratio is 0.73.

[0241] More generally, when the release click strength is based on a first strength value representing contact strength, the ratio of the release click strength threshold to the first strength value has a first value; when the release click strength is based on a second strength value representing contact strength that is greater than the first strength value, the ratio of the release click strength threshold to the second strength value has a second value that is different from the first value (e.g., smaller or larger than the first value).

[0242] In some embodiments, Figure 5JThe ratio shown is applied to (e.g., multiplied by) a characteristic intensity of the input (e.g., the peak intensity of the input during the first intensity increase, or the intensity value obtained by low-pass filtering the intensity during the first intensity decrease) to determine the release click intensity threshold I U The multiplier of .

[0243] Figure 5K The release click intensity threshold I is shown to dynamically change as the intensity of the input changes during the first intensity decrease 542 after the electronic device has detected the first intensity increase 540. U(t) More specifically, during the first intensity reduction 542, the input intensity is low-pass filtered to generate a first time-varying value I LPup 544. The first time-varying value is then multiplied by a fixed value such as 0.7 or a strength-based value such as Figure 5J The ratio shown is used to generate a time-varying release click intensity threshold I U(t) , where the “(t)” symbol indicates that the value is time-varying. When the input intensity matches or decreases below the time-varying release click intensity threshold I U(t) , as shown by indicator 548, the release click detection criteria are met.

[0244] Figure 5L 540 and 542 after the electronic device has detected a first intensity increase 540 and a first intensity decrease 542, during a second intensity increase 550, the press-click intensity threshold I dynamically changes as the intensity of the input changes. D(t) More specifically, during the second intensity increase 550, the input intensity is low-pass filtered to generate a second time-varying value I LPdown 552. In some embodiments, the low pass filtered intensity (I LPdown 552) is initially set at the beginning of the second intensity increase 550 of the contact to the lowest intensity I input during the first intensity decrease 534 of the contact. Valley .

[0245] The second time-varying value I LPdown 552 is then multiplied by a fixed value such as 1.4, (or alternatively, divided by a fixed value such as 0.7) or a strength-based value such as Figure 5J The ratio shown is used to generate a time-varying release click intensity threshold I U(t) 554, where the "(t)" symbol indicates that the value is time-varying. When the strength of the input matches or increases above the time-varying press-click strength threshold I D(t) At 554 , as indicated by indicator 556 , the press click detection criteria are met.

[0246] Figure 5M5 shows a release click intensity threshold I that changes dynamically as the intensity of the input changes during a second intensity decrease 558 (sometimes referred to as a second release click) after the electronic device has detected a second intensity increase 550. U2(t) More specifically, during the second intensity reduction 558, the input intensity is low-pass filtered to generate a second time-varying value I LP2up 560. The second time-varying value is then multiplied by a fixed value such as 0.7 or an intensity-based value such as Figure 5J The ratio shown is used to generate a time-varying release click intensity threshold I U2(t) 561, where the "(t)" symbol indicates that the value is time-varying. When the strength of the input matches or decreases below the time-varying release click strength threshold I U2(t) 561 , as shown by indicator 562 , the release click detection criteria are met.

[0247] Figure 5N and Figure 5M Similar, except that the minimum release click strength threshold I UM Applied to Figure 5M The time-varying release click intensity threshold I U2(t) 561, thereby generating a modified time-varying release click intensity threshold I U2M(t) 563. In other words, the time-varying release click intensity threshold I U2M(t) 563 is equal to the time-varying release click intensity threshold I at each time point U2(t) 561 and minimum release click strength threshold I UM The larger of .

[0248] Figure 5O-5Y A user interface, user input intensity, and corresponding intensity thresholds are shown for distinguishing between a single-click input or gesture and a double-click input or gesture, wherein recognition of a single-click gesture is accelerated or expedited (e.g., based on a determination that a double-click input will not be performed). Figure 5O-5P A single tap gesture, sometimes referred to as a single tap or single tap gesture, is shown on a primary button 204 of an electronic device that also includes a touch-sensitive display 112. Primary button 204 includes an intensity sensor for measuring the intensity of a user input on the primary button. Fig.5O The electronic device 100, the display 112 and the main button 204 are shown, and the first pressing click intensity threshold I is also shown. D , first release click intensity threshold I U , and the early confirmation threshold I A , sometimes called the accelerated confirmation threshold. Fig.5O As shown, touch input 523 on primary button 204 has an intensity that varies over time, including a first intensity increase 532 (sometimes referred to as a first press click) reaching a peak intensity I Peak,, the peak intensity is higher than the first press click intensity threshold I D Therefore, the electronic device or its module (such as the touch / motion module 130, Figure 1A ) detects that the strength increase satisfies the press-click detection criteria, as indicated by indicator 510 at time T1, which requires the strength of the input to increase above the first press-click strength threshold I D so that the pressed click detection criteria are satisfied.

[0249] After the first intensity increase 532, the intensity of the touch input 523 on the primary button 204 decreases, and the electronic device detects a first intensity decrease 534 of the contact, sometimes referred to as a first release click. Fig.5O In the example shown, the first intensity reduction of the input satisfies the release click detection criteria, such as at time T 2a As indicated by indicator 512, the input satisfies the release click detection criteria requiring the input strength to decrease below the first release click strength threshold I U If the first intensity increase satisfies the press-click detection criteria and the first intensity decrease satisfies the release-click detection criteria, the device identifies at least a portion of the input intensity change as a first event, such as a click event, sometimes referred to as a first click.

[0250] If the current context of the electronic device 100 allows both single clicks and double clicks on the primary button 204, the execution of the first operation associated with identifying the single click is delayed until the device determines that the user is not inputting a double click, or equivalently, that the first click is not the first part of a double click. In some embodiments, the delay in the execution of the first operation is limited to a default delay time, such as 300ms or 500ms. However, if the electronic device can determine before the default delay time expires that the input meets the early confirmation criteria, indicating that the input will not be a double click, the electronic device can cause the execution of the first operation as soon as that determination is made.

[0251] In some embodiments, the early confirmation criterion requires that the intensity of the input remain below the confirmation intensity threshold I during the first intensity reduction period. A For example, in some embodiments, the confirmation intensity threshold I A is 100g, and the release click detection threshold I U is 150g or 200g or more, so confirm the strength threshold I A Below the release click detection threshold.

[0252] In some embodiments, the intensity threshold I is determined A According to the peak characteristic intensity I of the input detected during the detected increase in the intensity of the input before the decrease in the intensity of the input on the input element is detectedPeak For example, in some embodiments, the release click detection threshold I U According to the peak characteristic intensity I of the input detected during the detected increase in the intensity of the input before the decrease in the intensity of the input on the input element is detected Peak To determine and confirm the intensity threshold I A According to the release click detection threshold I U In some such embodiments, the intensity threshold I is determined. A is set to be lower than the release click detection threshold I U The level of a predefined amount of less than 50 g, for example, and in other such embodiments, the intensity threshold I is confirmed. A Set to release click detection threshold I U In some embodiments, the release click threshold is the predefined multiple or percentage (e.g., 0.90 times or 90%) of Figures 5A-5N The dynamically determined release click threshold.

[0253] Optionally, a tactile output 502 (e.g., a tactile output having a MicroTap (270 Hz) tactile output mode, Figure 4F ) is generated in conjunction with the electronic device detecting that the intensity of the input increases to meet the press-click criteria (sometimes referred to as detecting or identifying a press-click) or in conjunction with the electronic device detecting that the intensity of the input decreases to meet the release-click criteria (sometimes referred to as detecting or identifying a release-click or single-click input). In some embodiments, the tactile output of the release-click is different from the tactile output of the press-click (for example, the tactile output of the release-click has a reduced amplitude relative to the press-click tactile output, such as a MicroTap (270Hz) tactile output pattern with a gain of 0.5 relative to a MicroTap (270Hz) tactile output pattern with a gain of 1.0 for the press-click).

[0254] exist Figure 5P middle, Fig.5O The first decrease 534 of the input is shown to continue, crossing the confirmation strength threshold I at time T2. A , as indicated by indicator 564. However, in this example, the input ends at time T offThe finger or stylus is lifted off of the touchscreen, thereby ending the gesture. In some embodiments, terminating the input by lifting off of the contact during the first intensity reduction is treated as a confirmation gesture is a single click (or equivalently, a single tap). Therefore, the electronic device performs a first operation, which in this example includes ending display of the user interface of the application displayed before the single tap was received, so that the display 112 of the electronic device changes from displaying the user interface of the first application (e.g., a time application) to displaying a set of application launch icons in the application launch user interface.

[0255] exist Figure 5Q In the example, on the main button 204 (e.g., on the touch-sensitive display 112, a Fig.5O The touch input 525 detected during the application user interface (similar to the user interface shown in FIG. 1 ) has an intensity that varies over time, including a first intensity increase 532 (sometimes referred to as a first press click) followed by a first intensity decrease 534. Indicator 512 indicates the point in time when the release click detection criteria are met, and indicator 564 indicates that the first intensity decrease reaches the confirmation intensity threshold I. A At the time point. Figure 5Q In the example shown, when the confirmation strength threshold I is reached A After that, the input remains below the confirmation strength threshold I A The intensity of the input decreases from time T2 to a time period that lasts at least the confirmation time threshold. The timeout period, called the "fast timeout period", starts at time T2 and is determined when the intensity of the input decreases below the confirmation threshold I A When the duration of the fast timeout period reaches the confirmation time threshold, at time T3, the input is confirmed as a single click, as indicated by indicator 565. Since the fast timeout period reaches the confirmation time threshold, at Figure 5Q In the example of FIG. 1 , at time T3, a first operation (e.g., an operation associated with recognizing a single tap) is performed, or execution of the first operation is triggered. As described above, in these examples, the first operation includes ending the user interface of the application that was displayed before the single tap was received (e.g., Fig.5O The display 112 of the electronic device changes from displaying the user interface of the first application (eg, the time application) to displaying a group of application launch icons in the application launch user interface.

[0256] Figure 5R Show Figure 5Q, a continuation of the input shown in . In this example, after confirming (e.g., at time T3) that the initial input is a single tap input and performing or causing the first operation to be performed, a second single tap is received by the electronic device. In particular, the second single tap includes a second intensity increase 566 (as indicated by indicator 567) that satisfies the press-click detection criteria, and a second intensity decrease 568 (as indicated by indicator 569) that satisfies the release-click detection criteria. In this example, when the second single tap is detected, for example, when the second intensity decrease 568 satisfies the release-click detection criteria, another operation is performed. Figure 5Q and 5R In the example shown, the operation includes launching an icon screen (such as Figure 5D Scroll to another icon screen of the application launch user interface (as shown in Figure 5R As shown) (including a second group of application launch icons, the second group of application launch icons including application launch icons that are not in the first group of application launch icons).

[0257] It should be pointed out that in Figure 5R In the example shown, during the second intensity reduction 568, the intensity also drops below the confirmation intensity threshold I A , as indicated by indicator 570. However, unless the electronic device is configured to detect single, double, and triple-click inputs, the intensity has dropped below the confirmation intensity threshold I during the second intensity reduction period. A Any additional actions of the electronic device may not be triggered. For further considerations on detecting multi-tap gestures, see the following discussion of Figure 5Y Discussion.

[0258] exist Figure 5S , the touch input 527 on the primary button 204 has an intensity that varies over time, including a first intensity increase 532 and a second intensity increase 533 at time T 2A Decrease below the release click intensity threshold I U The first intensity of the click is reduced 534, as indicated by indicator 512. When the intensity is reduced to below the release click intensity threshold I U (This can be considered the first event) After that, the intensity of the touch input 527 remains below the release click intensity threshold I U But above the confirmation strength threshold I AExecution of the first operation in response to the first event is delayed until a default delay time period has passed, for example at time T4, as indicated by indicator 571. As described above, in these examples, the first operation includes ending display of a user interface of an application that was displayed before the single tap was received, so that the display 112 of the electronic device transitions from displaying a user interface of a first application (e.g., a time application) to displaying a set of application launch icons in an application launch user interface.

[0259] In some embodiments, the delay time is the time from when the first press click is detected (eg Figure 5S In some other embodiments, the delay time is the time from when the first release click is detected (e.g., Figure 5S Medium time T 2A In some embodiments, if the intensity of the input increases to above the release click intensity threshold I before the delay time reaches the default delay period, U , the measurement of the delay time stops and the measured delay time is reset to zero. The default delay period is usually significantly longer than the early confirmation time threshold. For example, in some embodiments, the default delay period is a value between 300ms and 500ms, while the early confirmation time threshold is a value between 100ms and 200ms.

[0260] like Figure 5T , 5U and 5V, in some embodiments, the early confirmation criterion is when the input strength is below the early confirmation threshold I A The criterion is met when the cumulative amount of time reaches the early confirmation time threshold. Figure 5T In the example shown, after the first intensity increase 532 and the first intensity decrease 534, the intensity of the input decreases to below the release click intensity threshold I at time T2. U and early confirmation threshold I A , as indicated by indicator 568, but then at time T 2-1 Increases above the early confirmation threshold I A The value does not meet the early confirmation criteria. Figure 5U As shown, the input intensity at time T 2-2 Reduced again to below the early confirmation threshold I A , as indicated by indicator 572, and then as Figure 5V As shown, it remains below the early confirmation threshold I A Until the input strength has remained below the early confirmation threshold I A The cumulative amount of time at time T 3-2 The early confirmation time threshold is reached, as indicated by indicator 573. Therefore, at time T3-2 , the electronic device performs or causes the first operation to be performed. As described above, in these examples, the first operation includes ending the display of the user interface of the application displayed before the single tap is received, so the display 112 of the electronic device changes from displaying the user interface of the first application (e.g., the time application) to displaying a group of application launch icons in the application launch user interface.

[0261] exist Figure 5W In the example, the touch input 531 on the primary button 204 has an intensity that varies over time, including a first intensity increase 532 and a decrease to below the release click intensity threshold I U The first intensity of decreases 534, as indicated by indicator 512, and then falls below the early confirmation threshold I at time T2. A , as indicated by indicator 568. The input then continues with a second intensity increase 536 to above the press click intensity threshold I D strength (or alternatively, to a strength above the second press click strength threshold I D2 The strength of Figure 5G ), as indicated by indicator 574. The second intensity increase 536 occurs before the default delay period expires and before the intensity of the input meets the early confirmation criteria, so the electronic device recognizes the second input event (e.g., double-click) and the first operation is not performed. Figure 5W In the example shown, the intensity drops below the early confirmation intensity threshold I A The time T2 until the intensity rises above the early confirmation threshold I A Time T 2-1 The time period is shorter than the early confirmation time threshold.

[0262] Conversely, in response to the second intensity increasing 536 to above the press click intensity threshold I D The electronic device changes from displaying a previous user interface such as an application user interface (such as Fig.5O ) to display a multitasking user interface, such as Figure 5W shown.

[0263] Optionally, tactile output 503 is generated in conjunction with the electronic device identifying a second input event (e.g., detecting that second intensity increase 536 continues to an intensity above the press click intensity threshold before the default delay period expires). In some embodiments, tactile output 503 is a tactile output mode with MiniTap (270 Hz) ( Figure 4F )’s tactile output.

[0264] like Figure 5XAs shown, the touch input 533 on the primary button 204 has an intensity that varies over time, including a first intensity increase 532 and a decrease to below the release click intensity threshold I U The first intensity of decreases 534, as indicated by indicator 512, and then falls below the early confirmation threshold I A In some embodiments, the measurement of the delay time (for delaying the execution of the first operation corresponding to the single-click input) begins when the intensity of the input decreases below the release click intensity threshold I U , and as long as the input remains below the release click intensity threshold I U Alternatively, in some embodiments, the measurement of the delay time begins when the input strength increases to above the press-click strength threshold I D ,like Figure 5S As shown, once a release click is detected, as long as the strength of input 533 remains below the release click strength threshold I U Just keep measuring.

[0265] exist Figure 5Y In the embodiment, the electronic device determines whether the strength of the touch input 531 meets the double-click standard or the triple-click standard, and thus determines whether the input is a double-click or a triple-click. Figure 5W The touch input 531 shown in FIG. 5 has a strength that varies over time, including a first strength increase 532 and a decrease to below a release click strength threshold I U The first intensity of decreases 534, as indicated by indicator 512, and then falls below the early confirmation threshold I at time T2. A , as indicated by indicator 568. Input 531 then continues with a second intensity increase 536 to above the press click intensity threshold I D strength (or alternatively, to a strength above the second press click strength threshold I D2 The strength of Figure 5G ), as indicated by indicator 574, and the second intensity decreases 538 to below the release click intensity threshold I U strength, as indicated by indicator 522.

[0266] At this point, the intensity of input 531 has followed the change required by the double click. After this point, four possible continuations of input 531 are shown. Figure 5Y middle. Figure 5Y One possibility shown is lift-off or cessation of touch input 531 , as indicated by indicator 576 , which in some embodiments is sufficient to satisfy the double-click detection criteria.

[0267] Figure 5Y The second possibility shown is that the intensity of touch input 531 drops below and remains below the early confirmation threshold I ALasts for at least the early confirmation time threshold, as indicated by indicator 577, and thus the double-click detection criteria is met. Figure 5Y The third possibility shown is that the intensity of touch input 531 drops below and remains below the release click intensity threshold I U (and above the early confirmation threshold I A ) for at least the default delay time threshold, as indicated by indicator 578, and thereby satisfying the double-click detection criteria. The electronic device performs or causes execution of a second operation, such as transitioning from a previous user interface to a multitasking user interface, in response to detecting that input 531 satisfies the double-click detection criteria. Figure 5Y shown.

[0268] Figure 5Y A fourth possibility is shown where, after the second intensity decrease 538 and where the touch input 531 does not meet the double-click detection criteria, the intensity of the touch input 531 has a third intensity increase 580 during which the intensity increases above the press-click intensity threshold I D (or alternatively, increase to a value above a corresponding down click intensity threshold I based on the historical intensity of touch input 531) D ), as indicated by indicator 579, and thereby satisfies the triple-click detection criteria. The electronic device performs or causes execution of a third operation, such as transitioning from a previous user interface to a predefined user interface (e.g., a device settings interface) or transitioning to an easy access mode, in response to detecting that input 531 satisfies the triple-click detection criteria. When there is a triple-click operation configured to be detected in response to a change in the intensity of the contact on the primary button, the device optionally imposes a delay in detecting a double-click operation after detecting the second release click to ensure that the triple-click is not detected. The delay in detecting a double-click is optionally set to the same as described above with reference to Figure 5O-5Y The delay for detecting a single click operates in a similar manner as described.

[0269] Figure 5Z-5II A touch input scenario is shown in which the length of time it takes to recognize a "long press" depends on the strength of the touch input. For example, if the default time for recognizing a long press input is assumed to be 500ms, then according to some embodiments, if the strength of the input reaches a predefined or corresponding strength level, the time for recognizing the input as a long press input is reduced to 400ms or even 300ms.

[0270] Figure 5Z A diagram is shown in which the rate (sometimes referred to as the recognition rate or timer rate) varies with the intensity of the input. More specifically, when the intensity is below the first press click intensity threshold I D When the intensity is between the first press click intensity threshold I D The second click intensity threshold I D+In some embodiments, as the input intensity changes from I to D Change to I D+ , the rate changes linearly from the first rate r1 to the second rate r2. However, in some embodiments, as the input intensity changes from I D Change to I D+ , the rate changes from a first rate r1 to a second rate r2 in a plurality of discrete steps, such as one or more steps.

[0271] In some embodiments, the timeout timer or counter is updated at periodic intervals according to the current rate determined by the current or most recently measured input intensity. For example, in some embodiments, the timeout timer or counter is updated as indicated by the pseudo-code representation in Table 1 of the timeout timer update function.

[0272]

[0273] In Table 1, "timeout_value" is the current value of the timer; start_value is the default timeout period, such as 500ms; current_intensity is the current intensity of the input or the last measured intensity; rate(current_intensity) is a rate function that maps the current intensity to a rate, an example of which is shown in Figure 5Z ; timeout_event is an event delivered to a corresponding software module such as contact / motion module 130 or application 136 when the timer "times out" (i.e., when timeout_value reaches or drops below 0). In some embodiments, the timeout timer update function first updates the input when the intensity reaches a predefined intensity threshold such as the first press click intensity threshold I. D , so the timeout timer's timeout_value is initialized to the default timeout period when the input intensity reaches the predefined intensity threshold.

[0274] The amount of time it takes for the timeout timer or counter to expire and then issue a timeout event varies depending on the strength of the input. Moreover, in some embodiments, if the strength of the input meets (e.g., reaches or drops below) the release click strength threshold, the timeout function is aborted without issuing a timeout event. For example, in some embodiments, if there is a lift-off of the contact, the strength of the input meets the release click strength threshold. In some embodiments, the release click strength threshold is the strength threshold described above with reference to FIG. Figures 5A-5N The dynamic intensity threshold.

[0275] exist Figure 5AAIn the example, the electronic device 100 receives a touch input 535 on the main button 204 and displays a user interface on the display 112, the touch input having the measured or detected intensity. In this example, the displayed user interface is a user interface of an application. Figure 5AA Included is a first graph showing the intensity of input 535 over a period of time, a second graph showing the value of the aforementioned timeout timer over the same period of time, and a third graph showing the timer rate during the same period of time, the timer rate being a function of the input intensity.

[0276] exist Figure 5AA When the strength of input 535 reaches or rises above the first press click strength threshold I D At time T1 indicated by indicator 582, the timeout timer is initialized to a starting value, such as a default timeout period for detecting a long press. In addition, the intensity of input 535 remains above the first press click intensity threshold I D , and the value of the timeout timer decreases at a rate determined by the intensity of input 535. Since, in this example, the intensity of input 535 remains constant during this period of time, the timeout timer decreases at a constant rate determined by the intensity of input 535.

[0277] exist Figure 5BB In the example shown, the intensity of input 543 remains constant from time T1 to time T2, as indicated by indicator 583, and at time T2, the value of the timeout period reaches zero, indicating that the timeout period for detecting a long press has expired, sometimes referred to as a long press being detected. Figure 5BB As shown, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., switches to a user help user interface) according to the corresponding determination. In some embodiments, the aforementioned determination includes determining that the strength of the input does not meet the release click detection criteria before the timeout period expires. In addition, in some embodiments, the operation performed when the long press is detected includes generating a tactile output 504. In some embodiments, the tactile output 504 is a tactile output mode with FullTap (125Hz) ( Figure 4F )’s tactile output.

[0278] and Figure 5BB Different, in Fig.5II In the embodiment, the electronic device 100 receives the touch input 543 on the main button 204 and displays the user interface on the display 112 (in Fig.5II Not shown, but can be Figure 5AA ), the touch input has an intensity that is measured or detected. Fig.5IIIncluded is a first graph showing the intensity of input 543 over a period of time, a second graph showing the value of the aforementioned timeout timer over the same period of time, and a third graph showing the timer rate during the same period of time, the timer rate being a function of the input intensity.

[0279] exist Fig.5II , the intensity of input 543 remains below the first press click intensity threshold I D , passing through the time period from time T1 to time T6, as indicated by indicator 587. The timeout timer is initialized to a starting value, such as a default timeout period for detecting a long press. In addition, since the intensity of input 535 remains below the first press click intensity threshold I D , so the value of the timeout timer decreases at the default rate. Since in this example the intensity of input 535 remains below the first down click intensity threshold I during this time period, D , so the timeout timer value decreases at a constant default rate and therefore expires after a default period of time, in Fig.5II It is represented by the time from T1 to T6.

[0280] In addition, if Fig.5II As shown, once the default time period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., switches to a user help user interface) according to the corresponding determination. In some embodiments, the aforementioned determination includes determining that the intensity of the input does not meet the release click detection criteria before the default timeout period expires. In addition, in some embodiments, the operation performed when the long press is detected includes generating a tactile output 504. In some embodiments, the tactile output 504 is a tactile output mode with FullTap (125Hz) ( Figure 4F )’s tactile output.

[0281] exist Figure 5CC In the example, the electronic device 100 receives a touch input 537 on the main button 204 and displays a user interface on the display 112, the touch input having the measured or detected intensity. In this example, the displayed user interface is a user interface of an application. Figure 5CC and Figure 5AA Similar, except that the intensity of touch input 537 is greater than Figure 5AA The touch input 535 in is higher or has a larger magnitude. Therefore, Figure 5CC The timer rate ratio in the example shown is Figure 5AA The timer rate in the example shown in is higher or greater, and the timeout timer value is Figure 5CC China-Israel-Belgium Figure 5AA The faster the rate of decrease.

[0282] exist Figure 5DDIn the example shown, the intensity of input 537 remains constant from time T1 to time T3, which is before time T2, and the value of the timeout timer reaches zero at time T3, as indicated by indicator 584, indicating that the timeout period for detecting a long press has expired. Therefore, the timeout period for detecting a long press (T1 to T3) is Figure 5CC-5DD In the example shown Figure 5AA-5BB The timeout period (T1 to T2) in the example shown is short because Figure 5CC-5DD The intensity ratio of the touch input in the example shown Figure 5AA-5BB The example shown is taller and Figure 5CC-5DD The corresponding timer rate ratio in the example shown is Figure 5AA-5BB The example shown is taller.

[0283] In addition, if Figure 5DD As shown, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., switches to a user help user interface) according to the corresponding determination. In some embodiments, the aforementioned determination includes determining that the strength of the input does not meet the release click detection standard before the timeout period expires.

[0284] exist Figure 5EE In the example, the electronic device 100 receives a touch input 539 on the main button 204 and displays a user interface on the display 112, the touch input having the measured or detected intensity. In this example, the displayed user interface is a user interface of an application. Figure 5EE and Figure 5AA and 5CC Similar, except that the intensity of touch input 539 is greater than Figure 5AA The touch input 535 and Figure 5CC The touch input 537 in is higher or has a larger magnitude. Therefore, Figure 5EE The timer rate ratio in the example shown is Figure 5AA and 5CC The timer rate in the example shown is higher or greater, and the timeout timer value is Figure 5EE China-Israel-Belgium Figure 5AA and 5CC The faster the rate of decrease.

[0285] exist Figure 5FF In the example shown, the intensity of input 539 remains constant from time T1 to time T4, which is before time T3 and time T2, and the value of the timeout timer reaches zero at time T4, as indicated by indicator 585, indicating that the timeout period for detecting a long press has expired. Therefore, the timeout period for detecting a long press (T1 to T4) is Figures 5EE-5FF In the example shown Figure 5AA-5BB The timeout period (T1 to T2) and Figure 5CC-5DDThe timeout periods (T1 to T3) in the example shown are short because Figures 5EE-5FF The intensity ratio of the touch input in the example shown Figure 5AA-5BB and 5CC-5DD are higher, and Figures 5EE-5FF The corresponding timer rate ratio in the example shown is Figure 5AA-5BB And the examples shown with 5CC-5DD are taller.

[0286] In addition, if Figure 5FF As shown, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., switches to a user help user interface) according to the corresponding determination. In some embodiments, the aforementioned determination includes determining that the strength of the input does not meet the release click detection standard before the timeout period expires.

[0287] exist Figure 5GG In the example, the electronic device 100 receives a touch input 541 on the main button 204 and displays a user interface on the display 112, the touch input having the measured or detected intensity. In this example, the displayed user interface is a user interface of an application. Figure 5GG and Figure 5AA Similarly, the touch input 541 has the same Figure 5AA The same or similar strength of touch input 535 is obtained. Figure 5GG The timer rate in the example shown is the same as Figure 5AA The timer rates in the examples shown are the same or similar, and the timeout timer value is Figure 5GG China-Israel Figure 5AA decrease at the same or similar rate.

[0288] exist Figure 5HH In the example shown, the intensity of input 541 changes or varies between time T1 and time T5. In this example, the intensity of input 541 at time T5, indicated by indicator 586, is at or about the intensity level associated with the maximum timer rate of the timeout timer. Due to the change in intensity between time T1 and time T5, the timer rate changes between time T1 and time T5 based on the intensity of input 541, and therefore, the rate of decrease of the value of the timeout timer changes between time T1 and time T5. In this example, the rate of decrease of the value of the timeout timer changes between time T1 and time T5 based on the intensity of input 541. 3b Alternatively, if the input intensity increases at time T 3b has decreased, the rate at which the timeout timer value decreases has slowed down.

[0289] In this example, the value of the timeout timer reaches zero at time T5, indicating that the timeout period for detecting a long press has expired. The timeout period in this example is the period from time T1 to time T5.

[0290] In addition, if Figure 5HH As shown, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., switches to a user help user interface) according to the corresponding determination. In some embodiments, the aforementioned determination includes determining that the strength of the input does not meet the release click detection standard before the timeout period expires.

[0291] Figures 6A-6F 1 is a flow chart showing a method 600 of monitoring input on an intensity-sensitive input element and detecting a release click and / or a press click in the monitored input using one or more intensity thresholds based on previous input intensities of the input. The method 600 is performed in an electronic device (e.g., device 300, device 301) having a display, a touch-sensitive surface, and one or more sensors 165 for detecting intensity of contact with the touch-sensitive surface. Figure 3 ; or a portable multifunction device 100, Figure 1A ) is performed at. In some embodiments, the electronic device executing method 600 has a main button 204 including one of sensors 165 in addition to the touch-sensitive surface. In some embodiments, the main button 204 is separated from the display and optionally includes a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display. In some embodiments, the main button 204 is a virtual main button displayed on the display (for example, having a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of input using the virtual main button). In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 600 are optionally combined, and / or the order of some operations is optionally changed.

[0292] As described below, method 600 provides a way to accurately determine user intent as to whether a touch input includes a release click or a press click by considering the intensity of the user's input during the portion of the input immediately before the intensity decreases or immediately before the intensity increases. Method 600 reduces "false positives" such as input being incorrectly detected as including a release click or a press click, and "false negatives" such as input being incorrectly detected as not including a corresponding release click or a press click, thereby generating a more efficient human-computer interface. For battery-powered electronic devices, considering the priority intensity of the user's touch input enables the user to input gestures, such as one or more of a single-tap, double-tap, and triple-tap gestures, more quickly and efficiently, which saves power and increases the time between battery charges.

[0293] The device detects (602) an increase in a first intensity of an input on an input element that satisfies a press-click detection criterion, and after detecting the increase in the first intensity of the input on the input element, detects (608) a decrease in a first intensity of the contact. Figure 5B As shown, touch input 505 on primary button 204 has an intensity that varies over time, including a first intensity increase 520 (sometimes referred to as a first press click) reaching a peak intensity I Peak, , the peak intensity is higher than the first press click intensity threshold I D .like Figure 5C As shown, after the first intensity increase, the intensity of the touch input on the main button 204 decreases, and the electronic device detects a first intensity decrease 522 of the contact, sometimes referred to as a first release click.

[0294] In some embodiments, the input on the input element includes (604) input on the touch-sensitive surface. Figure 5B and 5C In the example shown, the input on the input element is input 505 on primary button 204. Additionally, in some embodiments, for a first intensity increase, the press-click detection criteria require (606) that the intensity of the input increases above a first press-click intensity threshold, such as Figure 5B and 5C The first press click intensity threshold I shown D , so that the press click detection criteria are satisfied. In some embodiments, and Figure 5B and 5C In the example shown, the down-click intensity threshold is a fixed predefined value and is therefore neither time-varying nor based on the intensity of the input immediately prior to detecting the first intensity increase of the input (602).

[0295] In response to detecting (610) that the first intensity of the input is decreased (eg, decreased by 552, Figure 5C ), method 600 includes determining (612) whether a first intensity reduction of the input satisfies a release click detection criterion, wherein (A) for the first intensity reduction, the release click detection criterion requires that the intensity of the input decrease to below a first release click intensity threshold (e.g., threshold I U , Figure 5C ) so that the release click detection criterion is satisfied, (B) a first release click intensity threshold (e.g., threshold I U , Figure 5C ) based on an increase in intensity of the contact detected before a first decrease in intensity of the input is detected (e.g., first increase 520, Figure 5B-5C ) during the first decrease 522. As described in more detail below, the first release click intensity threshold is based on the peak intensity I input before the first decrease 522. Peak (See Figure 5C), or another characteristic intensity or representative intensity of the input. In some embodiments, the peak intensity, or the characteristic intensity or representative intensity is multiplied by a multiplier such as a value between 0.6 and 0.75 to determine the first release click intensity threshold.

[0296] In some embodiments, the first release click intensity threshold is time-varying based on a low-pass filter of the detected intensity input during the first intensity reduction of the contact (632). For example, Figure 5K The release click intensity threshold I is shown to change dynamically as the intensity of the input changes during the first intensity reduction 542. U(t) 546. In the above reference Figure 5K In the example described, during the first intensity reduction 542, the input intensity is low-pass filtered to generate a first time-varying value I LPup 544. The first time-varying value is then multiplied by a fixed value such as 0.7 or a strength-based value such as Figure 5J The ratio shown is used to generate a time-varying release click intensity threshold I U(t) , where the “(t)” symbol indicates that the value is time-varying.

[0297] In some embodiments, as described above with reference to Figure 5J-5N As described above, release the click intensity threshold I U The ratio of the intensity value representing the intensity of the contact is changed based on the intensity value representing the intensity of the contact (e.g., the peak intensity value of the input after low pass filtering) (642), so that when the release click intensity is based on the first intensity value representing the intensity of the contact (e.g., the intensity I D1 , Figure 5J ), the ratio of the release click intensity threshold to the first intensity value has a first value (e.g., a2, Figure 5J ); when the click intensity is released, the second intensity value representing the intensity of the contact is greater than the first intensity value (e.g., intensity I D2 , Figure 5J ), the ratio of the release click intensity threshold to the second intensity value has a second value (e.g., a1) that is different from the first value (e.g., lower or higher than the first value) (e.g., when the current low-pass filtered intensity of the contact is 500g, the ratio is 0.6, and when the current low-pass filtered intensity of the contact is 300g, the ratio is 0.73). In some embodiments, the ratio is changed based on the intensity of the contact to account for thermal drift that increases at higher intensity levels.

[0298] In some embodiments, as described above with reference to Figure 5J-5NAs described, the magnitude of the release click strength threshold is set (644) by multiplying an intensity value representing the intensity of the contact (e.g., the peak intensity of the contact before a decrease in the intensity of the contact is detected, or the current intensity of the contact after a low pass filter) by an adjustment value (e.g., a value between 0 and 1) determined at least in part based on the magnitude of the intensity value representing the intensity of the contact (e.g., the peak intensity of the contact before a decrease in the intensity of the contact is detected, or the current intensity of the contact after a low pass filter).

[0299] In some embodiments, as described above with reference to Figure 5J-5N The ratio of the release click intensity threshold to the intensity value representing the intensity of the contact varies according to the maximum characteristic intensity of the input (646). For example, the multiplier varies from a predefined maximum value when the detected intensity is below a first intensity value (e.g., 300g) to a predefined minimum value when the detected intensity is above a second intensity value (e.g., 500g).

[0300] In some embodiments, such as Figure 5J As shown, the ratio of the release click intensity threshold to the intensity value representing the intensity of the contact changes smoothly (648) from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact changes between a first intensity value and a second intensity value, wherein the first intensity value is less than the second intensity value. In some embodiments, "smoothly changing" means that the ratio changes from a predefined maximum value to a predefined minimum value in two or more steps, or three or more steps, as the intensity value representing the intensity of the contact changes between the first intensity value and the second intensity value. More generally, the ratio of the release click intensity threshold to the intensity value representing the intensity of the contact changes monotonically from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact changes between the first intensity value and the second intensity value.

[0301] In some embodiments, the release click intensity threshold is not less than a predefined minimum release click intensity threshold (649). For example, when the aforementioned ratio or multiplier is applied to the intensity value representing the intensity of the contact, the release click intensity threshold is set to the larger of the predefined minimum release click intensity threshold (e.g., 130g) and the release click intensity threshold determined using the ratio or multiplier. In formula form, imposing the predefined minimum release click intensity threshold can be expressed as:

[0302] I U =max(IT min ,I representative *β)

[0303] Among them, IT min is the predefined minimum release click strength threshold, I representative is a strength value representing the strength of the contact, and β is the aforementioned ratio or multiplier.

[0304] References in the above Figure 5J-5N In some embodiments, examples of which are discussed, the release click intensity threshold is determined based on a multiplier (634) having a value greater than zero and less than one applied to the characteristic intensity of the input. The multiplier is actually the aforementioned ratio, which is multiplied by the intensity value representing the intensity of the contact. As described in more detail elsewhere, the intensity value representing the intensity of the contact in some embodiments is the peak intensity before the release click or the lowest intensity immediately before the press click. In some other embodiments, the intensity value representing the intensity of the contact is a low-pass filtered intensity value of the corresponding portion of the input, such as Figure 5L For the second press click as shown and as Figure 5K and 5N Release for the first or second click as shown.

[0305] In some embodiments, the multiplier varies according to the characteristic intensity of the input (636). For example, in some embodiments, the multiplier varies smoothly (638) from a predefined maximum value to a predefined minimum value as the characteristic intensity of the input varies between a first intensity value and a second intensity value, wherein the first intensity value is less than the second intensity value. In some embodiments, "smoothly varying" means that the multiplier varies from a predefined maximum value to a predefined minimum value in two or more steps, or three or more steps, as the intensity value representing the intensity of the contact varies between the first intensity value and the second intensity value. For example, the multiplier varies from a predefined maximum value when the detected intensity is lower than the first intensity value (e.g., 300g) to a predefined minimum value when the detected intensity is higher than the second intensity value (e.g., 500g). More generally, the multiplier varies monotonically from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact varies between the first intensity value and the second intensity value.

[0306] In some embodiments, the release click intensity threshold is the greater of a predefined minimum release click intensity threshold and a value determined based on a multiplier (e.g., a value greater than zero and less than one) applied to the characteristic intensity of the input (640). In formula form, imposing a predefined minimum release click intensity threshold can be expressed as:

[0307] I U =max(IT min ,I char *β)

[0308] Among them, IT min is the predefined minimum release click strength threshold, I char is the characteristic strength of the contact and β is the aforementioned multiplier.

[0309] In addition, based on determining (616) that the first intensity reduction of the input meets the release click detection criteria, the electronic device provides a first feedback indicating that the first intensity reduction of the input is recognized as a release click, and based on determining (616) that the intensity reduction of the input does not meet the release click detection criteria, the electronic device abandons providing the first feedback. Figure 5C As shown, when the intensity reduction 522 of the input satisfies the release click detection criteria (indicated by indicator 512), the device provides a tactile output 502 to indicate that the intensity reduction 522 of the input is recognized as a release click, and as shown in FIG. Fig. 5F As shown, when the intensity reduction 530 of the input does not meet the release click detection criteria, the electronic device does not generate a tactile output (eg, tactile output 502 ) indicating that the intensity reduction of the input (eg, reduction 530 ) is recognized as a release click.

[0310] Now that various aspects of detecting a first click have been considered, various aspects of detecting a second click are now considered. In particular, after determining that the first intensity decrease satisfies the release click detection criteria, the electronic device detects (614) a second intensity increase of the input. Various scenarios for detecting the second intensity increase are described above with reference to Figure 5D-5E , 5G-5I and 5L-5N were discussed.

[0311] In response to detecting the second increase in intensity of the input (620), based on determining that the second increase in intensity of the input satisfies the press-click detection criteria, the electronic device provides second feedback indicating that the second increase in intensity is recognized as part of a double-click input. Figure 5G As described above, the second tactile output 503 is generated in conjunction with detecting the release click of the second click of the input 511 on the main button 204. For example, in some embodiments, the second (press click) tactile output is generated by one or more tactile output generators 167 ( Figure 1A ) to generate. For another example, providing the second feedback includes using one or more speakers (e.g., speaker 111, Figure 1A ) generates an audio output, and / or displays it on a display of an electronic device (e.g., display 112, Figure 5G , 5H ) changes according to the double-click operation, such as displaying a multi-tasking user interface, such as Figure 5H shown.

[0312] For the second intensity increase, the press-click detection criteria require (622) that the intensity of the input increases above a second press-click intensity threshold (e.g., intensity threshold I D2 , Figure 5G ) so that the down-click detection criteria are satisfied, and a second down-click intensity threshold for a second intensity increase is selected based on the intensity of the input during the first intensity decrease of the contact (624). Figure 5G In the example shown, the second down click intensity threshold for the second intensity increase is based on the lowest intensity I input during the first intensity decrease 534 of the contact. Valley To choose.

[0313] In some embodiments, the second down-click intensity threshold is time-varying, and the second down-click intensity threshold is selected based on a low-pass filter of the detected intensity of the input during a second intensity reduction of the contact detected after the first intensity reduction of the input (626). For example, the intensity reduction of the input satisfies the release-click detection criteria. Figure 5L The second pressing click intensity threshold I is shown as time-varying D(t) 554 based on a low pass filter (I LPdown 552) to select or determine. In some embodiments, the low-pass filtered intensity (I LPdown 552) is initially set at the beginning of the second intensity increase 550 of the contact to the lowest intensity I input during the first intensity decrease 534 of the contact. Valley .

[0314] In response to detecting the second increase in intensity of the input (620), the electronic device forgoes providing the second feedback based on determining (628) that the second increase in intensity of the input does not satisfy the press click detection criteria. Figure 5G If the input reaches the intensity threshold I for the second press, D2 If the touch input stops before (for example, if the contact is lifted off), the second increase in intensity of the input will not meet the press-click detection criteria, and the electronic device will forgo providing the second feedback.

[0315] In some embodiments, method 600 includes generating (650) a first tactile output in conjunction with detecting that the input's intensity increases to satisfy a press-click detection criterion. Figure 5C As described, first tactile output 502 is generated in conjunction with detecting a press click during a decrease in intensity 522 of input 505 on primary button 204 .

[0316] In some embodiments, method 600 includes generating (652) a second tactile output in conjunction with detecting that the intensity of the input decreases to satisfy the release click detection criteria. Figure 5G As described above, the second tactile output 503 is generated in conjunction with detecting the release click of the second click input 511 on the primary button 204 .

[0317] In some embodiments, method 600 includes generating (654) a response displayed on a display of the electronic device in response to detecting an increase in intensity of an input on an input element that satisfies a press-click detection criterion. In one example, the displayed response is a response that visually distinguishes an object in a user interface or an area of ​​a user interface whose position corresponds to the input.

[0318] In some embodiments, providing the first feedback includes generating (656) a response that is displayed on a display of the electronic device. Figure 5B-5C In the example shown, the response, which may be referred to as a first tap or click response, is switching from displaying a user interface of a first application (eg, a timer application) to displaying an application launch user interface.

[0319] In some embodiments, method 600 includes generating (658) the same first tactile output in conjunction with detecting multiple instances in which an increase in intensity of an input detected on the input element satisfies a down-click detection criterion (including instances in which the down-click detection criterion is associated with different down-click intensity thresholds). For example, generating the same tactile output for both the first down-click and the second down-click (e.g., a tactile output having a MicroTap (270 Hz) tactile output mode, for example) even when the first down-click and the second down-click are detected at different intensity thresholds. Figure 4F ).

[0320] In some embodiments, method 600 includes generating (660) the same second tactile output in conjunction with detecting multiple instances in which a decrease in intensity of an input detected on the input element satisfies a release-click detection criterion (including instances in which the release-click detection criterion is associated with different release-click intensity thresholds). For example, generating the same tactile output for both the first release-click and the second release-click (e.g., a tactile output having a MiniTap (270 Hz) tactile output mode) even when the first release-click and the second release-click are detected at different intensity thresholds. Figure 4F ). In some embodiments, the release-click tactile output pattern has a lower gain than the press-click tactile output pattern (e.g., the release-click tactile output pattern is a reduced amplitude version of the press-click tactile output pattern).

[0321] In some embodiments, method 600 includes, upon detecting a second increase in intensity of input on the input element (eg, increase in intensity 536 of input 513, Figure 5H ), after which a second intensity decrease in the contact is detected (662) (e.g., intensity decrease 538 of input 513, Figure 5H ), and in response to detecting a decrease in the second intensity of the input, providing a third feedback (e.g., performing a double-click operation, an example of which is described above with reference to Figure 5HIn one embodiment, the electronic device detects that the second intensity reduction is recognized as a release click input. In such embodiments, for the second intensity reduction, the release click detection criteria require that the intensity of the input decreases to below a second release click intensity threshold so that the release click detection criteria are met; and the second release click intensity threshold is selected based on the intensity of the input during the second intensity increase of the contact. In addition, based on determining that the second intensity reduction of the input does not meet the release click detection criteria, the electronic device abandons providing the third feedback (e.g., by abandoning performing the double-click operation).

[0322] In some embodiments, the third feedback is Figure 5H The third feedback is generated or triggered at or immediately after the time indicated by the indicator 522, at which time the second intensity decrease satisfies the release click detection criteria. In some embodiments, the third feedback is or includes a transition to a multitasking user interface, such as Figure 5H In some embodiments, the third feedback is or includes generating a tactile output 503, such as Figure 5H In some embodiments, the tactile output 503 is a tactile output mode having a MiniTap (270 Hz) Figure 4F )’s tactile output.

[0323] It should be understood that Figures 6A-6F The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize a variety of ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein in conjunction with other methods described herein (e.g., method 700 and method 800) are also applicable in a similar manner to the details of the processes described above in conjunction with 6A to 6F The method 600 is described. For example, the above reference Figures 6A-6E The method 600 of monitoring input on a strength-sensitive input element and detecting a release click and / or a press click in the monitored input using one or more strength thresholds based on previous input strengths of the input optionally has one or more features of the single-click acceleration detection method described herein with reference to method 700 and / or the long-press input acceleration detection method described herein with reference to method 800. For the sake of brevity, these details are not repeated here.

[0324] According to some embodiments, Fig. 9 900 is a functional block diagram of an electronic device configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of various described embodiments. It should be understood by those skilled in the art that Fig. 9The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0325] like Fig. 9 As shown, the electronic device 900 includes a display unit 902 (e.g., corresponding to the display 112) configured to display a user interface, an intensity-sensitive input unit 904 (e.g., corresponding to the main button 204) configured to detect the intensity of the user input performed with the input element, and a processing unit 910 coupled to the display unit 902 and the intensity-sensitive input unit 904. In some embodiments, the electronic device 900 also includes a touch-sensitive surface unit 906 for receiving a touch input on a surface (such as a display surface of the display unit 902), and one or more tactile output units 908 for generating tactile output, which are also coupled to the processing unit 910. In some embodiments, the processing unit 910 includes one or more of the following subunits: an intensity monitoring unit 912, a release click determination unit 914, a press click determination unit 916, and a feedback unit 918. In some embodiments, the feedback unit 918 includes a response display unit 920, and a tactile output unit 922.

[0326] The processing unit 910 is configured to detect (e.g., using the intensity monitoring unit 912) a first intensity increase of the input on the input element that satisfies the press-click detection criteria, and after detecting the first intensity increase of the input on the input element, detect a first intensity decrease of the contact. In response to detecting the first intensity decrease of the input, the processing unit 910 determines (e.g., using the release-click determination unit) whether the first intensity decrease of the input satisfies the release-click detection criteria, wherein for the first intensity decrease, the release-click detection criteria requires that the intensity of the input decrease below a first release-click intensity threshold in order for the release-click detection criteria to be satisfied, the first release-click intensity threshold being selected based on the intensity of the input during the intensity increase of the contact detected before detecting the first intensity decrease of the input.

[0327] The processing unit 910 provides first feedback (for example, using the feedback unit 918) to indicate that the first reduction in input intensity is identified as a release click based on determining that the first reduction in input intensity meets the release click detection criteria, and abandons providing the first feedback based on determining that the reduction in input intensity does not meet the release click detection criteria.

[0328] In some embodiments, for the first intensity increase, the press-click detection criteria require that the intensity of the input increase above a first press-click intensity threshold in order for the press-click detection criteria to be satisfied. In some such embodiments, the processing unit 910 is further configured to, after determining that the first intensity decrease satisfies the release-click detection criteria, detect a second intensity increase of the input (e.g., using the intensity monitoring unit 912); and in response to detecting the second intensity increase of the input, based on determining (e.g., using the press-click determination unit 916) that the second intensity increase of the input satisfies the press-click detection criteria, provide second feedback (e.g., using the feedback unit 918) indicating that the second intensity increase is identified as part of a double-click input. For the second intensity increase, the press-click detection criteria require that the intensity of the input increase above a second press-click intensity threshold in order for the press-click detection criteria to be satisfied. In these embodiments, the second press-click intensity threshold for the second intensity increase is selected based on the intensity of the input during the first intensity decrease of the contact, and based on determining (e.g., using the press-click determination unit 916) that the second intensity increase of the input does not satisfy the press-click detection criteria, the processing unit 910 forgoes providing the second feedback.

[0329] In some embodiments, the second down click intensity threshold is time-varying and is selected by processing unit 912 (e.g., using down click determination unit 916) based on low-pass filtering of the detected intensity of the input during a second increase in intensity of the contact detected after a first decrease in intensity of the input.

[0330] In some embodiments, for a first intensity increase, the down-click detection criteria requires that the intensity of the input increase above a first down-click intensity threshold in order for the down-click detection criteria to be met.

[0331] In some embodiments, input on the intensity-sensitive input unit 904 includes input on a touch-sensitive surface.

[0332] In some embodiments, the first release click intensity threshold is time-varying based on low-pass filtering of the detected intensity input during the first intensity reduction of the contact.

[0333] In some embodiments, the ratio of the release click strength threshold to the strength value representing the strength of contact varies based on the strength value representing the strength of contact, so that when the release click strength is based on a first strength value representing the strength of contact, the ratio of the release click strength threshold to the first strength value has a first value; when the release click strength is based on a second strength value representing the strength of contact that is greater than the first strength value, the ratio of the release click strength threshold to the second strength value has a second value different from the first value.

[0334] In some embodiments, the magnitude of the release click intensity threshold is set (e.g., by processing unit 910 or release click determination unit 914) by multiplying the intensity value representing the intensity of the contact by an adjustment value determined at least in part based on the magnitude of the intensity value representing the intensity of the contact. In some embodiments, the ratio of the release click intensity threshold to the intensity value representing the intensity of the contact varies based on the maximum characteristic intensity of the input. In some embodiments, the ratio of the release click intensity threshold to the intensity value representing the intensity of the contact varies smoothly from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact varies between a first intensity value and a second intensity value, where the first intensity value is less than the second intensity value.

[0335] In some embodiments, the release click strength threshold is no less than a predefined minimum release click strength threshold.

[0336] In some embodiments, the release click intensity threshold is determined (by processing unit 910 or release click determination unit 914) based on a multiplier having a value greater than zero and less than one applied to the characteristic intensity of the input. In some embodiments, the multiplier varies based on the characteristic intensity of the input. In some embodiments, the multiplier varies smoothly from a predefined maximum value to a predefined minimum value as the characteristic intensity of the input varies between a first intensity value and a second intensity value, where the first intensity value is less than the second intensity value.

[0337] In some embodiments, the release click intensity threshold is the greater of a predefined minimum release click intensity threshold and a value determined based on a multiplier having a value greater than zero and less than one applied to the characteristic intensity of the input.

[0338] In some embodiments, the processing unit 910 is further configured to generate a first tactile output (e.g., using the tactile output unit 922 of the feedback unit 918) in conjunction with detecting that the intensity of the input increases to satisfy the press-click detection criteria.

[0339] In some embodiments, the processing unit 910 is further configured to generate a second tactile output (e.g., using the tactile output unit 922 of the feedback unit 918) in conjunction with detecting that the intensity of the input has decreased to satisfy the release click detection criteria.

[0340] In some embodiments, the processing unit is further configured to generate a response displayed by the display unit 902 of the electronic device (e.g., using the response display unit 920 of the feedback unit 918) in response to detecting an increase in the intensity of the input on the input element that meets the press click detection criteria.

[0341] In some embodiments, providing the first feedback includes generating a response that is displayed by the display unit 902 of the electronic device (e.g., using the response display unit 920 of the feedback unit 918).

[0342] In some embodiments, the processing unit is further configured to generate the same first tactile output in conjunction with detecting multiple instances in which an increase in intensity of an input detected on the input element satisfies a down-click detection criterion, including instances in which the down-click detection criterion is associated with different down-click intensity thresholds.

[0343] In some embodiments, the processing unit is further configured to, after detecting an increase in the second intensity of the input on the input element, detect a decrease in the second intensity of the contact; and in response to detecting the decrease in the second intensity of the input, provide third feedback indicating that the second intensity decrease is identified as a release click input based on determining that the second intensity decrease of the input satisfies the release click detection criteria. In such embodiments, for the second intensity decrease, the release click detection criteria require that the intensity of the input decrease to be below a second release click intensity threshold in order for the release click detection criteria to be satisfied; and the second release click intensity threshold is selected based on the intensity of the input during the second intensity increase of the contact. In such embodiments, the processing unit is configured to forgo providing the third feedback based on determining (e.g., release click determination unit 914) that the second intensity decrease of the input does not satisfy the release click detection criteria.

[0344] In some embodiments, the processing unit is further configured to generate the same second tactile output (e.g., using the tactile output unit 922 of the feedback unit 918) in conjunction with detecting multiple instances in which a decrease in intensity of an input detected on the input element satisfies a release-click detection criterion (including instances in which the release-click detection criterion is associated with different release-click intensity thresholds).

[0345] References Figures 6A-6F The operations in the information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0346] References Figure 6A-6F The operation is optionally performed by Figure 1A-1B or Figure 3112. For example, detection operations 602, 608, 614, etc. and determination operations 612, etc. are optionally implemented by contact / motion module 130, feedback operations that provide tactile output are implemented by tactile feedback module 133, and some other operations are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 transmits event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another orientation. When a corresponding predefined event or sub-event is detected, event identifier 180 activates event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A to Figure 1B How the components depicted in the figure can implement other processes.

[0347] Figures 7A-7E 1 is a flow chart showing a method 700 for monitoring input on an intensity-sensitive input element and detecting whether the input is a single click or a double click. The method 700 is performed in an electronic device (e.g., device 300, Figure 3 ; or a portable multifunction device 100, Figure 1A) is performed at. In some embodiments, the electronic device executing method 700 has a main button 204 including one of sensors 165 in addition to the touch-sensitive surface. In some embodiments, the main button 204 is separated from the display and optionally includes a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display. In some embodiments, the main button 204 is a virtual main button displayed on the display (for example, having a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of input using the virtual main button). In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 700 are optionally combined, and / or the order of some operations is optionally changed.

[0348] As described below, method 700 provides a way to accurately determine the user's intent as to whether a touch input is a single click or a double click by considering the intensity of the user's input during the portion of the input immediately following the identification of a first event (e.g., a first intensity reduction of the input has dropped below a release click intensity threshold). Method 700 reduces latency by identifying a single click more quickly than would otherwise be possible, while avoiding "false positives" (such as an input being incorrectly detected as a single click), thereby generating a more efficient human-computer interface. For battery-powered electronic devices, considering the priority intensity of the user's touch input enables the user to input gestures such as single clicks or double clicks more quickly and efficiently, which saves power and increases the time between battery charges.

[0349] The device detects (702) input on an input element (e.g., input 523 on the main button 204, Fig.5O ) includes detecting (702) an increase in the intensity of an input on an input element (e.g., an increase 532, Fig.5O ), followed by a decrease in the intensity of the input on the input element (eg, decrease 534, Fig.5O Depending on the circumstances, the increase in intensity followed by the decrease in intensity is sometimes referred to as a click, or a first click, or a single click. In some embodiments, the input on the input element is or includes a touch-sensitive surface (704) (such as the surface of the primary button 204 ( Fig.5O ) or an input on a touch-sensitive surface of a touch-sensitive display 112 of an electronic device). In some embodiments, detecting a change in intensity of the input on the input element includes continuously detecting (706) the input on the input element.

[0350] Method 700 also includes identifying (708) at least a portion of the change in intensity of the input as a first input event associated with the first operation. As described above, the first input event is sometimes referred to as a click or a first click. An example is provided above with reference to Fig.5O Described.

[0351] After identifying the first input event, method 700 includes delaying (710) execution of the first operation while monitoring a subsequent intensity change of the input for a second input event, wherein the delay is limited by a default delay period. Thus, execution of the first operation (such as changing the displayed user interface from a currently displayed application user interface to a home screen or application launch user interface) is delayed or postponed while the device continues to monitor the intensity change of the input. As described above with reference to Figure 5P As described, in some embodiments, if there is lift-off of the contact, monitoring of changes in intensity of the input stops, so that execution of the first operation is not delayed after the lift-off is detected.

[0352] Method 700 also includes, after delaying the execution of the first operation (712), executing (714) the second operation and abandoning the execution of the first operation based on determining that the second input event (e.g., has been recognized before the default delay period has passed). Figure 5W As described, this situation corresponds to detecting a double click, sometimes achieved by detecting "one and a half" clicks before the default delay time expires.

[0353] Method 700 also includes, after delaying the execution of the first operation (712): based on determining that the early confirmation criteria for the first input event have been met before the default delay period has passed and the second input event has not been recognized, executing (716) the first operation before the default delay period has passed. Figure 5Q-5V As described, when the early confirmation criteria are met before the default delay period has passed and the second input event is not recognized, the single click is recognized and acted upon by performing the first operation at an accelerated rate (e.g., before the default delay period has passed). This reduces the delay for the device to recognize the user input as a single click. Various ways in which the early confirmation criteria for the first input event can be met are described in more detail below with reference to operations 722-738, and are also described above with reference to Figure 5Q-5V Explained in some detail.

[0354] The method 700 also includes, after delaying the execution of the first operation (712), executing (718) the first operation once the default delay period has passed based on determining that the default delay period has passed without the early confirmation criteria for the first input event being met and the second input event being recognized. Figure 5SAs described, this situation occurs when the early confirmation criteria are not met before the default delay period has passed and the second input event is not recognized, so only after the default delay period has passed is the single click recognized and acted upon by performing the first action. This is effectively a backup or default, where it takes the normal or default amount of time to detect that the input is a single click and not a double click.

[0355] In some embodiments, the early confirmation criteria for the first input event (used in operation 716, as described above) include the criterion that the intensity of the input remains below the confirmation intensity threshold for longer than the early confirmation time threshold (720). Figure 5Q In the example, the early confirmation criteria for the first input event include the input strength remaining below the release click strength threshold I U Low confirmation strength threshold I A Lasts longer than the early confirmation time threshold, which is the early confirmation time threshold at Figure 5Q and 5R corresponds to the length of time between T2 and T3. For comparison, the default delay period is Figure 5S , or alternatively, T 2A (release click detection) and T4, as mentioned above Figure 5S Described in more detail.

[0356] In some embodiments, the early confirmation time threshold is less than half of the default delay period 722. For example, in some embodiments, the default delay period is 500 ms, and the early confirmation time threshold is 150 ms, or 200 ms, or a value between 150 ms and 240 ms.

[0357] In some embodiments, a strength threshold (e.g., Figure 5O-5Y The confirmation strength threshold I A ) is lower than the release click intensity threshold for identifying a second input event (e.g. Figure 5O-5Y The release click intensity threshold I U )(724). For example, in some embodiments, the confirmation intensity threshold is 150g and the release click intensity threshold is 200g. In some other embodiments, the confirmation intensity threshold is less than 150g and the release click intensity threshold is greater than 150g. Further in some such embodiments, the release click intensity threshold is determined based on the characteristic intensity of the input during the detected increase in intensity of the input during which the input reaches a peak intensity before a decrease in the intensity of the input on the input element is detected (726). Examples of release click intensity thresholds (sometimes referred to as time-varying intensity thresholds) determined in this manner are described above with reference to Figure 5K and5M Described. Figure 5K and 5M An example of determining a release click intensity threshold using a low-pass filtered intensity of a contact is shown.

[0358] In some embodiments, method 700 includes monitoring the duration of the fast timeout period starting from when the strength of the input decreases below the confirmation strength threshold (728). Figure 5Q , 5R 5T-5V, where monitoring for the fast timeout period begins at time T2, which in those examples is when the strength of the input decreases below the confirmation strength threshold I A hour.

[0359] In some embodiments, the duration of the fast timeout period is the cumulative amount of time that the intensity of the input is below the confirmed intensity threshold after the fast timeout period begins (730). Measuring the duration of the fast timeout period as the cumulative amount of time that the intensity of the input is below the confirmed intensity threshold after the fast timeout period begins is described above with reference to Figure 5T-5V Discuss in more detail.

[0360] In some embodiments, method 700 includes determining, before detecting a decrease in the intensity of the input on the input element, based on a peak characteristic intensity of the input detected during an increase in the detected intensity of the input (e.g., I Peak , Fig.5O ) to determine (732) a confirmation strength threshold (e.g. Figure 5O-5Y The threshold I A ). References Fig.5O Provides further explanation and examples on how to determine the confirmation strength threshold.

[0361] In some embodiments, the confirmation intensity threshold is independent of the peak characteristic intensity of the input detected during the detected increase in intensity of the input (734). For example, the confirmation intensity threshold is set to a fixed intensity threshold such as 100g or 150g, regardless of the peak characteristic intensity of the input detected during the detected increase in intensity of the input.

[0362] In some embodiments, method 700 includes monitoring (736) the duration of the default timeout period starting from when the intensity of the input increases to when the click intensity threshold is pressed. In some other embodiments, method 700 includes monitoring (738) the duration of the default timeout period starting from when the intensity of the input decreases to when the click intensity threshold is released. Both of these options for monitoring the duration of the default timeout period are described above with reference to Figure 5S Describe it. Figure 5S In the example, time T1 is when the input strength has increased to the click strength threshold I Dtime, and time T 2A The input strength has decreased to the release click strength threshold I U time.

[0363] In some embodiments, method 700 includes, after delaying the performance of the first operation, performing (739) a third operation based on determining that the second input event has been recognized after the default delay period has elapsed. Figure 5S , after the default time period has expired, at time T4, if the second input event is recognized, it will be treated as a separate event such as a separate click, and the third operation will correspond to the operation performed by the electronic device in response to the click. A similar example is Figure 5R , where the input includes a single click after the early confirmation criteria are satisfied at time T3. In this example, the third operation that occurs is scrolling from an icon screen (including the first group of application launch icons) in the application launch user interface to another icon screen in the application launch user interface.

[0364] In some embodiments, method 700 includes detecting (741) a series of different inputs on the input element separated by a period of time when no input was detected on the input element, and repeating the identifying and delaying for a plurality of inputs in the series of different inputs. Figure 5E As shown and as referenced above Figure 5E The device may detect a first input (eg, a touch input 505, Figure 5B-5C ) is followed by a touch press of the first input and a lift-off of the second input on the input element (e.g., touch input 507, Figure 5E ) is touched down. In response, the electronic device repeats the recognition of the first input event for each input in the series of different inputs and delays the execution of the first operation.

[0365] In some embodiments, method 700 includes, after delaying performance of the first operation, performing a third operation and abandoning performance of the first operation and the second operation based on determining that the first input event satisfies (740) the long press criteria before the second input event has been recognized. For example, the first input event can be a press click, in which case the third operation can be a long press operation, such as invoking a virtual assistant or operating a dictation mode. Reference Figures 5Z to 5II and Figures 8A-8C Recognition of and response to long press input is discussed in more detail.

[0366] In some embodiments, method 700 includes combining the recognition of a first input event (e.g., recognition of a press click at time T1, or recognition of a press click at time T2) with the recognition of a first input event (e.g., recognition of a press click at time T3). 2a Identify release click, Fig.5O) and generate (742) a first tactile output (eg, first tactile output 502, Fig.5O ).

[0367] In some embodiments, method 700 includes generating (744) a second tactile output in conjunction with identifying a second input event, as described above with reference to Figure 5W described.

[0368] In some embodiments, the first operation is or includes (746) stopping displaying the user interface of the application (and optionally returning to the home screen or application launch screen of the display device), such as Figures 5O to 5Q As shown in the transition of , the second operation includes a multitasking operation (e.g., switching between applications or displaying a user interface that provides options for switching between multiple different applications (such as simultaneously opened applications or recently used applications), such as Figure 5W shown).

[0369] In some embodiments, the first operation is or includes (748) scrolling from one icon screen of the application launch user interface (e.g., an application launch user interface including a first set of application launch icons) to another icon screen of the application launch user interface (e.g., including a second set of application launch icons, the second set of application launch icons including application launch icons that are not in the first set of application launch icons), such as from Figures 5Q to 5R , and the second operation includes a multitasking operation (e.g., switching between applications or displaying a user interface that provides options for switching between multiple different applications (such as simultaneously open applications or recently used applications), such as Figure 5W shown).

[0370] In some embodiments, the recognition of the first input event is based on detecting a change in the characteristic intensity of the input relative to a first intensity threshold (e.g., a release click intensity threshold); the recognition of the second input event is based on detecting a change in the characteristic intensity of the input relative to a second intensity threshold different from the first intensity threshold (e.g., a press click intensity threshold). Figure 5Y , used to identify the release click intensity threshold I of the first event U is different from (e.g., lower than) the press click intensity threshold I for identifying the second event. D .

[0371] In some embodiments, method 700 includes, upon identifying a second input event (e.g., a second release click event, corresponding to Figure 5Y522 in the example embodiment, delaying (760) the performance of the second operation while monitoring a subsequent intensity change of the input for a third input event (e.g., a third down click, sometimes referred to as a triple tap event, corresponding to indicator 579), wherein the delay is limited by the second default delay period. Method 700 also includes, after delaying (762) the performance of the second operation, determining that a third event (e.g., corresponding to indicator 579, Figure 5Y ), performing (764) a third operation (e.g., a triple-click operation) and abandoning performance of the second operation (e.g., a double-click operation). Otherwise, after delaying (762) performance of the second operation, method 700 includes, based on determining that the early confirmation criteria for the second input event are satisfied before the second default delay period has elapsed (e.g., as indicated by indicator 577, Figure 5Y ) and the third input event is not recognized, performing (766) a second operation (e.g., a double-click operation) before the second default delay period has elapsed. Additionally, after delaying (762) the execution of the second operation, method 700 includes, based on determining that the second default delay period has elapsed and the early confirmation criteria for the second input event are not met (e.g., as indicated by indicator 578, Figure 5Y ) and the third input event is not recognized, once the second default delay period has passed, a second operation (e.g., a double-click operation) is performed (768). In some embodiments, the second default delay period is the same as the first default delay period. In some other embodiments, the second default delay period is longer or shorter than the first default delay period.

[0372] According to some embodiments, Fig.10 1 shows a functional block diagram of an electronic device 1000 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of various described embodiments. It should be understood by those skilled in the art that Fig.10 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0373] like Fig.10As shown, the electronic device 1000 includes a display unit 1002 (e.g., corresponding to the display 112) configured to display a user interface, an intensity-sensitive input unit 1004 (e.g., corresponding to the main button 204) configured to detect the intensity of the user input performed with the input element, and a processing unit 1010 coupled to the display unit 1002 and the intensity-sensitive input unit 1004. In some embodiments, the electronic device 1000 also includes a touch-sensitive surface unit 1006 for receiving a touch input on a surface (such as a display surface of the display unit 1002), and one or more tactile output units 1008 for generating tactile output, which are also coupled to the processing unit 1010. In some embodiments, the processing unit 1010 includes one or more of the following subunits: an intensity monitoring unit 1012, a single-click determination unit 1014, a double-click determination unit 1016, a feedback unit 1018, and a delay unit 1024. In some embodiments, the single-click determination unit 1014 includes an early confirmation unit 1015. In some embodiments, the feedback unit 1018 includes a response display unit 1020 and a tactile output unit 1022.

[0374] The processing unit 1010 is configured to: (for example, using the intensity monitoring unit 912) detect a change in the intensity of an input on an intensity-sensitive input unit, which includes detecting an increase in the intensity of the input on the intensity-sensitive input unit, followed by a decrease in the intensity of the input on the intensity-sensitive input unit; (for example, using the single-click confirmation unit 1014) identify at least a portion of the change in the intensity of the input as a first input event associated with a first operation; and after identifying the first input event, (for example, using the delay unit 1024) delay the execution of the first operation, while monitoring subsequent intensity changes of the input for a second input event, wherein the delay is limited by a default delay time period.

[0375] The processing unit 1010 is further configured to, after delaying the execution of the first operation: based on determining that the second input event has been identified before the default delay time period has passed (for example, using the double-click determination unit 1016), execute the second operation and abandon the execution of the first operation; based on determining that the early confirmation criteria for the first input event have not been met before the default delay time period has passed (for example, using the early confirmation unit 1014) and the second input event has not been identified, execute the first operation before the default delay time period has passed; and based on determining that the default delay time period has passed and the early confirmation criteria for the first input event have not been met (for example, using the early confirmation unit 1014) and the second input event has not been identified (for example, using the double-click determination unit), execute the first operation once the default delay time period has passed.

[0376] In some embodiments, the processing unit is further configured to, after delaying execution of the first operation, perform a third operation based on determining that a second input event has been identified after the default delay period has elapsed (eg, using double-click determination unit 1016).

[0377] In some embodiments, the input on the intensity-sensitive input unit is or includes an input on a touch-sensitive surface. In some embodiments, detecting a change in the intensity of the input on the intensity-sensitive input element includes continuously detecting the input on the intensity-sensitive input element.

[0378] In some embodiments, the processing unit is further configured to detect a series of different inputs on the intensity-sensitive input element separated by time periods in which no input is detected on the intensity-sensitive input element, and to repeat the identification (e.g., using a single-click confirmation unit) and delay (e.g., using delay unit 1024) for multiple inputs in the series of different inputs.

[0379] In some embodiments, the early confirmation criteria for the first input event is or includes a criterion that the intensity of the input remains below a confirmation intensity threshold for longer than an early confirmation time threshold. In some embodiments, the early confirmation time threshold is less than half of the default delay time period. In some embodiments, the confirmation intensity threshold is lower than a release click intensity threshold used to identify the second input event.

[0380] In some embodiments, the release click intensity threshold is determined (e.g., using single click determination unit 1014) based on a characteristic intensity of the input during a detected increase in intensity of the input during which the input reaches a peak intensity before a decrease in intensity of the input on the intensity-sensitive input element is detected.

[0381] In some embodiments, the processing unit is further configured to monitor the duration of the fast timeout period starting from when the intensity of the input decreases below the confirmation intensity threshold (e.g., using early confirmation unit 1015). In some embodiments, the duration of the fast timeout period is the cumulative amount of time that the intensity of the input is below the confirmation intensity threshold after the start of the fast timeout period.

[0382] In some embodiments, the processing unit is further configured to determine a confirmation intensity threshold based on a peak characteristic intensity of the input detected during an increase in the detected intensity of the input, prior to detecting a decrease in the intensity of the input on the intensity-sensitive input element (e.g., using single-click determination unit 1014 or early confirmation unit 1015). In some other embodiments, the confirmation intensity threshold is independent of the peak characteristic intensity of the input detected during an increase in the detected intensity of the input.

[0383] In some embodiments, the processing unit is further configured to monitor the duration of the default timeout period starting from when the intensity of the input increases to the click intensity threshold being pressed (e.g., using the single click determination unit 1014). In some other embodiments, the processing unit is further configured to monitor the duration of the default timeout period starting from when the intensity of the input decreases to the click intensity threshold being released.

[0384] In some embodiments, the processing unit is further configured to generate a first tactile output in conjunction with identifying a first input event (e.g., using tactile output unit 1022). In some embodiments, the processing unit is further configured to generate a second tactile output in conjunction with identifying a second input event (e.g., using tactile output unit 1022).

[0385] In some embodiments, the processing unit is further configured to, after identifying the second input event (e.g., using the double-click determination unit), delay the execution of the second operation (e.g., using the delay unit 1024) while monitoring the subsequent intensity change of the input for the third input event, wherein the delay is limited by the second default delay time period. The processing unit is further configured to, after delaying the execution of the second operation, perform the third operation and abandon the execution of the second operation based on determining that the third input event has been identified before the second default delay time period has passed. The processing unit is also configured to perform the second operation before the second default delay time period has passed based on determining that the early confirmation criteria for the second input event have been met before the second default delay time period has passed and the third input event has not been identified. In addition, the processing unit is configured to perform the second operation once the second default delay time period has passed based on determining that the second default delay time period has passed and the early confirmation criteria for the second input event have not been met and the third input event has not been identified.

[0386] In some embodiments, the first operation includes stopping displaying the user interface of the application, and the second operation includes a multitasking operation. In some other embodiments, the first operation includes scrolling from an icon screen in the application launch user interface to another icon screen in the application launch user interface, and the second operation includes a multitasking operation.

[0387] In some embodiments, the processing unit is further configured to, after delaying execution of the first operation (e.g., using delay unit 1024), execute a third operation and abandon execution of the first operation and the second operation based on determining that the first input event meets the long press input criteria before the second input event has been recognized.

[0388] In some embodiments, recognition of a first input event is based on detecting a change in a characteristic intensity of the input relative to a first intensity threshold; recognition of a second input event is based on detecting a change in a characteristic intensity of the input relative to a second intensity threshold that is different from the first intensity threshold.

[0389] References Figures 7A-7E The operations in the information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0390] References Figure 7A-7E The operation is optionally performed by Figure 1A-1B or Figure 3 180. For example, detection operations 702, 706, etc. and identification and determination operations 708, 714, 716, 718, etc. are optionally implemented by contact / motion module 130, feedback operations that provide tactile output are implemented by tactile feedback module 133, and some other operations are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 transmits event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186, and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a corresponding predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update the application internal state 192. In some embodiments, event handler 190 accesses a corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand that based on Figure 1A to Figure 1B How the components depicted in the figure can implement other processes.

[0391] Figures 8A-8C 1 is a flow chart of a method 800 for monitoring input on an intensity-sensitive input element, detecting a long press with acceleration, and performing a corresponding operation if a long press is detected. The method 800 is performed in an electronic device (e.g., device 300, device 300) having a display, a touch-sensitive surface, and one or more sensors 165 for detecting intensity of contact with the touch-sensitive surface. Figure 3; or a portable multifunction device 100, Figure 1A ) is performed at. In some embodiments, the electronic device executing method 800 has a main button 204 including one of sensors 165 in addition to the touch-sensitive surface. In some embodiments, the main button 204 is separated from the display and optionally includes a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display. In some embodiments, the main button 204 is a virtual main button displayed on the display (for example, having a set of one or more intensity sensors separate from the intensity sensor for detecting the intensity of input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of input using the virtual main button). In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 800 are optionally combined, and / or the order of some operations is optionally changed.

[0392] As described below, method 800 provides a way to accurately determine user intent as to whether a touch input is a long press more quickly by considering the strength of the user's input. Method 800 reduces latency by identifying a long press input more quickly than would otherwise be possible, while avoiding "false positives" (such as input being incorrectly detected as a long press), thereby generating a more efficient human-machine interface. For battery-powered electronic devices, considering the strength of the user's touch input enables the user to input gestures such as long presses more quickly and efficiently, which saves power and increases the time between battery charges.

[0393] The device detects (802) an input sequence including an increase in the intensity of an input corresponding to a first input event (e.g., a press click event). In some embodiments, the input on the input element is or includes (804) an input on a touch-sensitive surface. In some embodiments, or in some cases, the increase in the intensity of the input is followed by a decrease in the intensity of the input, such as Figure 5BB The method 800 includes, in response to detecting the input sequence (806), performing (808) a first operation (e.g., a single-click operation) based on determining that a second input event (e.g., a release click event) including a decrease in the intensity of the input after the first input event is detected within a first time period after the first input event is detected. For example, Figure 5X An input is shown with an increase in intensity 532 followed by a decrease in intensity 534. If a second input event corresponding to a decrease occurs within a first period of time after the first input event is detected, a long press has not been detected and a first operation such as a single-click operation is performed.

[0394] Method 800 also includes determining that a second input event (e.g., an event corresponding to a reversal of the first input event, such as a release click event) has not been detected for a second time period longer than the first time period and the input has a characteristic intensity above a corresponding intensity threshold between when the first input event is detected and when the second time period elapses (e.g., Figure 5BB As shown, the characteristic intensity of the input is higher than the corresponding intensity threshold (for example, I D )), performing (809) a second operation (e.g., a long press operation) once a second time period has passed (e.g., in response to the second time period passing (e.g., earlier / faster than a normal long press operation)), wherein the second time period is determined at least in part based on the intensity of the input after the first input event is detected.

[0395] The method 800 also includes determining that the second input event is within a third time period longer than the second time period (e.g., Fig.5II the default time period) and the input does not have a characteristic intensity above the corresponding intensity threshold between when the first input event is detected and when the second time period has passed, once the third time period has passed (e.g., in response to the third time period passing, corresponding to Fig.5II Time T5 in Figure 5A , 5B and 5II) execute (810) a second operation (eg, a long press operation).

[0396] Optionally, method 800 includes determining that the second input event is not detected within the second time period and the input does not have a characteristic intensity above the corresponding intensity threshold between when the first input event is detected and when the second time period has passed (e.g., Fig.5II ), abandon (512) once the second time period has passed, the second operation (e.g., a long press operation) is performed, at least until a third time period (e.g., calling the above Fig.5II Therefore, in this case, because the strength of the input is not higher than the corresponding strength threshold, the long press operation is not accelerated.

[0397] In some embodiments, method 800 includes identifying (814) a first input event based on an increase in intensity of the input that satisfies a first intensity threshold and identifying a second input event based on a decrease in intensity of the input that satisfies a second intensity threshold that is different from the first intensity threshold. Figure 5X Shows that the click intensity threshold I is met D The intensity increases by 532, followed by the click intensity threshold I being met. D Different release click intensity thresholds IU In some embodiments, the corresponding intensity threshold is greater than the first intensity threshold (816) (see, for example, Figure 5Z and 5AA , the corresponding intensity threshold is an intensity threshold such as I D+ , which is greater than the click intensity threshold I D ), and a second intensity threshold (e.g., a release click intensity threshold I U ) is less than a first strength threshold (e.g., a press-click strength threshold I D ).

[0398] In some embodiments, method 800 includes monitoring (818) the duration of a timeout period starting from when the increase in intensity of the input satisfies the first intensity threshold, and comparing the duration of the timeout period to at least one of the first time period, the second time period, and the third time period. Figure 5BB , 5DD and the examples described in 5FF.

[0399] In some embodiments, method 800 includes stopping (820) monitoring the duration of the timeout period when the intensity of the input decreases to meet a second intensity threshold. Figure 5X If the input strength drops below the release click strength threshold I U , then monitoring of the duration of the timeout period is stopped.

[0400] In some embodiments, method 800 includes accelerating the rate at which the timeout period accumulates when the intensity of the input exceeds a first predefined intensity threshold (e.g., the corresponding intensity threshold), wherein the accelerated rate is higher than the default rate. Figure 5BB , 5DD As described in 5FF and 5FF, when the intensity of the input exceeds a first predefined intensity threshold (in those examples, a press click intensity threshold I D ), the rate at which timeout periods accumulate is an accelerated rate. In some embodiments, method 800 also includes decelerating (824) the rate at which timeout periods accumulate when the intensity of the input decreases.

[0401] In some embodiments, the third time period is (826) the longest duration of the timeout period before performing the second operation (e.g., a default timeout period, such as 500 ms). In some embodiments, monitoring the duration of the timeout period continues until it is determined that the second input event is detected or the duration of the timeout period is equal to the third time period (whichever occurs first), and the second time period includes the shortest duration of the timeout period before performing the second operation.

[0402] In some embodiments, the second time period is constrained (828) to at least a minimum duration (e.g., in some embodiments, the time for detecting a long press input does not decrease below 300 ms, regardless of how high the intensity of the input reaches). In some embodiments, the minimum duration (e.g., 300 ms) is longer than half of the maximum duration (500 ms).

[0403] In some embodiments, monitoring the duration of the timeout period includes decaying (830) the time value from an initial time value at a rate that varies according to the strength of the input. Figure 5BB , 5DD , 5FF and 5HH, and the discussion of those figures above.

[0404] In some embodiments, the first operation is or includes closing (832) an application, and the second operation includes displaying a virtual assistant user interface.

[0405] In some embodiments, the first operation is or includes (834) scrolling from one icon screen in the application launch user interface (e.g., an application launch user interface including a first set of application launch icons) to another icon screen in the application launch user interface (e.g., including a second set of application launch icons, the second set of application launch icons including application launch icons not in the first set of application launch icons), such as from Figures 5Q to 5R , and the second operation is or includes displaying a virtual assistant user interface (e.g., Figure 5BB , 5DD , 5FF and 5HH).

[0406] According to some embodiments, Fig.11 1 shows a functional block diagram of an electronic device 1100 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of various described embodiments. It should be understood by those skilled in the art that Fig.11 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0407] like Fig.11As shown, the electronic device 1100 includes a display unit 1002 (e.g., corresponding to the display 112) configured to display a user interface, an intensity-sensitive input unit 1104 (e.g., corresponding to the main button 204) configured to detect the intensity of the user input performed with the input element, and a processing unit 1110 coupled to the display unit 1102 and the intensity-sensitive input unit 1104. In some embodiments, the electronic device 1000 also includes a touch-sensitive surface unit 1106 for receiving a touch input on a surface (such as a display surface of the display unit 1102), and one or more tactile output units 1108 for generating tactile output, which are also coupled to the processing unit 1010. In some embodiments, the processing unit 1110 includes one or more of the following subunits: an intensity monitoring unit 1112, a long press determination unit 1114, a single click determination unit 1116, and a feedback unit 1118. In some embodiments, the feedback unit 1118 includes a response display unit 1120, and a tactile output unit 1122.

[0408] Processing unit 1110 is configured to: detect an input sequence (e.g., using intensity monitoring unit 912) including detecting an increase in intensity of an input corresponding to a first input event; and, in response to detecting the input sequence: perform a first operation based on determining (e.g., using single-click determination unit 1116) that a second input event (including a decrease in intensity of the input after the first input event) is detected within a first time period after the first input event is detected; perform a first operation based on determining (e.g., using single-click determination unit 1116) that the second input event is not detected within a second time period longer than the first time period and determining (e.g., using long press determination unit 1114) that the input is within the first time period The input event has a characteristic intensity higher than the corresponding intensity threshold between when the input event is detected and when a second time period has passed, and the second operation is performed once the second time period has passed, wherein the second time period is determined at least in part based on the intensity of the input after the first input event is detected; and based on (for example, using the single-click determination unit 1116) determining that the second input event is not detected within a third time period longer than the second time period and (for example, using the long press determination unit 1114) determining that the input does not have a characteristic intensity higher than the corresponding intensity threshold between when the first input event is detected and when the second time period has passed, the second operation is performed once the third time period has passed.

[0409] In some embodiments, the processing unit 1110 is further configured to abandon performing the second operation once the second time period has passed, based on determining (e.g., using the single-click determination unit 1116) that the second input event has not been detected within the second time period and determining (e.g., using the single-click determination unit 1116) that the input does not have a characteristic intensity greater than the corresponding intensity threshold between when the first input event is detected and when the second time period has passed, at least until a third time period has passed.

[0410] In some embodiments, input on the intensity-sensitive input unit includes input on a touch-sensitive surface.

[0411] In some embodiments, processing unit 1110 is further configured to identify a first input event based on an increase in the intensity of the input that satisfies a first intensity threshold and to identify a second input event based on a decrease in the intensity of the input that satisfies a second intensity threshold that is different from the first intensity threshold (e.g., using single-click determination unit 1116).

[0412] In some embodiments, the corresponding intensity threshold is greater than the first intensity threshold, and the second intensity threshold is less than the first intensity threshold. D (corresponding to the first intensity threshold) and the release click intensity threshold I U (corresponding to the first intensity threshold) Figure 5X Discussion.

[0413] In some embodiments, the processing unit 1110 is further configured to monitor the duration of the timeout period from when the intensity of the input increases to meet the first intensity threshold, and compare the duration of the timeout period with at least one of the first time period, the second time period, and the third time period (e.g., using the intensity monitoring unit 1112 and / or the long press determination unit 1114). In some embodiments, the processing unit 1110 is further configured to stop monitoring the duration of the timeout period when the intensity of the input decreases to meet the second intensity threshold. In some embodiments, the processing unit 1110 is further configured to accelerate the rate of accumulation of the timeout period when the intensity of the input exceeds the first predefined intensity threshold, wherein the accelerated rate is higher than the default rate. In some embodiments, the processing unit 1110 is further configured to decelerate the rate of accumulation of the timeout period when the intensity of the input decreases.

[0414] In some embodiments, the third time period includes a maximum duration of a timeout period before performing the second operation. In some embodiments, the second time period is constrained to at least a minimum duration.

[0415] In some embodiments, monitoring the duration of the timeout period includes decaying the time value from an initial time value at a rate that varies based on the strength of the input.

[0416] In some embodiments, the first operation is or includes closing the application, and the second operation includes displaying the virtual assistant user interface. In some other embodiments, the first operation is or includes launching an icon screen from an application launch user interface (e.g., an application launch user interface including a first set of application launch icons, an example of which is shown in FIG. Figure 5Q ) scrolls to another icon screen in the application launch user interface (e.g., an application launch user interface including a second set of application launch icons, the second set of application launch icons including application launch icons not included in the first set of application launch icons, an example of which is shown in Figure 5R ), and the second operation includes displaying a virtual assistant user interface (e.g., an example of which is shown in Figure 5BB , 5DD , and 5FF).

[0417] References Figures 8A-8C The operations in the information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0418] References Figure 8A-8C The operation is optionally performed by Figure 1A-1B or Figure 31. For example, detection operations 802, 808, etc. and identification and determination operations 808, 810, 812, etc. are optionally implemented by contact / motion module 130, feedback operations that provide tactile output are implemented by tactile feedback module 133, and some other operations are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 transmits event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another orientation. When a corresponding predefined event or sub-event is detected, event identifier 180 activates event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A to Figure 1B How the components depicted in the figure can implement other processes.

[0419] For the purpose of explanation, the above description has been described...

Claims

1. A method, include: At an electronic device having a display and a strength-sensitive input element, wherein the strength-sensitive input element is used to detect the strength of a user input performed with the strength-sensitive input element: detecting an input sequence including an increase in intensity of an input corresponding to a first input event; and In response to detecting the input sequence: performing a first operation in response to determining that a second input event including a decrease in intensity of the input after the first input event is detected within a first time period after the first input event is detected; performing a second operation once the second time period has elapsed based on determining that the second input event has not been detected for a second time period that is longer than the first time period and that the input had a characteristic intensity above a corresponding intensity threshold between when the first input event was detected and when the second time period elapsed, wherein the second time period is determined at least in part based on an intensity of the input after the first input event was detected; as well as Based on determining that the second input event is not detected within a third time period longer than the second time period and the input does not have a characteristic intensity above the corresponding intensity threshold between when the first input event is detected and when the second time period passes, performing the second operation once the third time period has passed.

2. The method according to claim 1, further comprising: include: Based on determining that the second input event is not detected within the second time period and the input does not have a characteristic intensity above the corresponding intensity threshold between when the first input event is detected and when the second time period passes, abandoning the performance of the second operation once the second time period has passed, at least until the third time period has passed. 3 . The method of claim 1 , wherein the input on the intensity-sensitive input element comprises an input on a touch-sensitive surface. The method of claim 1 , wherein the second time period is constrained to at least a minimum duration.

5. The method of claim 1, wherein the first operation comprises closing an application and the second operation comprises displaying a virtual assistant user interface.

6. The method of claim 1, wherein the first operation comprises scrolling from one icon screen in an application launch user interface to another icon screen in the application launch user interface, and the second operation comprises displaying a virtual assistant user interface.

7. The method of claim 1 , comprising identifying the first input event based on an increase in intensity of the input satisfying a first intensity threshold, and identifying the second input event based on a decrease in intensity of the input satisfying a second intensity threshold different from the first intensity threshold.

8. The method of claim 7, wherein the corresponding intensity threshold is greater than the first intensity threshold, and the second intensity threshold is less than the first intensity threshold.

9. The method of any one of claims 7 to 8, comprising beginning to monitor the duration of a timeout period when the increase in intensity of the input satisfies the first intensity threshold, and comparing the duration of the timeout period to at least one of the first time period, the second time period, and the third time period.

10. The method of claim 9, comprising ceasing the monitoring of the duration of the timeout period when the intensity of the input decreases to meet the second intensity threshold.

11. The method of claim 9, comprising accelerating the rate at which the timeout period accumulates when the intensity of the input exceeds a first predefined intensity threshold, wherein the accelerated rate is higher than a default rate.

12. The method of claim 11, comprising slowing down the rate at which the timeout periods accumulate as the intensity of the input decreases.

13. The method of claim 9, wherein the third time period comprises a maximum duration of the timeout period before performing the second operation.

14. The method of claim 9, wherein monitoring the duration of the timeout period comprises decaying a time value starting from an initial time value at a rate that varies according to the strength of the input.

15. An electronic device, include: monitor; an intensity-sensitive input element for detecting intensity of a user input made with said intensity-sensitive input element; one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method of any one of claims 1-14.

16. A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions that, when executed by an electronic device having a display and an intensity-sensitive input element, cause the electronic device to perform the method described in any one of claims 1-14, wherein the intensity-sensitive input element is used to detect the intensity of user input performed using the intensity-sensitive input element.

Images