Device, method, and graphical user interface for providing feedback during interaction with intensity-sensing buttons

Abstract

To provide devices, methods, and graphical user interfaces for providing feedback during interaction with an intensity-sensitive button.SOLUTION: An electronic device is configured to: display a home button configuration user interface having a plurality of different tactile output settings for a home button; while displaying the home button configuration user interface, detect selection of a respective tactile output setting; in accordance with detecting a first input of a first type on the home button, determine whether the respective tactile output setting is a first tactile output setting or a second tactile output setting; if the tactile output setting is the first tactile output setting, provide a first tactile output without dismissing the home button configuration user interface; and, if the respective tactile output setting is a second tactile output setting, provide a second tactile output without dismissing the home button configuration user interface.SELECTED DRAWING: Figure 5A13

Description

[Technical Field]

[0001] The present invention relates in general to an electronic device having a display and intensity-sensitive buttons, and includes, but is not limited to, an electronic device having a display and intensity-sensitive home buttons. [Background technology]

[0002] Many electronic devices with displays include mechanical buttons, such as a mechanical home button, to navigate between different user interfaces displayed on the device. However, mechanical buttons provide little, if any, feedback to the user beyond a fixed down-click and fixed up-click. [Overview of the project]

[0003] Therefore, there is a need for improved methods and interfaces for providing visual, haptic, and / or auditory feedback during button interaction in electronic devices, thereby enabling more efficient and intuitive operation of user interfaces. Such methods and interfaces can optionally complement or replace conventional methods of providing feedback during button interaction. Such methods and interfaces reduce the number, extent, and / or types of user input by helping the user understand the connection between the input provided and the device response to that input, thereby generating a more efficient human-machine interface.

[0004] The above-mentioned drawbacks and other problems related to user interfaces for electronic devices having buttons are mitigated or eliminated by the disclosed device, which includes one or more intensity-sensitive buttons. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device such as a watch). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a “touchscreen” or “touchscreen display”). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger touch and gestures on the touch-sensitive surface. In some embodiments, the functions are optional and include image editing, drawing, presentation, word processing, spreadsheet creation, gameplay, making phone calls, video conferencing, sending emails, instant messaging, training support, digital photography, digital videography, web browsing, digital music playback, note-taking, and / or digital video playback. The executable instructions for performing those functions are optional and may be contained in a non-temporary computer-readable storage medium or in other computer program products configured to be executed by one or more processors.

[0005] In some embodiments, the method is performed in an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting the intensity of contact with a home button of the electronic device. The method includes displaying a first user interface on the display, and detecting an input indicated on the first user interface while the first user interface is being displayed. The method includes stopping the display of the first user interface and displaying a second user interface separate from the first user interface in response to the detection of an input indicated on the first user interface. The method includes detecting contact on the home button while the second user interface is being displayed. The method includes detecting an input sequence, which includes (i) detecting an increase in the characteristic intensity of the contact that exceeds a first intensity threshold while continuously detecting contact on the home button, (ii) indicating the start of a transition from a second user interface back to the first user interface in response to the detection of an increase in the characteristic intensity of the contact relative to the first intensity threshold, and (iii) detecting a press input that includes an increase in the characteristic intensity of the contact that exceeds a first intensity threshold while indicating the start of a transition from the second user interface back to the first user interface. In response to the detection of an input sequence, the method includes (i) determining that the input sequence satisfies a first criterion (for example, a first criterion requiring that the characteristic intensity of the contact increases above a second intensity threshold before the end of the press input is detected), (A) stopping the display of the second user interface, and (B) redisplaying the first user interface on the display; and (ii) determining that the input sequence satisfies a second criterion (for example, a second criterion requiring that the characteristic intensity of the contact remains below a second intensity threshold before the end of the press input is detected), (A) reversing the start of the transition from the second user interface back to the first user interface, and (B) redisplaying the second user interface on the display.

[0006] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch-sensing surface unit for receiving contact, one or more sensor units for detecting contact intensity with the device's home button, a display unit, a touch-sensing surface unit, and a processing unit coupled to one or more sensor units. The processing unit displays a first user interface on the display, detects inputs indicated to the first user interface while displaying the first user interface, and in response to detecting inputs indicated to the first user interface, (i) stops displaying the first user interface, (ii) displays a second user interface separate from the first user interface, detects contact on the home button while displaying the second user interface, and while continuously detecting contact on the home button, (i) detects an increase in the characteristic intensity of the contact exceeding a first intensity threshold, (ii) in response to detecting an increase in the characteristic intensity of the contact relative to the first intensity threshold, displays the start of a transition from the second user interface back to the first user interface, and (iii) displays the start of a transition from the second user interface back to the first user interface. During this time, the system detects an input sequence that includes detecting a pressing input, which includes an increase in the characteristic intensity of the contact that exceeds a first intensity threshold. In response to the detection of the input sequence, the system is configured to: (i) if the input sequence satisfies a first criterion, which requires that the characteristic intensity of the contact increases above a second intensity threshold before the end of the pressing input is detected, (A) stop displaying the second user interface and (B) redisplay the first user interface on the display; and (ii) if the input sequence satisfies a second criterion, which requires that, for example, the characteristic intensity of the contact remains below a second intensity threshold before the end of the pressing input is detected, (A) reverse the start of the transition from the second user interface back to the first user interface and (B) redisplay the second user interface on the display.

[0007] According to some embodiments, the method is performed in an electronic device having a display and a touch-sensitive surface. The method includes displaying a first user interface and detecting a first input (e.g., a first input matching a request to display a second user interface and a request to display a third user interface) while the first user interface is being displayed. The method includes, in response to detecting the first input, initiating the display of a first animation transition from the first user interface to the second user interface. The method includes detecting a second input while the first animation transition is being displayed. In response to the detection of a second input, the method includes (i) determining that the second input matches a request to display a third user interface and that the second input was received at a first time, (A) interrupting the first animation transition from the first user interface to the second user interface at a first point in the first animation transition, and (B) displaying the second animation transition from the first point in the first animation transition to the third user interface; and (ii) determining that the second input matches a request to display a third user interface and that the second input was received at a second time after the first time, (A) interrupting the first animation transition from the first user interface to the second user interface at a second point in the first animation transition, after the first point in the first animation transition, and (B) displaying a third animation transition different from the second animation transition, from the second point in the first animation transition to the third user interface.

[0008] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch-sensing surface unit configured to receive contact, and a processing unit coupled to the display unit and the touch-sensing surface unit. The processing unit displays a first user interface, detects a first input that matches a request to display a second user interface and a request to display a third user interface while the first user interface is being displayed, and in response to the detection of the first input, starts displaying a first animation transition from the first user interface to the second user interface, detects a second input while the first animation transition is being displayed, and in response to the detection of the second input, (i) in accordance with the determination that the second input matches a request to display a third user interface and the second input was received at a first time, (A) at a first point in the first animation transition, transitions from the first user interface to the second user interface The system is configured to interrupt the first animation transition to the interface, (B) display the second animation transition from the first point in the first animation transition to the third user interface, and (ii) according to the determination that the second input matches the request to display the third user interface and the second input was received at a second time after the first time, (A) interrupt the first animation transition from the first user interface to the second user interface at the second point in the first animation transition, after the first point in the first animation transition, and (B) display a third animation transition different from the second animation transition, from the second point in the first animation transition to the third user interface.

[0009] According to some embodiments, the method is performed in an electronic device having a display, a touch-sensing surface, and one or more tactile output generators that produce tactile output. The method includes displaying a home button configuration user interface on the display, which includes displaying a plurality of different tactile output settings for a home button, the home button being available on the device in a plurality of different contexts to discard the currently displayed user interface in response to detecting a first type of input on the home button. The method includes detecting the selection of each of the plurality of different tactile output settings for the home button while the home button configuration user interface is displayed. The method includes detecting a first input of a first type on the home button while each of the tactile output settings for the home button is selected. The method includes, upon detection of a first input of a first type on the home button, (i) providing a first tactile output corresponding to a first tactile output setting for the home button via one or more tactile output generators without discarding the home button configuration user interface, in accordance with the determination that each tactile output setting is a first tactile output setting for the home button; and (ii) providing a second tactile output corresponding to a second tactile output setting for the home button via one or more tactile output generators without discarding the home button configuration user interface, in accordance with the determination that each tactile output setting is a second tactile output setting for the home button that is different from the first tactile output setting.

[0010] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch-sensing surface unit that accepts contact, one or more tactile output generators that produce tactile outputs, and a processing unit coupled to the display unit, the touch-sensing surface unit, and one or more tactile output generators. The processing unit causes a home button configuration user interface to be displayed on the display, which includes displaying a plurality of different tactile output settings for the home button, and the home button is available on the device in a plurality of different contexts to discard the currently displayed user interface in response to detecting a first type of input on the home button, and while the home button configuration user interface is displayed, the selection of each of the plurality of different tactile output settings for the home button is detected, and while each of the tactile output settings for the home button is selected, the first input of the first type on the home button is detected, and the first on the home button In response to the detection of a first input of type, the system is configured to (i) provide a first tactile output corresponding to the first tactile output setting for the home button via one or more tactile output generators, without discarding the home button configuration user interface, in accordance with the determination that each tactile output setting is a first tactile output setting for the home button, and (ii) provide a second tactile output corresponding to the second tactile output setting for the home button via one or more tactile output generators, without discarding the home button configuration user interface, in accordance with the determination that each tactile output setting is a second tactile output setting for the home button that is different from the first tactile output setting.

[0011] According to some embodiments, the method is performed in an electronic device having a display, a touch-sensing surface, and one or more tactile output generators that produce tactile outputs. The method includes displaying a first user interface on the display, wherein a home button is available on the device in several different contexts to discard the currently displayed user interface in response to detection of a first type of input on the home button. The method includes detecting a first input of a first type indicated on the first user interface while the first user interface is being displayed. In response to detecting a first input of a first type indicated on the first user interface, the method includes (i) performing a first action via one or more tactile output generators and providing a first tactile output corresponding to the first action, in accordance with a determination that user interface tactile outputs are enabled on the electronic device, and (ii) ceasing to perform a first action and provide at least a portion of a first tactile output corresponding to the first action, in accordance with a determination that user interface tactile outputs are disabled. The method includes detecting a second input of a first type on the home button after performing the first action. Upon detecting a second input of a first type on the home button, (i) perform a second action associated with the home button, and (ii) provide a tactile output associated with activating the home button via one or more tactile output generators, regardless of whether tactile output of the user interface is enabled on the device.

[0012] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch-sensing surface unit that accepts contact, one or more tactile output generators that produce tactile outputs, and a processing unit coupled to the display unit, the touch-sensing surface unit, and one or more tactile output generators. The processing unit displays a first user interface on the display, and the home button is available on the device in several different contexts to discard the currently displayed user interface in response to detecting a first type of input on the home button, and while displaying the first user interface, it detects a first input of a first type indicated on the first user interface, and in response to detecting a first input of a first type indicated on the first user interface, (i) performs a first operation in accordance with the determination that a user interface tactile output has been activated in the electronic device, and generates one or more tactile outputs. The device is configured to: (ii) provide a first tactile output corresponding to a first operation via a device; (ii) perform the first operation in accordance with the determination that user interface tactile output is disabled, and cease providing at least a portion of the first tactile output corresponding to the first operation; after performing the first operation, detect a second input of a first type on the home button; and, in response to the detection of a second input of a first type on the home button, (i) perform a second operation related to the home button; and (ii) provide a tactile output related to the activation of the home button via one or more tactile output generators, regardless of whether user interface tactile output is enabled on the device.

[0013] According to some embodiments, the method is performed in an electronic device having a display, a touch-sensing surface, one or more tactile output generators that generate tactile outputs, and one or more sensors for detecting the intensity of contact with the device's home button. The method includes displaying a user interface on the display and detecting an input sequence on the home button, including detecting a first press input on the home button while the user interface is being displayed. Detecting the first press input includes detecting an increase in the characteristic intensity of contact on the home button. In response to detecting the first press input on the home button, (A) perform a first action to change the user interface displayed on the display, according to the determination that (i) the first press input includes an increase in the characteristic intensity of contact exceeding a first intensity threshold, and that a change in the characteristic intensity of contact close to the time when the characteristic intensity of contact increases above the first intensity threshold has a first value of the intensity change metric; (B) generate a first discrete tactile output via one or more tactile output generators corresponding to the increase in the characteristic intensity of contact exceeding the first intensity threshold; and (ii) In accordance with the determination that a pressing input of 1 includes an increase in the characteristic intensity of the contact exceeding a first intensity threshold, and that a change in the characteristic intensity of the contact close to the time when the characteristic intensity of the contact increases above the first intensity threshold has a second value of the intensity change metric that is different from a first value of the intensity change metric, (A) perform a first action to change the user interface displayed on the display, and (B) generate a second discrete tactile output via one or more tactile output generators that corresponds to the increase in the characteristic intensity of the contact exceeding the first intensity threshold and is different from the first discrete tactile output.

[0014] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch sensing surface unit for receiving contact, one or more tactile output generators for generating tactile output, and one or more sensor units for detecting the intensity of contact with the device's home button, and a processing unit coupled to the display unit, the touch sensing surface unit, one or more tactile output generators, and one or more sensor units. The processing unit displays a user interface on the display, and while displaying the user interface, it detects an input sequence on the home button, including detecting a first press input on the home button, which includes detecting an increase in the characteristic intensity of contact on the home button, and in response to detecting a first press input on the home button, (i) the first press input includes an increase in the characteristic intensity of contact exceeding a first intensity threshold, and the change in the characteristic intensity of contact approaching the time when the characteristic intensity of contact increases above the first intensity threshold has a first value of the intensity change metric, and (A) the user interface displayed on the display The system is configured to perform a first operation to change the interface, (B) generate a first discrete tactile output via one or more tactile output generators corresponding to an increase in the characteristic intensity of contact exceeding a first intensity threshold, and (II) according to the determination that a first pressing input includes an increase in the characteristic intensity of contact exceeding a first intensity threshold, and that a change in the characteristic intensity of contact close to the time when the characteristic intensity of contact increases above the first intensity threshold has a second value of the intensity change metric that is different from a first value of the intensity change metric, (A) perform a first operation to change the user interface displayed on the display, and (B) generate a second discrete tactile output via one or more tactile output generators that corresponds to an increase in the characteristic intensity of contact exceeding a first intensity threshold and is different from the first discrete tactile output.

[0015] According to some embodiments, the method is performed in an electronic device having a display, a touch-sensing surface, one or more tactile output generators that produce tactile outputs, and one or more sensors for detecting the intensity of contact with the device's home button. The method includes displaying a first user interface. While displaying the first user interface, the method detects (i) a first press input on the home button, and (ii) a second press input on the home button detected after the first press input. In response to the detection of the first press input and before the detection of the second press input, a first non-visual output having a first non-visual output profile is provided. The first non-visual output provides feedback indicating that a first press input has been detected and that the first non-visual output includes tactile outputs provided by one or more tactile output generators. The method, in response to detecting an input sequence including a first press input and a second press input on a home button, (i) based on the amount of time between a first time point corresponding to the first press input and a second time point corresponding to the second press input, determines that the first press input and the second press input are separate inputs, (A) performs a first operation related to the first press input, and (B) provides a second non-visual output having a first non-visual output profile, wherein the second non-visual output is a feed indicating that the second press input has been detected and that the second non-visual output includes a tactile output provided by one or more tactile output generators. The means of providing feedback includes (ii) based on the amount of time between a first time point and a second time point, in accordance with the determination that the first and second pressing inputs are part of an input pattern, (A) performing a second operation related to the input pattern, which is different from the first operation, and (B) providing a third non-visual output having a second non-visual output profile separate from the first non-visual output profile, wherein the third non-visual output provides feedback indicating that the second pressing input has been detected and that the third non-visual output includes tactile outputs provided by one or more tactile output generators.

[0016] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch-sensing surface unit for receiving contact, one or more tactile output generating units for generating tactile output, one or more sensor units for detecting the intensity of contact with the device's home button, and a processing unit coupled to the display unit, the touch-sensing surface unit, one or more tactile output generators, and one or more sensor units.The processing unit displays a first user interface, and while displaying the first user interface, it detects (i) a first press input on the home button, and (ii) a second press input on the home button detected after the first press input, and in response to the detection of the first press input and before the detection of the second press input, it provides a first non-visual output having a first non-visual output profile, and provides feedback indicating that the first non-visual output has detected the first press input and that the first non-visual output includes a tactile output provided by one or more tactile output generators, and in response to the detection of an input sequence including the first and second press inputs on the home button, it determines that the first and second press inputs are separate inputs based on the amount of time between a first time point corresponding to the first press input and a second time point corresponding to the second press input, and then (A) first The system is configured to perform a first operation related to a pressing input, (B) provide a second non-visual output having a first non-visual output profile, the second non-visual output providing feedback indicating that a second pressing input has been detected and that the second non-visual output includes tactile output provided by one or more tactile output generators, and (II) based on the amount of time between a first time point and a second time point, in accordance with the determination that the first and second pressing inputs are part of an input pattern, (A) perform a second operation related to an input pattern, which is different from the first operation, and (B) provide a third non-visual output having a second non-visual output profile separate from the first non-visual output profile, the third non-visual output providing feedback indicating that a second pressing input has been detected and that the third non-visual output includes tactile output provided by one or more tactile output generators.

[0017] According to some embodiments, the method is performed in an electronic device having a display, a touch-sensitive surface, one or more tactile output generators that produce tactile outputs, and one or more sensors for detecting the intensity of contact with each of the buttons of the device. The method includes detecting input on each of the buttons, wherein one or more tactile output generators are used to produce tactile outputs indicating that each button has been activated, instead of mechanical switches that detect the activation of each button when each button is mechanically pressed. The method includes, in response to detecting an input on each button, (i) in accordance with the determination that the input satisfies an activation criterion, which includes the requirement that in order to satisfy the activation criterion, the input contains an intensity exceeding the respective intensity threshold, (A) providing a first tactile output having a first tactile output pattern that includes vibrations of 0.5 to 4 cycles of one or more tactile output generators for one or more corresponding neutral positions of one or more tactile output generators, wherein the vibrations of one or more tactile output generators occur at a frequency of 80 Hz to 400 Hz, and (ii) ceasing to provide the first tactile output in accordance with the determination that the input does not satisfy the activation criterion.

[0018] According to some embodiments, the electronic device includes a display unit configured to display a user interface, a touch-sensing surface unit for receiving contact, one or more tactile output generating units for generating tactile output, one or more sensor units for detecting the intensity of contact with the device's home button, and a processing unit coupled to the display unit, the touch-sensing surface unit, one or more tactile output generators, and one or more sensor units. The processing unit is configured to detect input on each button, and one or more tactile output generators are used to generate a tactile output indicating that each button has been activated, instead of mechanical switches that detect the activation of each button when each button is mechanically pressed. Depending on the detection of input to each button, (i) if the input satisfies an activation criterion, which includes the requirement that the input contains an intensity exceeding the respective intensity threshold in order to satisfy the activation criterion, then a first tactile output is provided having a first tactile output pattern that includes vibrations of 0.5 to 4 cycles of one or more tactile output generators for one or more corresponding neutral positions of one or more tactile output generators, wherein the vibrations of one or more tactile output generators occur at a frequency of 80 Hz to 400 Hz; and (ii) if the input does not satisfy the activation criterion, then the provision of the first tactile output is stopped.

[0019] According to some embodiments, the electronic device includes a display, a touch-sensitive surface, one or more sensors that optionally detect the intensity of contact with the touch-sensitive surface, intensity-sensitive buttons (e.g., virtual or physical home buttons), one or more processors, memory, and one or more programs, the one or more programs being stored in memory and configured to be executed by the one or more processors, and the one or more programs including instructions to perform or cause to perform any of the operations described herein. According to some embodiments, a computer-readable storage medium internally stores instructions, which, when executed by an electronic device comprising a display, a touch-sensitive surface, intensity-sensitive buttons (e.g., virtual or physical buttons), and one or more sensors that optionally detect the intensity of contact with the touch-sensitive surface, cause the device to perform or provide performance of any of the operations described herein. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, intensity-sensitive buttons (e.g., virtual or physical home buttons), memory, and one or more processors for executing one or more programs stored in memory, includes one or more elements displayed in any of the methods described herein, the elements being updated in response to input as described in any of the methods described herein. According to some embodiments, the electronic device includes a display, a touch-sensitive surface, intensity-sensitive buttons (e.g., virtual or physical home buttons), optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, and means for performing or causing to perform any of the operations described herein. According to some embodiments, an information processing device used in an electronic device having a display, a touch-sensitive surface, intensity-sensitive buttons (e.g., virtual or physical home buttons), and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, includes means for performing or causing to perform any of the operations described herein.

[0020] Thus, an electronic device having a display, a touch-sensitive surface, one or more sensors for detecting the intensity of contact with a device button (e.g., a virtual or physical home button), optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, one or more tactile output generators, optionally one or more device orientation sensors, and optionally a voice system, provides an improved method and interface for providing feedback to the user during interaction with the buttons, thereby enhancing the effectiveness, efficiency, and user satisfaction of such a device. Such methods and interfaces can complement or replace conventional methods for providing haptic feedback to the user.

[0021] To better understand the various embodiments described, the following “Modes for Carrying Out the Invention” should be referenced in conjunction with the following drawings. Here, similar reference numerals refer to the corresponding parts throughout those drawings. [Brief explanation of the drawing]

[0022] [Figure 1A] This is a block diagram showing a portable multifunctional device having a touch-sensitive display, according to one embodiment. [Figure 1B] This is a block diagram showing exemplary components for event handling according to some embodiments. [Figure 1C] This is a block diagram of a tactile output module according to one embodiment. [Figure 2A] This figure shows a portable multifunctional device having a touchscreen, according to one embodiment. [Figure 2B] This is an exploded view of a force sensing input device according to one embodiment. [Figure 2C] This is an exploded view of a force sensing input device according to one embodiment. [Figure 3] This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface, according to one embodiment. [Figure 4A]An exemplary user interface for an application menu on a portable multifunction device, according to one embodiment, is shown. [Figure 4B] An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display, according to one embodiment, is shown. [Figure 4C] This figure shows an example of a dynamic intensity threshold according to one embodiment. [Figure 4D] This figure shows an example of a dynamic intensity threshold according to one embodiment. [Figure 4E] This figure shows an example of a dynamic intensity threshold according to one embodiment. [Figure 4F1] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4F2] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4F3] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4F4] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4G1] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4G2] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4G3] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4G4] This figure shows a set of sample tactile output patterns according to one embodiment. [Figure 4H] This figure shows an exemplary haptic audio output pattern versus time, used in conjunction with haptic output to simulate a button click, according to one embodiment. [Figure 4I] This figure shows an exemplary haptic audio output pattern versus time, used in conjunction with haptic output to simulate a button click, according to one embodiment. [Figure 4J] This figure shows an exemplary haptic audio output pattern versus time, used in conjunction with haptic output to simulate a button click, according to one embodiment. [Figure 4K] This figure shows an exemplary combination of tactile output patterns and tactile audio output patterns against time, according to one embodiment. [Figure 4L] Figure 4K is an enlarged view to clarify the combinations shown. [Figure 4M] Figure 4K is an enlarged view to clarify the combinations shown. [Figure 4N] Figure 4K is an enlarged view to clarify the combinations shown. [Figure 4O] Figure 4K is an enlarged view to clarify the combinations shown. [Figure 4P] Figure 4K is an enlarged view to clarify the combinations shown. [Figure 4Q] Figure 4K is an enlarged view to clarify the combinations shown. [Figure 5A1] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A2] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A3] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A4] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A5] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A6]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A7] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A8] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A9] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A10] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A11] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A12] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A13] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A14] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A15] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A16]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A17] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5A18] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B1] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B2] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B3] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B4] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B5] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B6] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B7] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B8]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B9] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B10] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B11] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B12] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B13] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B14] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B15] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B16] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B17] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B18]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B19] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B20] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B21] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B22] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B23] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B24] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B25] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B26] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B27] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B28]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B29] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B30] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B31] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B32] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B33] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B34] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B35] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B36] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B37] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B38]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B39] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B40] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B41] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B42] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B43] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B44] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B45] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B46] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B47] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B48]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B49] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B50] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B51] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B52] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B53] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B54] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B55] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B56] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B57] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B58]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B59] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B60] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B61] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B62] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B63] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B64] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B65] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B66] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B67] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B68]This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B69] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B70] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B71] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B72] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B73] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B74] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5B75] This figure shows an exemplary user interface for providing haptic and visual feedback for interacting with buttons, according to one embodiment. [Figure 5C1] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C2] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C3] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C4] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C5] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C6] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C7] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C8] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C9] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C10] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C11] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C12] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C13] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C14] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C15] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C16] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C17]This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C18] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 5C19] This figure shows an exemplary user interface for a home button configuration process according to one embodiment. [Figure 6A1] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A2] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A3] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A4] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A5] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A6] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A7] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A8] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A9] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A10] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A11] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A12] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A13] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A14] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A15] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A16] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A17] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A18] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A19] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A20] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A21]This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A22] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A23] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A24] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A25] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6A26] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B1] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B2] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B3] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B4] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B5] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B6]This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B7] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B8] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B9] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B10] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B11] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B12] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B13] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B14] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B15] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B16] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B17]This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B18] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B19] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B20] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B21] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B22] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B23] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B24] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B25] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 6B26] This document shows an exemplary user interface for controlling user interface haptics and home button haptics according to some embodiments. [Figure 7A] This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 7B]This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 7C] This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 7D] This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 7E] This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 7F] This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 7G] This flowchart illustrates a method for providing visual feedback regarding the activation of a user input device, according to one embodiment. [Figure 8] This is a functional block diagram of an electronic device according to one embodiment. [Figure 9A] This flowchart illustrates a method, according to one embodiment, for providing visual feedback and tactile output in response to multiple types of input on the home button of a device. [Figure 9B] This flowchart illustrates a method, according to one embodiment, for providing visual feedback and tactile output in response to multiple types of input on the home button of a device. [Figure 9C] This flowchart illustrates a method, according to one embodiment, for providing visual feedback and tactile output in response to multiple types of input on the home button of a device. [Figure 9D] This flowchart illustrates a method, according to one embodiment, for providing visual feedback and tactile output in response to multiple types of input on the home button of a device. [Figure 10] This is a functional block diagram of an electronic device according to one embodiment. [Figure 11A]This is a flowchart illustrating a method for configuring a home button according to one embodiment. [Figure 11B] This is a flowchart illustrating a method for configuring a home button according to one embodiment. [Figure 11C] This is a flowchart illustrating a method for configuring a home button according to one embodiment. [Figure 11D] This is a flowchart illustrating a method for configuring a home button according to one embodiment. [Figure 11E] This is a flowchart illustrating a method for configuring a home button according to one embodiment. [Figure 12] This is a functional block diagram of an electronic device according to one embodiment. [Figure 13A] This flowchart illustrates a method for controlling user interface haptics and home button haptics according to some embodiments. [Figure 13B] This flowchart illustrates a method for controlling user interface haptics and home button haptics according to some embodiments. [Figure 13C] This flowchart illustrates a method for controlling user interface haptics and home button haptics according to some embodiments. [Figure 13D] This flowchart illustrates a method for controlling user interface haptics and home button haptics according to some embodiments. [Figure 14] This is a functional block diagram of an electronic device according to one embodiment. [Figure 15A] This flowchart illustrates a method, according to some embodiments, for generating tactile outputs having different tactile output patterns that depend on input-based metrics or user interface-based metrics. [Figure 15B] This flowchart illustrates a method, according to some embodiments, for generating tactile outputs having different tactile output patterns that depend on input-based metrics or user interface-based metrics. [Figure 15C]This flowchart illustrates a method, according to some embodiments, for generating tactile outputs having different tactile output patterns that depend on input-based metrics or user interface-based metrics. [Figure 15D] This flowchart illustrates a method, according to some embodiments, for generating tactile outputs having different tactile output patterns that depend on input-based metrics or user interface-based metrics. [Figure 15E] This flowchart illustrates a method, according to some embodiments, for generating tactile outputs having different tactile output patterns that depend on input-based metrics or user interface-based metrics. [Figure 16] This is a functional block diagram of an electronic device according to one embodiment. [Figure 17A] This flowchart illustrates a method, according to some embodiments, for providing a tactile output for a second click of a double-click input, which is different from that for a first click of a double-click input. [Figure 17B] This flowchart illustrates a method, according to some embodiments, for providing a tactile output for a second click of a double-click input, which is different from that for a first click of a double-click input. [Figure 17C] This flowchart illustrates a method, according to some embodiments, for providing a tactile output for a second click of a double-click input, which is different from that for a first click of a double-click input. [Figure 17D] This flowchart illustrates a method, according to some embodiments, for providing a tactile output for a second click of a double-click input, which is different from that for a first click of a double-click input. [Figure 18] This is a functional block diagram of an electronic device according to one embodiment. [Figure 19A] This flowchart illustrates a method for providing discrete tactile outputs to indicate the activation of a persistent, non-mechanical button on a device, according to one embodiment. [Figure 19B]This flowchart illustrates a method for providing discrete tactile outputs to indicate the activation of a persistent, non-mechanical button on a device, according to one embodiment. [Figure 19C] This flowchart illustrates a method for providing discrete tactile outputs to indicate the activation of a persistent, non-mechanical button on a device, according to one embodiment. [Figure 20] This is a functional block diagram of an electronic device according to one embodiment. [Modes for carrying out the invention]

[0023] Many electronic devices with displays include mechanical buttons, such as mechanical home buttons, to navigate between different user interfaces displayed on the electronic device. However, mechanical buttons provide little, if any, feedback to the user beyond fixed down-clicks and fixed up-clicks. The methods described herein provide visual, haptic, and / or auditory feedback during interaction with buttons (e.g., virtual or physical home buttons) to make the operation of the user interface more efficient and intuitive for the user.

[0024] The methods, devices, and GUIs described herein use feedback to improve the interaction between the device and the user in multiple ways. ● To provide visual feedback regarding the activation of user input devices (e.g., virtual or physical home buttons), ● To provide visual feedback and tactile output in response to multiple types of input on the device's home button, ●Configuring the feedback provided by the home button, ● To control the user interface haptics and home button haptics, ●Generating tactile outputs with different tactile output patterns depending on input-based metrics or user interface-based metrics, ● To provide a tactile output for the second click of a double-click input, which is different from the output for the first click of a double-click input. ●Includes providing discrete tactile outputs to indicate the activation of a persistent, non-mechanical button on a device.

[0025] Example device Herein, we refer in detail to embodiments shown in the accompanying drawings. The following detailed description includes numerous specific details to provide a complete understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other examples, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the aspects of the embodiments.

[0026] In this specification, terms such as "first," "second," etc., are used to describe various elements in some embodiments, but it will be understood that these elements should not be limited by those terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first and second contacts are contacts, but they are not the same contact unless the context explicitly indicates otherwise.

[0027] The terms used in the descriptions of the various embodiments described herein are intended solely to describe specific embodiments and not to limit them. As used in the descriptions of the various embodiments and in the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural form unless the context explicitly indicates otherwise. As used herein, the terms “and / or” should also be understood to refer to and include all possible combinations of one or more of the enumerated items relating to the description. It will be further understood that, as used herein, the terms “includes,” “including,” “comprises,” and / or “comprising,” when used herein, specify the presence of the described features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0028] As used herein, the term "if" is interpreted, optionally and contextually, to mean "when," "upon," "in response to determining," or "in response to detecting." Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are interpreted, optionally and contextually, to mean "upon determining," "in response to determining," "upon detecting (the stated condition or event)," or "in response to detecting (the stated condition or event)."

[0029] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, which also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptop or tablet computers having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad), are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device but a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad).

[0030] The following discussion describes electronic devices including displays and touch-sensitive surfaces. However, please understand that the definition of an electronic device is optional and may include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0031] The device generally supports a variety of applications, including note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0032] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, optionally adjust and / or modify from one application to another, and / or within each application. Thus, the device's common physical architecture (such as the touch-sensitive surface) optionally supports a variety of applications with an intuitive and transparent user interface for the user.

[0033] Here, we focus on embodiments of portable devices having a touch-sensitive display. Figure 1A is a block diagram of a portable multifunction device 100 having a touch-sensitive display system 112 according to one embodiment. The touch-sensitive display system 112 may be referred to as a “touchscreen” for convenience, or simply as a touch-sensitive display. Device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on device 100 (for example, on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100). Device 100 includes one or more tactile output generators 167 for generating tactile output on Device 100 (for example, generating tactile output on a touch-sensitive surface such as the touch-sensitive display system 112 of Device 100 or the touchpad 355 of Device 300). These components optionally communicate through one or more communication buses or signal lines 103.

[0034] As used herein and in the claims, the term “tactile output” means a physical displacement of a device relative to its previous position, a physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of gravity of a device, as detected by the user through the user’s sense of touch. For example, in a situation where a device or component of a device is in contact with a touch-sensitive user’s surface (e.g., the user’s fingers, palm, or other part of their hand), the tactile output generated by the physical displacement is interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a “down-click” or “up-click” of a physical actuator button. In some cases, the user may feel a tactile sensation such as a “down-click” or “up-click” even when there is no movement of a physical actuator button associated with a touch-sensitive surface that is physically pressed (e.g., displaced) by the user’s movement. In another embodiment, the movement of the touch-sensitive surface is optional, and even when there is no change in the smoothness of the touch-sensitive surface, it is interpreted or perceived by the user as "roughness" of the touch-sensitive surface. While such user interpretations of touch depend on the user's personal sensory perception, there are many touch sensory perceptions common to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., "up-click," "down-click," "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the device or its components that produce the described sensory perception of a typical (or average) user.Using tactile output to provide tactile feedback to the user enhances device usability, makes the user-device interface more efficient (for example, by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and, in addition, reduces power consumption of the device and improves battery life by enabling the user to use the device more quickly and efficiently.

[0035] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the moving waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.

[0036] When a device generates tactile outputs with different tactile output patterns (for example, via one or more tactile output generators that move a movable mass to generate tactile outputs), the tactile outputs can evoke different tactile sensations when the user holds or touches the device. While the user's sensations are based on their perception of the tactile output, most users are able to distinguish changes in the waveform, frequency, and amplitude generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different actions have been performed. In this way, tactile outputs having tactile output patterns designed, selected, and / or devised to simulate the properties (e.g., size, material, weight, stiffness, smoothness, etc.), behavior (e.g., vibration, displacement, acceleration, rotation, expansion, etc.), and / or interaction (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or any combination thereof) provide the user with useful feedback that, in some situations, reduces input errors in the user's operation of the device and increases efficiency. In addition, tactile outputs are optionally generated to correspond to feedback unrelated to simulated physical properties, such as input thresholds or object selection. Such tactile outputs provide the user with useful feedback that, in some situations, reduces input errors in the user's operation of the device and increases efficiency.

[0037] In some embodiments, tactile output having an appropriate tactile output pattern serves as a cue for the occurrence of a target event behind a scene in the user interface or device. Examples of target events include the activation of affordances provided on the device or within the user interface (e.g., physical buttons, virtual buttons, or toggle switches), the success or failure of a requested action, reaching or crossing a boundary within the user interface, entering a new state, switching the focus of input between objects, activating a new mode, reaching or exceeding an input threshold, and detecting or recognizing the type of input or gesture. In some embodiments, tactile output is provided to serve as a warning or alert for an upcoming event or result that will occur unless a redirection or interruption input is detected in a timely manner. Tactile output is also used in other contexts to enhance the user experience, improve the accessibility of the device or other accessibility needs for users with visual or motor impairments, and / or improve the efficiency and functionality of the user interface and / or device. Tactile output is optional and, accompanied by audio output and / or a change to the visible user interface, further enhances the user experience when the user interacts with the user interface and / or device, further facilitates the delivery of information regarding the state of the user interface and / or device, and reduces user input errors when operating the device, thereby increasing efficiency.

[0038] Figure 4F1 provides a set of sample tactile output patterns that can be used individually or in combination, either as one or more transformations (e.g., modulation, amplification, truncation, etc.) or via them, to generate suitable haptic feedback for various scenarios and purposes, such as those described above and in relation to the user interfaces and methods referred to herein. Figures 4F2–4F4 are extended views of the graph shown in Figure 4F1. This example of a palette of tactile outputs shows how a set of three waveforms and eight frequencies can be used to generate an array of tactile output patterns. In addition to the tactile output patterns shown in this figure, the amplitude of each of these tactile output patterns can be selectively adjusted by changing the gain value relative to the tactile output pattern, for example, by changing the gains to 1.0, 0.75, 0.5, and 0.25 for full tap 80Hz, full tap 200Hz, mini tap 80Hz, mini tap 200Hz, micro tap 80Hz, and micro tap 200Hz, respectively, as shown in Figure 4G1. Figures 4G2 to 4G4 are extended views of the graph shown in Figure 4G1. As shown in Figure 4G1, the amplitude of the pattern is changed by changing the gain of the tactile output pattern without changing the pattern frequency or waveform shape. In some embodiments, some tactile output generators are limited by how much force can be applied to the movable mass, so changing the frequency of the tactile output pattern also results in smaller amplitudes, thereby constraining mass movement at higher frequencies to smaller amplitudes and ensuring that the acceleration required to create the waveform does not require forces outside the operating force range of the tactile output generator (for example, the peak amplitude of full tap at 230Hz, 270Hz, and 300Hz is lower than the amplitude of full tap at 80Hz, 100Hz, 125Hz, and 200Hz).

[0039] In Figure 4F1, each column represents a tactile output pattern with a specific waveform. The waveform of the tactile output pattern represents the neutral position (e.g., x) relative to the time it takes for the movable mass to pass through in order to generate the tactile output in that tactile output pattern. zeroThis represents the pattern of physical displacement relative to the tactile output pattern. For example, the first set of tactile output patterns shown in the left column of Figure 4F1 (e.g., the "full tap" tactile output pattern) each has a waveform that includes a complete two-cycle vibration (e.g., a vibration that starts and ends at the neutral position and crosses the neutral position three times). The second set of tactile output patterns shown in the middle column of Figure 4F1 (e.g., the "mini tap" tactile output pattern) has a waveform that includes a complete one-cycle vibration (e.g., a vibration that starts and ends at the neutral position and crosses the neutral position once). The third set of tactile output patterns shown in the right column of Figure 4F1 (e.g., the "micro tap" tactile output pattern) has a waveform that includes a vibration that includes half of a complete one-cycle vibration (e.g., a vibration that starts and ends at the neutral position and does not cross the neutral position). The waveforms of the tactile output patterns also include start and end buffers that represent the gradual increase and decrease in velocity of the movable mass at the start and end of the tactile output. The exemplary waveforms shown in Figures 4F1 to 4G1 represent the maximum and minimum movement of the movable mass x max Value and x min This includes the value. For larger electronic devices with greater movable mass, the minimum and maximum limits of mass movement may be greater or less. The examples shown in Figures 4F1 to 4G1 illustrate mass movement in one dimension, but similar principles apply to the movement of movable mass in two or three dimensions.

[0040] As shown in Figure 4F1, each tactile output pattern has a corresponding characteristic frequency that influences the "pitch" of the tactile sensation felt by the user from the tactile output having that characteristic frequency. For continuous tactile outputs, the characteristic frequency represents the number of cycles (e.g., cycles per second) that the movable mass of the tactile output generator completes within a given time period. For discrete tactile outputs, discrete output signals (e.g., 0.5 cycles, 1 cycle, or 2 cycles) are generated, and the characteristic frequency value identifies how fast the movable mass needs to move to generate the tactile output at that characteristic frequency. As shown in Figure 4F1, for each type of tactile output (e.g., defined by the respective waveforms such as full tap, mini tap, or micro tap), a higher frequency value corresponds to faster movement (one or more) by the movable mass, and therefore, generally, a shorter time to complete the tactile output (e.g., time to complete the number of cycles required for a discrete tactile output, including start and end buffer times). For example, a full tap with a characteristic frequency of 80 Hz takes longer to complete than a full tap with a characteristic frequency of 100 Hz (e.g., 35.4 ms vs. 28.3 ms in Figure 4F1). Furthermore, at a given frequency, tactile outputs with more cycles of their waveform at each frequency take longer to complete than tactile outputs with fewer cycles of their waveform at the same frequency. For example, a 150 Hz full tap takes longer to complete than a 150 Hz mini tap (e.g., 19.4 ms vs. 12.8 ms), and a 150 Hz mini tap takes longer to complete than a 150 Hz micro tap (e.g., 12.8 ms vs. 9.4 ms). However, this regularity may not apply to tactile output patterns at different frequencies (e.g., a tactile output with more cycles but a higher frequency may take less time than a tactile output with fewer cycles but a lower frequency, and vice versa). For example, at 300 Hz, a full tap takes the same amount of time as a mini tap (e.g., 9.9 ms).

[0041] As shown in Figure 4F1, the tactile output pattern also has a characteristic amplitude that affects the amount of energy contained in the tactile signal, or the "intensity" of the tactile sensation that the user may feel through the tactile output having that characteristic amplitude. In some embodiments, the characteristic amplitude of the tactile output pattern refers to an absolute or normalized value representing the maximum displacement of the movable mass from the neutral position when generating the tactile output. In some embodiments, the characteristic amplitude of the tactile output pattern is adjustable by a fixed or dynamically determined gain coefficient (e.g., a value between 0 and 1) according to various conditions (e.g., customized based on the context and behavior of the user interface) and / or pre-configured metrics (e.g., input-based metrics and / or user interface-based metrics). In some embodiments, the input-based metric (e.g., intensity change metric or input velocity metric) measures the characteristics of the input (e.g., the rate of change of the characteristic intensity of the contact in a pressing input, or the speed of the contact moving across the touch-sensing surface) during the input that triggers the generation of the tactile output. In some embodiments, a user interface-based metric (e.g., a metric for speed across boundaries) measures the characteristics of a user interface element (e.g., the speed of an element moving across a hidden or visible boundary in the user interface) during a user interface change that triggers the generation of a tactile output. In some embodiments, the characteristic amplitude of the tactile output pattern may be modulated by an "envelope," and the peaks of adjacent cycles may have different amplitudes, and one of the above waveforms may be further modulated by multiplying by an envelope parameter that changes over time (e.g., from 0 to 1) to gradually adjust the amplitude of a portion of the tactile output over time while the tactile output is being generated.

[0042] For illustrative purposes, Figure 4F1 shows specific frequencies, amplitudes, and waveforms represented by sample tactile output patterns; however, tactile output patterns with other frequencies, amplitudes, and waveforms can be used for similar purposes. For example, waveforms of 0.5 to 4 cycles can be used. Other frequencies in the range of 60 Hz to 400 Hz can also be used. Table 1 provides examples of specific tactile feedback behaviors, their configurations, and their uses.

[0043] [Table 1-1]

[0044] [Table 1-2]

[0045] [Table 1-3]

[0046] The examples shown in Table 1 illustrate the range of environments in which tactile outputs can be generated for different inputs and events. Table 1 should not be interpreted as a requirement for a device to respond to each of the listed inputs or events using the indicated tactile outputs. Rather, Table 1 is intended to illustrate how tactile outputs change and / or resemble each other for different inputs and / or events (e.g., based on tactile output patterns, frequency, gain, etc.). For example, Table 1 shows how an "event success" tactile output changes from an "event failure" tactile output, and how a retargeting tactile output differs from an impact tactile output.

[0047] Figures 4H to 4J show exemplary haptic audio output patterns versus time, used in conjunction with haptic output to simulate a button click, according to some embodiments.

[0048] Figure 4K shows exemplary combinations of tactile output patterns and tactile audio output patterns against time, according to one embodiment. Figures 4L to 4Q may be enlarged versions of the combinations shown in Figure 4K for clarity.

[0049] In Figure 4H, if the rate of change of contact intensity at the control activation threshold exceeds the threshold rate of change (for example, if the contact is a "normal" strong / fast press), the top tactile sound pattern "Click A1 sound" is an audio output played along with "Click A" normal minitap (230Hz) to simulate the first down click in a "normal" first click, as shown in Figure 4K (first row of the first click column) and the top of Figure 4L. In this example, "Click A1 sound" is offset by 2ms from the start of the "Click A" normal minitap (230Hz) tactile output. In some cases, the same "Click A1 sound" and "Click A" normal minitap (230Hz) are played to simulate the first up click following the first down click. In some cases, the gain of "Click A1 sound" and / or "Click A" normal minitap (230Hz) is reduced in the up click compared to the preceding down click (for example, by only 50%).

[0050] If the rate of change of contact intensity at the control activation threshold is less than the threshold rate of change (for example, if the contact is "weak" and / or slow), the top tactile sound pattern "Click A1 sound" is also played along with a "Click A" weak minitap (230Hz) to simulate a first down click in a "weak" first click, as shown in Figure 4K (second row of the first click column) and the bottom of Figure 4L. To simulate a "weak" down click, the gain of "Click A1 sound" and "Click A" weak minitap (230Hz) is reduced in a "weak" down click compared to a "normal" down click (for example, by only 50%). In this example, "Click A1 sound" is offset by 2ms from the start of the "Click A" weak minitap (230Hz) tactile output. In some cases, the same "Click A1 sound" and "Click A" weak minitap (230Hz) are played to simulate a first up click following a first down click. In some cases, the gain of the "Click A1 audio" and / or "Click A" weak mini-tap (230Hz) is reduced (for example, by only 50%) in the up-click compared to the preceding down-click.

[0051] In Figure 4H, if the rate of change of contact intensity at the control activation threshold exceeds the threshold rate of change (for example, if the contact in the second click is a "normal" strong / fast press), the bottom tactile sound pattern "Click A2 sound" is an audio output played along with "Click A" normal minitap (230Hz) to simulate a second down click in a "normal" second click following a first click (for example, a second click in a double-click input) within a predetermined time period, as shown in Figure 4K (first row of the second click column) and the top of Figure 4M. In this example, "Click A2 sound" is offset by 2ms from the start of the "Click A" normal minitap (230Hz) tactile output. In some cases, the same "Click A2 sound" and "Click A" normal minitap (230Hz) are played to simulate a second up click following a second down click. In some cases, the gain of the "Click A2 audio" and / or "Click A" normal mini-tap (230Hz) is reduced (for example, by only 50%) in the second up-click compared to the preceding second down-click.

[0052] If the rate of change of contact intensity at the control activation threshold is below the threshold rate of change (for example, if the contact is "weak" and / or slow), the bottom tactile sound pattern "Click A2 sound" is also played along with a "Click A" weak minitap (230Hz) to simulate a second down click in a "weak" second click following a first click within a predetermined time period (for example, as the second click in a double-click input), as shown in Figure 4K (second row of the second click column) and the bottom of Figure 4M. To simulate a "weak" down click, the gain of "Click A2 sound" and "Click A" weak minitap (230Hz) is reduced in the "weak" down click compared to a "normal" down click (for example, by only 50%). In this example, "Click A2 sound" is offset by 2ms from the start of the "Click A" weak minitap (230Hz) tactile output. In some cases, the same "Click A2 audio" and a weak "Click A" mini-tap (230Hz) are played to simulate a second up-click following a second down-click. In some cases, the gain of the "Click A2 audio" and / or the weak "Click A" mini-tap (230Hz) is reduced (for example, by only 50%) in the second up-click compared to the preceding second down-click.

[0053] In Figure 4I, when the rate of change of contact intensity at the control activation threshold exceeds the threshold rate of change (for example, when the contact becomes a "normal" strong / fast press), the top tactile sound pattern "Click B1 sound" is an audio output played along with "Click B" normal minitap (270Hz) to simulate the first down click in a "normal" first click, as shown in Figure 4K (third row of the first click column) and the top of Figure 4N. In this example, "Click B1 sound" is offset by 2.8ms from the start of the "Click B" normal minitap (270Hz) tactile output. In some cases, the same "Click B1 sound" and "Click B" normal minitap (270Hz) are played to simulate the first up click following the first down click. In some cases, the gain of "Click B1 sound" and / or "Click A" normal minitap (270Hz) is reduced (e.g., by only 50%) in the up click compared to the preceding down click.

[0054] If the rate of change of contact intensity at the control activation threshold is less than the threshold rate of change (for example, if the contact is "weak" and / or slow), the top tactile sound pattern "Click B1 sound" is also played along with a "Click B" weak minitap (270Hz) to simulate a first down click in a "weak" first click, as shown in Figure 4K (fourth row of the first click column) and the bottom of Figure 4N. To simulate a "weak" down click, the gain of "Click B1 sound" and the "Click B" weak minitap (270Hz) is reduced to a low value (e.g., only 50%) in a "weak" down click compared to a "normal" down click. In this example, "Click B1 sound" is offset by 2.8ms from the start of the "Click B" weak minitap (270Hz) tactile output. In some cases, the same "Click B1 sound" and "Click B" weak minitap (270Hz) are played to simulate a first up click following a first down click. In some cases, the gain of the "Click B1 audio" and / or "Click B" weak mini-tap (230Hz) is reduced (for example, by only 50%) in the up-click compared to the preceding down-click.

[0055] In Figure 4I, if the rate of change of contact intensity at the control activation threshold exceeds the threshold rate of change (for example, if the contact in the second click is a "normal" strong / fast press), the bottom tactile sound pattern "click B2 sound" is an audio output played along with "click B" normal minitap (270Hz) to simulate a second down click in a "normal" second click following the first click (for example, the second click in a double-click input) within a predetermined time period, as shown in Figure 4K (third row of the second click column) and the top of Figure 4O. In this example, "click B2 sound" is offset by 2.8ms from the start of the "click B" normal minitap (270Hz) tactile output. In some cases, the same "click B2 sound" and "click B" normal minitap (230Hz) are played to simulate a second up click following a second down click. In some cases, the gain of the "Click B2 audio" and / or "Click B" normal mini-tap (270Hz) is reduced (for example, by only 50%) in the second up-click compared to the preceding second down-click.

[0056] If the rate of change of contact intensity at the control activation threshold is below the threshold rate of change (for example, if the contact is "weak" and / or slow), the bottom tactile sound pattern "Click B2 sound" is also played along with a "Click B" weak minitap (270Hz) to simulate a second down click in a "weak" second click following a first click within a predetermined time period (for example, the second click in a double-click input), as shown in Figure 4K (fourth row of the second click column) and the bottom of Figure 4O. To simulate a "weak" down click, the gain of the "Click B2 sound" and the "Click B" weak minitap (270Hz) is reduced to a low value (e.g., only 50%) for a "weak" down click compared to a "normal" down click. In this example, the "Click B2 sound" is offset by 2.8ms from the start of the "Click B" weak minitap (270Hz) tactile output. In some cases, the same "Click B2 audio" and a weak "Click B" mini-tap (270Hz) are played to simulate a second up-click following a second down-click. In some cases, the gain of the "Click B2 audio" and / or the weak "Click B" mini-tap (270Hz) is reduced (for example, by only 50%) in the second up-click compared to the preceding second down-click.

[0057] In Figure 4J, when the rate of change of contact intensity at the control activation threshold exceeds the threshold rate of change (for example, when the contact becomes a "normal" strong / fast press), the top tactile sound pattern "click C1 sound" is an audio output played along with a "click C" normal minitap (300Hz) to simulate the first down click in a "normal" first click, as shown in Figure 4K (fifth row of the first click column) and the top of Figure 4P. In this example, the "click C1 sound" is offset by 1.9ms from the start of the "click C" normal minitap (300Hz) tactile output. In some cases, the same "click C1 sound" and "click C" normal minitap (300Hz) are played to simulate the first up click following the first down click. In some cases, the gain of the "click C1 sound" and / or the "click C" normal minitap (300Hz) is reduced in the up click compared to the preceding down click (for example, by only 50%).

[0058] If the rate of change of contact intensity at the control activation threshold is below the threshold rate of change (for example, if the contact is "weak" and / or slow), the top tactile sound pattern "Click C1 sound" is also played along with a "Click C" weak minitap (300Hz) to simulate a first down click in a "weak" first click, as shown in Figure 4K (sixth row of the first click column) and the bottom of Figure 4P. To simulate a "weak" down click, the gain of the "Click C1 sound" and the "Click C" weak minitap (300Hz) is reduced in a "weak" down click compared to a "normal" down click (for example, by only 50%). In this example, the "Click C1 sound" is offset by 1.9ms from the start of the "Click C" weak minitap (300Hz) tactile output. In some cases, the same "Click C1 sound" and the "Click C" weak minitap (270Hz) are played to simulate a first up click following a first down click. In some cases, the gain of the "Click C1 audio" and / or "Click C" weak mini-tap (300Hz) is reduced (e.g., by only 50%) in the up-click compared to the preceding down-click.

[0059] In Figure 4J, if the rate of change of contact intensity at the control activation threshold exceeds the threshold rate of change (for example, if the contact in the second click is a "normal" strong / fast press), the bottom tactile sound pattern "click C2 sound" is an audio output played along with a "click C" normal minitap (300Hz) to simulate a second down click in a "normal" second click following a first click (for example, the second click in a double-click input) within a predetermined time period, as shown in Figure 4K (the fifth row of the second click column) and the top of Figure 4Q. In this example, the "click C2 sound" is offset by 1.9ms from the start of the "click C" normal minitap (300Hz) tactile output. In some cases, the same "click C2 sound" and "click C" normal minitap (300Hz) are played to simulate a second up click following a second down click. In some cases, the gain of the "Click C2 audio" and / or "Click C" normal mini-tap (300Hz) is reduced (for example, by only 50%) in the second up-click compared to the preceding second down-click.

[0060] If the rate of change of contact intensity at the control activation threshold is below the threshold rate of change (for example, if the contact is "weak" and / or slow), the bottom tactile sound pattern "Click C2 sound" is also played along with a "Click C" weak minitap (300Hz) to simulate a second down click in a "weak" second click following a first click within a predetermined time period (for example, as the second click in a double-click input), as shown in Figure 4K (sixth row of the second click column) and the bottom of Figure 4Q. To simulate a "weak" down click, the gain of the "Click C2 sound" and the "Click C" weak minitap (300Hz) is reduced in the "weak" down click compared to a "normal" down click (for example, by only 50%). In this example, the "Click C2 sound" is offset by 1.9ms from the start of the "Click C" weak minitap (300Hz) tactile output. In some cases, the same "click C2 audio" and a weak "click C" mini-tap (300Hz) are played to simulate a second up-click following a second down-click. In some cases, the gain of the "click C2 audio" and / or the weak "click C" mini-tap (300Hz) is reduced (for example, by only 50%) in the second up-click compared to the preceding second down-click.

[0061] Device 100 is merely one embodiment of a portable multifunction device, and it should be understood that Device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of components. The various components shown in Figure 1A are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.

[0062] Memory 102 optionally includes high-speed random-access memory and optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by CPU(s) 120 and other components of device 100 such as peripheral interface 118 is optionally controlled by memory controller 122.

[0063] The peripheral device interface 118 is used to connect the device's input and output peripherals to the CPU(s) 120 and memory 102. One or more processors 120 operate or execute various software programs and / or instruction sets stored in memory 102 to perform various functions for device 100 and process data.

[0064] In some embodiments, the peripheral interface 118, the CPU(s) 120, and the memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0065] The RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals, or electromagnetic signals to electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes, but is not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identification module (SIM) card, memory, and other well-known circuits for performing those functions. The RF circuit 108 optionally communicates wirelessly with other devices with networks such as the Internet, also known as the World Wide Web (WWW), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs). Wireless communication uses one of several communication standards, protocols, and technologies at will. These communication standards, protocols, and technologies include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-speed Downlink Packet Access (HSDPA), High-speed Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long-Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), and Code Division Multiple Access (W-CDMA).Bluetooth®, Wireless Fidelity® (Wi-Fi®) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), Instant Messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE)), Instant Messaging and Presence Services (Instant Examples include, but are not limited to, Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

[0066] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits the audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved by the peripheral interface 118 from memory 102 and / or RF circuit 108 and / or transmitted to memory 102 and / or RF circuit 108. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., 212 in Figure 2). The headset jack provides an interface between the audio circuit 110 and a detachable audio input / output peripheral device, such as an output-only headphone or a headset having both an output (e.g., headphones for one or both ears) and an input (e.g., a microphone).

[0067] The I / O subsystem 106 connects input / output peripherals on device 100, such as the touch-sensitive display system 112 and other input or control devices 116, to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for the other input or control devices. One or more input controllers 160 receive electrical signals from and transmit electrical signals to the other input or control devices 116. The other input or control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) 160 are optionally connected to (or not connected to) any of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., 208, Figure 2A) are optional and include up / down buttons for volume control of speaker 111 and / or microphone 113. One or more buttons are optional and include push buttons (e.g., 206, Figure 2A).

[0068] The touch-sensitive display system 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as “graphics”). In some embodiments, some or all of the visual output corresponds to user interface objects. As used herein, the term “affordance” refers to a user-interactive graphical user interface object (for example, a graphical user interface object configured to respond to input directed toward the graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0069] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or set of sensors that accept user input based on tactile and / or haptic contact. The touch-sensitive display system 112 and the display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of contact) on the touch-sensitive display system 112 and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In one exemplary embodiment, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0070] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light-emitting polymer display) technology, or LED (light-emitting diode) technology, but other display technologies are used in other embodiments. The touch-sensitive display system 112 and the display controller 156 optionally use any of several currently known or future-developed touch-sensing technologies, including but not limited to capacitive technology, resistive technology, infrared technology, and surface acoustic wave technology, as well as other proximity sensor arrays or other elements for determining one or more contact points with the touch-sensitive display system 112, to detect contact and any movement or interruption thereof. In exemplary embodiments, projected mutual capacitive sensing technology is used, such as that found in iPhone®, iPod Touch®, and iPad® from Apple Inc. (Cupertino, California).

[0071] The touch-sensitive display system 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the video resolution of the touchscreen is greater than 400 dpi (e.g., 500 dpi, 800 dpi, or higher). The user optionally touches the touch-sensitive display system 112 using any suitable object or attachment such as a stylus or finger. In some embodiments, the user interface is designed to function with finger-based touch and gestures, which may be less precise than stylus-based input due to the larger area of ​​finger contact on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform user-desired actions.

[0072] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0073] Device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power within the portable device.

[0074] Device 100 also optionally includes one or more optical sensors 164. Figure 1A shows an optical sensor coupled with an optical sensor controller 158 in the I / O subsystem 106. The optical sensor(s) 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor(s) 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with the imaging module 143 (also called the camera module), the optical sensor(s) 164 optionally captures still images and / or video. In some embodiments, the optical sensor is located on the back of device 100, opposite the touch-sensitive display system 112 on the front of the device, so that the touchscreen can be used as a viewfinder for acquiring still images and / or video images. In some embodiments, a separate light sensor is located on the front of the device so that the user's image can be captured (for example, for selfies, or for video conferencing while the user is viewing other video conference participants on the touchscreen).

[0075] Device 100 also optionally includes one or more contact strength sensors 165. Figure 1A shows a contact strength sensor coupled with a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor(s) 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, pressure-power sensors, optical force sensors, capacitive touch-sensing surfaces, or other strength sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensing surface). The contact strength sensor(s) 165 receive contact strength information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact strength sensor is positioned juxtaposed with or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112). In some embodiments, at least one contact strength sensor is located on the back of Device 100, opposite the touchscreen display system 112 which is located on the front of Device 100.

[0076] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows a proximity sensor 166 coupled to a peripheral interface 118. Alternatively, the proximity sensor 166 is coupled to an input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is positioned near the user's ear (for example, when the user is making a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0077] Device 100 also optionally includes one or more tactile output generators 167. Figure 1A shows a tactile output generator coupled with a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator(s) 167 optionally includes one or more electroacoustic devices, such as a speaker or other sound component, and / or electromechanical devices that convert energy into linear motion, such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts an electrical signal into a tactile output on the device). The tactile output generator(s) 167 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output on device 100 that can be sensed by the user of device 100. In some embodiments, at least one tactile output generator is positioned adjacent to or near a touch-sensitive surface (e.g., a touch-sensitive display system 112) and optionally generates a tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward from the surface of device 100) or laterally (e.g., forward / backward within the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100, opposite the touch-sensitive display system 112 which is located on the front of device 100.

[0078] Device 100 also optionally includes one or more accelerometers 168. Figure 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in a vertical or horizontal view based on an analysis of data received from one or more accelerometers. Device 100 optionally includes, in addition to the accelerometers (one or more) 168, a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the position and orientation (e.g., vertical or horizontal) of Device 100.

[0079] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / mobility module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, as shown in Figures 1A and 3, memory 102 stores device / global internal state 157. The device / global internal state 157 includes one or more of the following: an active application state indicating which application is currently active, if there is an active application; a display state indicating which applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state including information obtained from various sensors and other input or control devices 116 of the device; and location and / or positional information relating to the location and / or orientation of the device.

[0080] An operating system 126 (for example, an embedded operating system such as iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks) includes various software components and / or drivers for controlling and managing overall system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware components and software components.

[0081] The communication module 128 facilitates communication with other devices through one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to connect with other devices directly or indirectly through a network (e.g., the Internet, Wi-Fi, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors identical or similar to and / or compatible with the 30-pin connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external ports are Lightning connectors identical or similar to and / or compatible with the Lightning connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California.

[0082] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in cooperation with the display controller 156) and contact with other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger-down event), determining the intensity of contact (e.g., the force or pressure of contact, or an alternative to the force or pressure of contact), determining whether there is movement of contact and tracking movement across the touch-sensitive surface (e.g., detecting a drag event of one or more fingers), and determining whether contact has stopped (e.g., detecting a finger-up event or interruption of contact). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by a series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These operations are optional and apply to single contacts (e.g., one-finger contact or stylus contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple-finger contacts). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0083] The contact / motion module 130 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different movement, timing, and / or intensity of the detected contact). Thus, gestures are optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture involves detecting a finger downward event, followed by a finger lift-off event at the same (or substantially the same) location as the downward event (e.g., at the icon's location). In another embodiment, detecting a finger swipe gesture on the touch-sensitive surface involves detecting a finger downward event, followed by a drag event of one or more fingers, and then a finger lift-off event. Similarly, taps, swipes, drags, and other gestures are optionally detected with respect to the stylus by detecting a specific contact pattern with respect to the stylus.

[0084] In some embodiments, detecting a finger tap gesture depends on the length of time between detecting a finger-down event and detecting a finger-up event, but not on the intensity of finger contact between the two events. In some embodiments, a tap gesture is detected based on the determination that the length of time between the finger-down and finger-up events is shorter than a predetermined value (e.g., shorter than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of finger contact between taps meets a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a shallow or deep pressure intensity threshold. Thus, a finger tap gesture can satisfy certain input criteria that do not require the characteristic intensity of contact to meet a given intensity threshold in order for a particular input criterion to be met. To clarify, finger contact in a tap gesture generally needs to meet a nominal contact detection intensity threshold below which contact is not detected in order for a finger-down event to be detected. Similar analysis applies to detecting tap gestures or other contacts made by a stylus. In cases where the device can detect contact from a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold is optional and does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0085] In a similar manner, the same concept applies to other types of gestures. For example, swipe gestures, pinch gestures, de-pinch gestures, and / or long press gestures are optional and are detected based on whether they meet criteria that are either unrelated to the intensity of the contacts involved in the gesture, or do not require the contacts performing the gesture to reach an intensity threshold in order to be recognized. For example, swipe gestures are detected based on the amount of movement of one or more contacts, pinch gestures are detected based on the movement of two or more contacts toward each other, de-pinch gestures are detected based on the movement of two or more contacts toward each other, and long press gestures are detected based on the duration of contact on the touch-sensitive surface that is less than a threshold amount of movement. Thus, the statement that a particular gesture recognition criterion does not require the intensity of one or more contacts to meet their respective intensity thresholds for that criterion to be satisfied means that a particular gesture recognition criterion can be satisfied when one or more contacts in a gesture do not reach their respective intensity thresholds, and can also be satisfied when one or more of the contacts in a gesture do not reach or exceed their respective intensity thresholds. In some embodiments, a tap gesture is detected based on the determination that a finger-down event and a finger-up event were detected within a predetermined time period, regardless of whether the contacts exceed or fall below their respective intensity thresholds during a predetermined time period, and a swipe gesture is detected based on the determination that the movement of the contacts is greater than a predetermined magnitude, even if the contacts exceed their respective intensity thresholds at the end of the movement of the contacts. Even in embodiments where gesture detection is influenced by the intensity of the contact performing the gesture (for example, the device detects longer presses more quickly when the contact intensity exceeds an intensity threshold, or the device is slower to detect tap inputs when the contact intensity is higher), the detection of those gestures does not require the contact to reach a specific intensity threshold, as long as the criteria for recognizing the gesture can be met in situations where the contact does not reach a specific intensity threshold (for example, even if the amount of time required to recognize the gesture changes).

[0086] Contact intensity thresholds, duration thresholds, and movement thresholds are combined in various different combinations to create heuristics for distinguishing two or more different gestures directed towards the same input element or region in certain situations, thereby enabling multiple different interactions with the same input element to provide a richer set of user interactions and responses. A description that a particular set of gesture recognition criteria does not require that the intensity of a contact(s) meets each intensity threshold for a particular gesture recognition criterion to be satisfied does not preclude the simultaneous evaluation of other intensity-dependent gesture recognition criteria for identifying other gestures whose criteria are satisfied when the gesture includes contacts with an intensity exceeding each intensity threshold. For example, in certain situations, a first gesture recognition criterion for a first gesture that does not require that the intensity of a contact(s) meet each intensity threshold for a first gesture recognition criterion to be satisfied is in competition with a second gesture recognition criterion for a second gesture that depends on the contact(s) reaching each intensity threshold. In such competition, a gesture is optional and will not be recognized as satisfying the first gesture recognition criterion for the first gesture if the second gesture recognition criterion for the second gesture is first satisfied. For example, if a contact reaches its respective intensity threshold before it moves a predetermined amount, a deep press gesture will be detected instead of a swipe gesture. Conversely, if a contact moves a predetermined amount before it reaches its respective intensity threshold, a swipe gesture will be detected instead of a deep press gesture. Even in such situations, the first gesture recognition criterion for the first gesture does not still require the intensity of the contact(s) to meet their respective intensity thresholds for the first gesture recognition criterion to be satisfied, because if the contact remains below its respective intensity threshold until the end of the gesture (for example, a swipe gesture with contacts that do not increase to an intensity above their respective intensity thresholds), the gesture is recognized as a swipe gesture by the first gesture recognition criterion.In this way, a particular gesture recognition criterion that does not require the intensity of a single or multiple contact to satisfy its respective intensity threshold for that criterion to be met still depends on the intensity of the contact with respect to the intensity threshold, in the sense that (A) in some situations it ignores the intensity of the contact with respect to the intensity threshold (e.g., for a tap gesture), and / or (B) in some situations the particular gesture recognition criterion (e.g., for a long press gesture) does not function if a competing set of intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-dependent gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input (e.g., for a long press gesture competing with a deep press gesture for recognition).

[0087] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other display, including components for modifying the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual characteristics). As used herein, the term “graphics” includes, but is not limited to, any objects that can be displayed to a user, including, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0088] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like, as needed, specifying the graphics to be displayed along with coordinate data and other graphic characteristic data, and then generates screen image data to be output to the display controller 156.

[0089] The haptic feedback module 133 includes various software components for generating instructions to be used by the haptic output generator(s) 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0090] The text input module 134 is optionally a component of the graphic module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).

[0091] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for location-based dialing, to the camera 143 as metadata for photos / videos, and to applications that provide location-based services such as weather widgets, local yellow page widgets, and map / navigation widgets).

[0092] The application 136 optionally includes the following modules (or sets of instructions), or subsets or supersets thereof. ● Contact module 137 (which may also be referred to as an address book or contact list), ● Phone module 138, ● Video conferencing module 139, ● Email client module 140, ● Instant messaging (IM) module 141, ● Training support module 142, ● Camera module 143 for still and / or video images, ● Image management module 144, ● Browser module 147, ● Calendar module 148, ●Optionally, a widget module 149 including one or more of the following: weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, other widgets obtained by the user, and user-created widgets 149-6. ●Widget creation module 150 for creating user-created widget 149-6, ●Search module 151, ●Optionally, a video and music player module 152 is configured with a video player module and a music player module. ●Memo Module 153, ●Map module 154, and / or ● Online video module 155.

[0093] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0094] Together with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 includes executable instructions for managing an address book or contact list (stored, for example, in the application internal state 192 of the contact module 137 in memory 102 or memory 370), which include adding names(s) to the address book, deleting names(s) from the address book, associating telephone numbers(s) to names, email addresses(s) to names, addresses(s) to names, or other information, associating images to names, categorizing and sorting names, providing telephone numbers and / or email addresses to initiate and / or facilitate communication by telephone 138, video conferencing 139, email 140, or IM 141, and so on.

[0095] In conjunction with the RF circuit 108, voice circuit 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the telephone module 138 includes executable commands to input a series of characters corresponding to a telephone number, access one or more telephone numbers in the address book 137, modify the entered telephone number, dial each telephone number, conduct a conversation, and disconnect or hang up when the conversation is complete. As described above, wireless communication may optionally use any of several communication standards, protocols, and technologies.

[0096] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, the video conferencing module 139 includes executable commands to start, conduct, and end a video conference between the user and one or more other participants, according to the user's instructions.

[0097] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the email client module 140 includes executable commands for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with the image management module 144, the email client module 140 makes it very easy to create and send emails containing still or video images captured by the camera module 143.

[0098] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the instant messaging module 141 includes executable instructions for inputting a series of characters corresponding to an instant message, modifying previously entered characters, sending each instant message (for example, using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant messaging, or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0099] Together with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, the training support module 142 includes executable commands for creating training (e.g., with time, distance, and / or calorie consumption targets), communicating with training sensors (in sports devices and smartwatches), receiving training sensor data, calibrating sensors used to monitor training, selecting and playing music for training, and displaying, storing, and transmitting training data.

[0100] Together with the touch-sensitive display system 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0101] Together with the touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, text input module 134, and camera module 143, the image management module 144 includes executable commands for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0102] Together with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, the browser module 147 includes executable commands for browsing the internet according to user instructions, including searching, linking, receiving, and displaying web pages or parts thereof, as well as attachments and other files linked to web pages.

[0103] Together with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and data associated with calendars (e.g., calendar items, to-do lists, etc.) in accordance with user instructions.

[0104] Together with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphic module 132, text input module 134, and browser module 147, the widget module 149 is optionally a mini-application downloaded and used by the user (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or a mini-application created by the user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript® file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0105] In conjunction with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget creation module 150 includes executable commands for creating widgets (for example, converting user-specified portions of a web page into widgets).

[0106] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions to search for text, music, sounds, images, videos, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0107] In cooperation with the touch sensing display system 112, display system controller 156, contact module 130, graphic module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music player module 152 includes executable instructions that enable a user to download and play recorded music or other sound files stored in one or more file formats such as MP3 or AAC files, and executable instructions to display, present, or otherwise play videos (e.g., on the touch sensing display system 112 or on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

[0108] In cooperation with the touch sensing display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the memo module 153 includes executable instructions to create and manage memos, lists of things to do, etc. according to user instructions.

[0109] In cooperation with the RF circuit 108, touch sensing display system 112, display system controller 156, contact module 130, graphic module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 includes executable instructions to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data about stores and other points of interest at or near a particular location, and other location-based data) according to user instructions.

[0110] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the audio circuit 110, the speaker 111, the RF circuit 108, the text input module 134, the email client module 140, and the browser module 147, the online video module 155 includes executable instructions that enable the user to access, view, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen 112, or on an external display connected wirelessly or via the external port 124), send emails containing links to specific online videos, and otherwise manage them. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos.

[0111] Each of the modules and applications identified above corresponds to an executable instruction set that performs one or more of the functions described above, as well as a method described herein (e.g., a computer method and other information processing method described herein). These modules (i.e., instruction sets) do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of these modules can be optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the identified modules and data structures. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0112] In some embodiments, device 100 is a device on which a set of predefined functions are operated exclusively through a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.

[0113] A set of predefined functions, optionally performed exclusively through the touchscreen and / or touchpad, includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to the main menu, home menu, or root menu. In such embodiments, a “menu button” is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

[0114] Figure 1B is a block diagram showing exemplary components for event handling according to some embodiments. In some embodiments, memory 102 (in Figure 1A) or 370 (in Figure 3) includes an event sorting unit 170 (e.g., within the operating system 126) and each application 136-1 (e.g., any of the applications 136, 137-155, 380-390 described above).

[0115] The event sorting unit 170 receives event information and determines the application 136-1 and the application view 191 of application 136-1 to which the event information is distributed. The event sorting unit 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view(s) displayed on the touch-sensitive display system 112 when the application is active or running. In some embodiments, the device / global internal state 157 is used by the event sorting unit 170 to determine which application(s) are currently active, and the application internal state 192 is used by the event sorting unit 170 to determine the application view(s) to which the event information is distributed.

[0116] In some embodiments, the application internal state 192 includes additional information such as resume information used when the application 136-1 resumes execution, user interface state information indicating information that is displayed or ready to be displayed by the application 136-1, a state queue to allow the user to return to a previous state or view of the application 136-1, and a redo / undo queue for previous actions taken by the user.

[0117] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (for example, a user touch on the touch-sensitive display system 112 as part of a multi-touch gesture). The peripheral interface 118 transmits information received from the I / O subsystem 106, or from sensors such as the proximity sensor 166, one or more accelerometers 168, and / or a microphone 113 (through the audio circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display system 112 or the touch-sensitive surface.

[0118] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when a significant event occurs (e.g., receiving input that exceeds a predetermined noise threshold and / or for a longer period than predetermined).

[0119] In some embodiments, the event sorting unit 170 also includes a hit view determination module 172 and / or an active event recognition determination module 173.

[0120] The hit view determination module 172 provides software procedures for determining where in one or more views a sub-event occurred when the touch-sensitive display system 112 displays two or more views. A view consists of control and other elements that the user can see on the display.

[0121] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of each application) in which a touch is detected is optional and corresponds to the program level within the application's program hierarchy or view hierarchy. For example, the lowest-level view in which a touch is detected is optional and is called the hit view, and the set of events recognized as appropriate input is optional and is determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0122] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchy that should process the sub-events. In most situations, the hit view is the lowest-level view where the initiating sub-event (i.e., the first sub-event in a series of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source that identified it as the hit view.

[0123] The active event recognition determination module 173 determines which view(s) in the view hierarchy should receive a particular set of sub-events. In some embodiments, the active event recognition determination module 173 determines that only the hit view should receive a particular set of sub-events. In other embodiments, the active event recognition determination module 173 determines that all views, including the physical location of the sub-events, are actively involved views, and therefore all actively involved views should receive a particular set of sub-events. In other embodiments, even if the touch sub-events are entirely confined to an area associated with one particular view, higher-level views in the hierarchy remain actively involved views.

[0124] The event dispatcher module 174 dispatches event information to an event recognition unit (for example, an event recognition unit 180). In embodiments including an active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores the event information acquired by each event receiving unit module 182 in an event queue.

[0125] In some embodiments, the operating system 126 includes an event sorting unit 170. Alternatively, application 136-1 includes an event sorting unit 170. In yet other embodiments, the event sorting unit 170 is either a standalone module or part of another module stored in memory 102, such as a contact / motion module 130.

[0126] In some embodiments, application 136-1 includes a plurality of event processing units 190 and one or more application views 191, each containing instructions for handling touch events occurring within each view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognition units 180. Typically, each application view 191 includes a plurality of event recognition units 180. In other embodiments, one or more of the event recognition units 180 are part of a separate module, such as a user interface kit (not shown) or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, each event handler 190 includes one or more event data 179 received from a data update unit 176, an object update unit 177, a GUI update unit 178, and / or an event sort unit 170. The event handler 190 optionally utilizes or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 to update the application's internal state 192. Instead, one or more of the application views 191 include one or more respective event handlers 190. Also, in some embodiments, one or more of the data update unit 176, object update unit 177, and GUI update unit 178 are included in the respective application views 191.

[0127] Each event recognition unit 180 receives event information (e.g., event data 179) from the event sorting unit 170 and identifies an event from the event information. The event recognition unit 180 includes an event receiving unit 182 and an event comparison unit 184. In some embodiments, the event recognition unit 180 also includes at least a subset of metadata 183 and event distribution commands 188 (optionally including sub-event distribution commands).

[0128] The event receiving unit 182 receives event information from the event sorting unit 170. The event information includes information about sub-events, for example, information about touches or the movement of touches. Depending on the sub-event, the event information also includes additional information such as the position of the sub-event. When the sub-event involves a touch movement, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, an event includes a rotation of the device from one orientation to another (for example, from vertical to horizontal, or vice versa), and the event information includes corresponding information about the device's current orientation (also called the device's orientation).

[0129] The event comparison unit 184 compares event information with a predefined event or sub-event definition, and based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. The event definition 186 includes, for example, definitions of events such as event 1 (187-1), event 2 (187-2) (e.g., a predefined series of sub-events). In some embodiments, the sub-events in event 187 include, for example, the start of a touch, the end of a touch, a move of a touch, a cancellation of a touch, and multiple touches. In one embodiment, the definition for event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch (start of touch) for a predetermined stage on the displayed object, a first lift-off (end of touch) for a predetermined stage, a second touch (start of touch) for a predetermined stage on the displayed object, and a second lift-off (end of touch) for a predetermined stage. In another embodiment, the definition of event 2(187-2) is a drag on a displayed object. The drag includes, for example, touching (or contacting) a predetermined number of stages on the displayed object, moving the touch across the touch-sensitive display system 112, and lifting off the touch (end of the touch). In some embodiments, the event also includes information about one or more associated event handlers 190.

[0130] In some embodiments, the event definition 187 includes an event definition for each user interface object. In some embodiments, the event comparison unit 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, the event comparison unit 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with its respective event handler 190, the event comparison unit uses the results of the hit test to determine which event handler 190 should be activated. For example, the event comparison unit 184 selects the sub-event and the event handler associated with the object that triggers the hit test.

[0131] In some embodiments, the definition of each event 187 also includes a delay action that delays the delivery of event information until it is determined whether a series of sub-events correspond to an event type of the event recognition unit.

[0132] When each event recognition unit 180 determines that a series of sub-events does not match any of the events in the event definition 186, each event recognition unit 180 enters an event impossible, event failed, or event terminated state and thereafter ignores subsequent sub-events of the touch-based gesture. In this situation, if there are other event recognition units that remain active for the hit view, those event recognition units continue to track and process the sub-events of the ongoing touch-based gesture.

[0133] In some embodiments, each event recognition unit 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate to the actively involved event recognition units how the event distribution system should perform sub-event distribution. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how or are made capable of interacting with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are distributed to various levels in the view hierarchy or program hierarchy.

[0134] In some embodiments, each event recognition unit 180 activates an event handler 190 associated with an event when one or more specific sub-events of an event are recognized. In some embodiments, each event recognition unit 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is separate from sending (and delaying the sending of) sub-events to the respective hit view. In some embodiments, the event recognition unit 180 sets a flag associated with the recognized event, and the event handler 190 associated with the flag captures the flag and performs a predefined process.

[0135] In some embodiments, the event distribution command 188 includes a sub-event distribution command that distributes event information about a sub-event without activating an event handler. Instead, the sub-event distribution command distributes event information to an event handler associated with a set of sub-events, or to a view that is actively involved. The event handler associated with the set of sub-events or the view that is actively involved receives the event information and performs predetermined processing.

[0136] In some embodiments, the data update unit 176 creates and updates data used in application 136-1. For example, the data update unit 176 updates telephone numbers used in contact module 137 or stores video files used in video player module 145. In some embodiments, the object update unit 177 creates and updates objects used in application 136-1. For example, the object update unit 177 creates new user interface objects or updates the position of user interface objects. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares display information and sends it to graphics module 132 for display on touch-sensitive display.

[0137] In some embodiments, the event handler(s) 190 includes, or has access to, a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are contained in a single module of their respective applications 136-1 or application view 191. In other embodiments, they are contained in two or more software modules.

[0138] The above description of event processing for user touch on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 using input devices, but it should be understood that not all of this begins on the touchscreen. For example, optional inputs such as mouse movement and mouse button presses, taps, drags, and scrolls on a touchpad, pen stylus input, device movement, verbal instructions, detected eye movements, biometric input, and / or any combination thereof may be used as inputs corresponding to sub-events that define the events to be recognized.

[0139] Figure 1C is a block diagram showing a tactile output module according to some embodiments. In some embodiments, the I / O subsystem 106 (e.g., a tactile feedback controller 161 (Figure 1A) and / or other input controllers (one or more) 160 (Figure 1A)) includes at least some of the exemplary components shown in Figure 1C. In some embodiments, the peripheral device interface 118 includes at least some of the exemplary components shown in Figure 1C.

[0140] In some embodiments, the tactile output module includes a tactile feedback module 133. In some embodiments, the tactile feedback module 133 collects and combines tactile outputs in response to user interface feedback from a software application on an electronic device (e.g., feedback in response to user input corresponding to a displayed user interface, and warnings indicating the execution of an action or the occurrence of an event in the user interface of the electronic device). The tactile feedback module 133 includes one or more of the following: a waveform module 123 (for providing waveforms used to generate tactile outputs), a mixer 125 (for mixing waveforms, such as waveforms of different channels), a compressor 127 (for reducing or compressing the dynamic range of the waveform), a low-pass filter 129 (for filtering out high-frequency signal components in the waveform), and a thermal controller 131 (for adjusting the waveform according to thermal conditions). In some embodiments, the tactile feedback controller 161 (Figure 1A) includes the tactile feedback module 133. In some embodiments, separate units of the haptic feedback module 133 (or separate implementations of the haptic feedback module 133) are also included in an audio controller (e.g., the audio circuit 110 in Figure 1A) and used to generate an audio signal. In some embodiments, a single haptic feedback module 133 is used to generate an audio signal and generate a waveform for the tactile output.

[0141] In some embodiments, the haptic feedback module 133 also includes a trigger module 121 (e.g., a software application, an operating system, or another software module that decides to generate a tactile output and initiates the process of generating the corresponding tactile output). In some embodiments, the trigger module 121 generates a trigger signal to initiate waveform generation (e.g., by the waveform module 123). For example, the trigger module 121 generates a trigger signal based on a preset timing criterion. In some embodiments, the trigger module 121 receives a trigger signal from an external haptic feedback module 133 (e.g., in some embodiments, the haptic feedback module 133 receives a trigger signal from a hardware input processing module 146 located outside the haptic feedback module 133) and relays the trigger signal to a software application that triggers an action (e.g., using the trigger module 121) based on the activation of other components within the haptic feedback module 133 (e.g., the waveform module 123) or a hardware input device (e.g., a home button). In some embodiments, the trigger module 121 also receives a tactile feedback generation command (e.g., from the haptic feedback module 133, Figures 1A and 3). In some embodiments, the trigger module 121 generates a trigger signal in response to a tactile feedback module 133 (or the trigger module 121 within the tactile feedback module 133) receiving a tactile feedback command (for example, from the tactile feedback module 133 in Figures 1A and 3).

[0142] The waveform module 123 receives a trigger signal as input (e.g., from the trigger module 121) and, in response to the received trigger signal, provides a waveform (selected from a predetermined set of waveforms designed for use by the waveform module 123, such as the waveforms described in more detail below with reference to Figures 4F1 to 4G1) for generating one or more tactile outputs.

[0143] Mixer 125 receives waveforms (e.g., waveform module 123) as input and synthesizes the waveforms. For example, if mixer 125 receives two or more waveforms (e.g., a first waveform on a first channel and a second waveform on a second channel that at least partially overlaps the first waveform), mixer 125 outputs a synthesized waveform corresponding to the sum of the two or more waveforms. In some embodiments, mixer 125 also modifies one or more of the two or more waveforms (e.g., by increasing the scale of a particular waveform(s) and / or decreasing the scale of the rest of the waveforms) to emphasize a particular waveform(s) across the rest of the two or more waveforms. In some situations, mixer 125 selects one or more waveforms to remove from the synthesized waveform (e.g., if there are waveforms from more than three sources that are requested to be output simultaneously by the tactile output generator 167, it removes the waveform from the oldest source).

[0144] The compressor 127 receives a waveform (e.g., a synthesized waveform from the mixer 125) as input and modifies the waveform. In some embodiments, the compressor 127 reduces the waveform (e.g., according to the physical specifications of the tactile output generator 167 (Figure 1A) or 357 (Figure 3)) so that the tactile output corresponding to the waveform is reduced. In some embodiments, the compressor 127 limits the waveform, for example by forcing a predetermined maximum amplitude on the waveform. For example, the compressor 127 reduces the amplitude of waveform portions that exceed a predetermined amplitude threshold while maintaining that the amplitude of waveform portions does not exceed a predetermined amplitude threshold. In some embodiments, the compressor 127 reduces the dynamic range of the waveform. In some embodiments, the compressor 127 dynamically reduces the dynamic range of the waveform so that the synthesized waveform remains within the performance specifications of the tactile output generator 167 (e.g., force and / or movable mass displacement limits).

[0145] The low-pass filter 129 receives a waveform (e.g., a compressed waveform from the compressor 127) as input and filters the waveform (e.g., smooths it) (e.g., removes or reduces high-frequency signal components of the waveform). For example, in some cases, the compressor 127 includes in the compressed waveform an external signal (e.g., a high-frequency signal component) that interferes with the generation of a tactile output according to the compressed waveform and / or exceeds the performance specifications of the tactile output generator 167 when a tactile output is generated. The low-pass filter 129 reduces or removes such external signals in the waveform.

[0146] The thermal controller 131 receives a waveform as input (e.g., a filtered waveform from the low-pass filter 129) and adjusts the waveform according to the thermal conditions of device 100 (e.g., based on the temperature of the tactile feedback controller 161 and / or the internal temperature detected within device 100, such as the external temperature detected by device 100). For example, in some cases, the output of the tactile feedback controller 161 changes with temperature (e.g., the tactile feedback controller 161 generates a first tactile output when it is at a first temperature, and a second tactile output when it is at a second temperature separate from the first temperature, depending on the waveform it receives). For example, the magnitude (or amplitude) of the tactile output may change with temperature. To reduce the effects of temperature changes, the waveform is modified (e.g., the amplitude of the waveform is increased or decreased based on temperature).

[0147] In some embodiments, a haptic feedback module 133 (e.g., a trigger module 121) is coupled to a hardware input processing module 146. In some embodiments, one or more other input controllers 160 in Figure 1A include the hardware input processing module 146. In some embodiments, the hardware input processing module 146 receives input from a hardware input device 145 (e.g., another input or control device 116 in Figure 1, such as a home button). In some embodiments, the hardware input device 145 is one of the following: a touch-sensitive display system 112 (Figure 1A), a keyboard / mouse 350 (Figure 3), a touchpad 355 (Figure 3), another input or control device 116 (Figure 1A), or an input device such as a strength-sensitive home button (e.g., a home button with a mechanical actuator as shown in Figure 2B or Figure 2C). In some embodiments, the hardware input device 145 comprises a strength-sensitive home button (e.g., a home button having a mechanical actuator as shown in Figure 2B or Figure 2C), and a touch-sensitive display system 112 (Figure 1A), a keyboard / mouse 350 (Figure 3), or a touchpad 355 (Figure 3). In some embodiments, in response to input from the hardware input processing module 145, the hardware input processing module 146 provides one or more trigger signals to the tactile feedback module 133 to indicate that a user input meeting predetermined input criteria has been detected, such as an input corresponding to a home button "click" (e.g., a "down click" or "up click"). In some embodiments, in response to an input corresponding to a home button "click," the tactile feedback module 133 provides a waveform corresponding to a home button "click" to simulate tactile feedback of pressing a physical home button.

[0148] In some embodiments, the tactile output module includes a tactile feedback controller 161 (tactile feedback controller 161 in Figure 1A) that controls the generation of tactile outputs. In some embodiments, the tactile feedback controller 161 is coupled to a plurality of tactile output generators, selects one or more of the plurality of tactile output generators, and transmits waveforms to the selected one or more tactile output generators to generate tactile outputs. In some embodiments, the tactile feedback controller 161 modifies one or more of two or more waveforms to emphasize a particular waveform(s) over the remainder of two or more waveforms (for example, by increasing the scale of a particular waveform(s) and / or decreasing the scale of the remainder of the waveforms, to prioritize a tactile output corresponding to the activation of the hardware input device 145 over tactile outputs corresponding to software events, for example, by increasing the scale of a particular waveform(s) and / or decreasing the scale of the remainder of the waveforms.

[0149] In some embodiments, the output of the haptic feedback controller 161 is coupled to the audio circuit of device 100 (e.g., audio circuit 110, Figure 11A) to provide an audio signal to the audio circuit of device 100. In some embodiments, the haptic feedback controller 161 provides both a waveform used to generate a tactile output and an audio signal used to provide an audio output in connection with the generation of a tactile output. In some embodiments, the haptic feedback controller 161 modifies the audio signal and / or waveform (used to generate the tactile output) so that the audio output and the tactile output are synchronized (e.g., by delaying the audio signal and / or waveform). In some embodiments, the haptic feedback controller 161 includes a DA converter used to convert the digital waveform to an analog signal, which is received by an amplifier 163 and / or a tactile output generator 167.

[0150] In some embodiments, the tactile output module includes an amplifier 163. In some embodiments, the amplifier 163 receives a waveform (e.g., from a tactile feedback controller 161) and amplifies the waveform before transmitting the amplified waveform to a tactile output generator 167 (e.g., either tactile output generator 167 (Figure 1A) or 357 (Figure 3)). For example, the amplifier 163 amplifies the received waveform to a signal level according to the physical specifications of the tactile output generator 167 (e.g., the voltage and / or current required by the tactile output generator 167 to generate a tactile output so that the signal transmitted to the tactile output generator 167 generates a tactile output corresponding to the waveform received from the tactile feedback controller 161), and transmits the amplified waveform to the tactile output generator 167. In response, the tactile output generator 167 generates a tactile output (e.g., by reciprocating a movable mass over a distance of one or more distances relative to the neutral position of the movable mass).

[0151] In some embodiments, the tactile output module includes a sensor 167 coupled to a tactile output generator 169. The sensor 169 detects the state or a change in state (e.g., mechanical position, physical displacement, and / or movement) of the tactile output generator 167 or one or more components of the tactile output generator 167 (e.g., one or more moving parts, such as a membrane used to generate tactile output). In some embodiments, the sensor 169 is a magnetic field sensor (e.g., a Hall effect sensor) or another displacement and / or motion sensor. In some embodiments, the sensor 169 provides information (e.g., the position, displacement, and / or movement of one or more parts in the tactile output generator 167) to a tactile feedback controller 161, and the tactile feedback controller 161 modulates the waveform output from the tactile feedback controller 161 (e.g., the waveform transmitted to the tactile output generator 167 via an amplifier 163, optionally) according to the information about the state of the tactile output generator 167 provided by the sensor 169.

[0152] Figure 2 shows a portable multifunctional device 100 having a touchscreen (e.g., a touch-sensitive display system 112, Figure 1A) according to one embodiment. The touchscreen optionally displays one or more graphics within the user interface (UI) 200. In this embodiment, and embodiments described later, the user can select one or more graphics by making gestures on the graphics using, for example, one or more fingers 202 (not shown in the figure to an exact scale) or one or more styluses 203 (not shown in the figure to an exact scale). In some embodiments, the selection of one or more graphics occurs when the user disconnects from one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, right to left, upward and / or downward), and / or rolling of a finger in contact with the device 100 (from right to left, left to right, upward and / or downward). In some implementations or situations, unintended contact with a graphic may not select the graphic. For example, when the gesture corresponding to selection is a tap, a swipe gesture (swiping over an application icon) is optional and will not select the corresponding application.

[0153] Device 100 also optionally includes one or more physical buttons, such as a "Home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that optionally run on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display.

[0154] In some embodiments, device 100 includes a touchscreen display, a menu button 204, a push button 206 for turning the power supply to the device on / off and locking the device, one or more volume control buttons 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and an external docking / charging port 124. The push button 206 is optionally used to turn the power on / off on the device by pressing down and holding the button down for a predetermined time interval, to lock the device by pressing down and releasing the button before the predetermined time interval has elapsed, and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts verbal input through a microphone 113 to activate or deactivate certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output to the user of device 100.

[0155] Figures 2B-2C are exploded views of a first input device suitable for use in the electronic devices shown in Figures 1A, 2A, 3 and / or 4A (e.g., home button 204). Figure 2B shows an example of a force-sensing home button having a capacitive sensor used to determine a range of force values ​​corresponding to the force applied to the force-sensing home button. Figure 2C shows an example of a home button having a mechanical switch element. Referring to Figure 2B, the input device stack 220 includes a cover element 222 and a trim 224. In the illustrated embodiment, the trim 224 completely surrounds the sides and top of the cover element 222. Other embodiments are not limited to this configuration. For example, in one embodiment, the sides and / or top of the cover element 222 may be partially surrounded by the trim 224. Alternatively, in other embodiments, the trim 224 may be omitted.

[0156] Both the cover element 222 and the trim 224 can be formed from suitable opaque, transparent, and / or translucent materials. For example, the cover element 222 may be made of glass, plastic, or sapphire, and the trim 224 may be made of metal or plastic. In some embodiments, one or more additional layers (not shown) can be placed beneath the cover element 222. For example, if the cover element 222 is made of a transparent material, an opaque ink layer can be placed beneath the cover element 222. The opaque ink layer can obscure other components in the input device stack 220 so that they are not visible through the transparent cover element 222.

[0157] The first circuit layer 226 can be located beneath the cover element 222. Any suitable circuit layer may be used. For example, the first circuit layer 226 may be a circuit board or a flexible circuit. The first circuit layer 226 may include one or more circuits, signal lines, and / or integrated circuits. In one embodiment, the first circuit layer 226 includes a biosensor 228. Any suitable type of biosensor can be used. For example, in one embodiment, the biosensor is a capacitive fingerprint sensor that captures at least one fingerprint when the user's finger(s) approach and / or come into contact with the cover element 222.

[0158] The first circuit layer 226 may be attached to the bottom surface of the cover element 222 via an adhesive layer 230. Any suitable adhesive can be used as the adhesive layer. For example, a pressure-sensitive adhesive layer may be used as the adhesive layer 230.

[0159] The conformity layer 232 is located beneath the first circuit layer 226. In one embodiment, the conformity layer 232 includes an opening 234 formed in the conformity layer 232. The opening 234 exposes the upper surface of the first circuit layer 226 and / or the biosensor 228 during the assembly of the device stack 220. In an exemplary embodiment, the conformity layer 232 is located around the inner circumference of the trim 224 and / or the periphery of the cover element 222. The conformity layer 232 is depicted as circular, but can have any given shape and / or dimensions, such as a square or an ellipse. The conformity layer 232 is shown as a continuous conformity layer in Figures 2B and 2C, but other embodiments are not limited to this configuration. In some embodiments, multiple discrete conformity layers may be used within the device stack 220. In addition, in some embodiments, the responsive layer 232 does not include an opening 234, and the responsive layer 232 extends over at least a portion of the input device stack 220. For example, the responsive layer 232 may extend over the bottom surface of the cover element 222 and the bottom surface of the first circuit layer 226, or over a portion of the bottom surface of the cover element 222 (e.g., surrounding the periphery of the cover element) and the bottom surface of the first circuit layer 226.

[0160] The second circuit layer 238 is located below the first circuit layer 226. Flexible circuits and circuit boards are examples of circuit layers that can be used in the second circuit layer 238. In some embodiments, the second circuit layer 238 may include a first circuit portion 240 and a second circuit portion 242. The first and second circuit portions 240 and 242 may be electrically connected to each other.

[0161] The first circuit portion 240 may include a first set of one or more intensity sensor components included in the intensity sensor. In some embodiments, the first circuit portion 240 may be electrically connected to a first circuit layer 226. For example, if the first circuit layer 226 includes a biosensor 228, the biosensor 228 may be electrically connected to the first circuit portion 240 of the second circuit layer 238.

[0162] The second circuit section 242 may include additional circuits such as signal lines, circuit components, and integrated circuits. In one embodiment, the second circuit section 242 may include a board-to-board connector 244 that electrically connects the second circuit layer 238 to other circuits in an electronic device. For example, the second circuit layer 238 may be operably connected to a processing device using a board-to-board connector 244. Additionally or alternatively, the second circuit layer 238 may be operably connected to a circuit that transmits signals (e.g., sense signals) received from one or more intensity sensor components in the first circuit section 240 to a processing device. Additionally or alternatively, the second circuit layer 238 may be operably connected to a circuit that provides signals (e.g., drive signals, reference signals) to one or more intensity sensor components in the first circuit section 240.

[0163] In some embodiments, the first circuit portion 240 of the second circuit layer 238 may be attached to the bottom surface of the first circuit layer 226 using an adhesive layer 236. In non-limiting examples, a die attach film may be used to attach the first circuit portion 240 to the bottom surface of the first circuit layer 226.

[0164] The third circuit layer 246 is located below the first circuit portion 240 of the second circuit layer 238. The third circuit layer 246 may include a second set of one or more intensity sensor components included in the intensity sensor. The third circuit layer 246 is supported by and / or mounted on the support element 248. In one embodiment, the support element 248 is mounted on the trim 224 to form an enclosure for other components in the device stack 220. The support element 248 may be mounted on the trim 224 using a preferred mounting mechanism.

[0165] A first set of one or more intensity sensor components in the first circuit section 240 and a second set of one or more intensity sensor components in the third circuit layer 246 together form an intensity sensor. The intensity sensor can use any suitable intensity sensing technology. Exemplary sensing technologies include, but are not limited to, capacitive, piezoelectric, piezoresistive, ultrasonic, and magnetic.

[0166] In the examples shown in Figures 2B and 2C, the intensity sensor is a capacitive force sensor. For a capacitive force sensor, a first set of one or more intensity sensor components includes a first set of one or more electrodes 250, and a second set of one or more force sensor components includes a second set of one or more electrodes 252. Although shown in a square shape in Figures 2B and 2C, each electrode in the first and second sets 250, 252 can have any given shape (e.g., rectangle, circle). In addition, one or more electrodes in the first and second sets 250, 252 may be arranged in any given pattern (e.g., one or more rows and one or more columns).

[0167] Figures 2B and 2C show two electrodes in the first and second sets of one or more electrodes 250, 252. However, other embodiments are not limited to this configuration. The first and second sets of one or more electrodes 250, 252 may each consist of a single electrode or multiple individual electrodes. For example, if the first set of one or more electrodes is a single electrode, the second set of one or more electrodes may consist of multiple individual electrodes. In some embodiments, the second set of one or more electrodes may be a single electrode, and the first set may include multiple individual electrodes. Alternatively, both the first and second sets of one or more electrodes may each include multiple individual electrodes.

[0168] Each electrode of a first set of one or more electrodes 250 is aligned with each electrode of a second set of one or more electrodes 252 in at least one direction (e.g., perpendicular) to form one or more capacitors. When a force input is applied to a cover element 222 (e.g., the input surface of an input device), at least one electrode in the first set 250 moves toward each electrode in the second set 252, changing the capacitance of the capacitor(s). The capacitance signal sensed from each capacitor represents the capacitance measurement of that capacitor. A processing device (not shown) receives the capacitance signal(s) and is configured to correlate the capacitance signal(s)(s) to the amount of force applied to the cover element 222. In some embodiments, the force sensor can be replaced with a switch element, and different force thresholds can be used to determine the activation event.

[0169] In some embodiments, such as the embodiment shown in Figure 2C, the switch element 54 may be positioned below the support element 248. The switch element 254 registers a user input when the force input applied to the cover element 222 exceeds a predetermined force (e.g., a force threshold related to the approach of the distance between the first circuit portion 240 and the third circuit layer 246). Any suitable switch element can be used. For example, the switch element 254 may be a dome switch that depresses when the force input applied to the cover element 222 exceeds a force threshold. When depressed, the dome switch terminates a circuit that is detected by the processing device and recognized as a user input (e.g., selection of an icon, function, or application). In one embodiment, the dome switch is positioned so that the top of the depressable dome is close to the bottom surface of the support plate 248. In another embodiment, the base of the depressable dome may be close to the bottom surface of the support plate 248.

[0170] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to one embodiment. The device 300 does not need to be portable. In some embodiments, the device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia playback device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (e.g., a home or commercial controller). The device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication buses 320 optionally include circuitry (sometimes called a chipset) that interconnects and controls communication between system components. The device 300 typically includes an input / output (I / O) interface 330, which includes a display 340, which is a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350, and a touchpad 355, a tactile output generator 357 for generating tactile output on a device 300 (similar to the tactile output generator(s) 167 described above with reference to Figure 1A, for example), and a sensor 359 (e.g., an optical sensor, an accelerometer, a proximity sensor, a touch sensor, and / or a contact intensity sensor similar to the contact intensity sensor(s) 165 described above with reference to Figure 1A). The memory 370 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and also optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or subsets thereof, that are stored in memory 102 of the portable multifunction device 100 (Figure 1A). Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of the portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk authoring module 388, and / or a spreadsheet module 390, while memory 102 of the portable multifunction device 100 (Figure 1A) optionally does not store those modules.

[0171] Each of the elements identified in Figure 3 above is optionally stored in one or more of the previously mentioned memory devices. Each of the identified modules corresponds to an instruction set that performs the function described above. The identified modules or programs (i.e., instruction sets) do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of those modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the identified modules and data structures. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0172] Here, we focus on the embodiments of the user interface ("UI") that are optionally implemented on the portable multi-functional device 100.

[0173] Figure 4A shows an exemplary user interface for an application menu on a portable multifunction device 100 according to one embodiment. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or a subset or superset thereof. ● Signal strength indicators (single or multiple) for wireless communication (single or multiple) such as cellular and Wi-Fi signals 402, ●Time 404, ●Bluetooth (registered trademark) indicator, ●Battery status indicator 406, ●Tray 408 containing icons for frequently used applications such as the following: An icon 416 for the phone module 138, labeled "Phone", optionally including an indicator 414 for the number of missed calls or voicemail messages. An optional icon 418 for the email client module 140, labeled "Mail," which includes an indicator 410 for the number of unread emails. ○ Icon 420 for browser module 147 labeled "Browser", and ○ Icon 422 for the video and music player module 152, also known as the iPod (trademark of Apple Inc.) module 152, which is labeled "iPod," and ● Icons for other applications, such as the ones listed below. ○ Icon 424 for IM module 141 labeled "Messages", ○ Icon 426 for calendar module 148, labeled "Calendar". ○ Icon 428 for image management module 144, labeled "Photos". ○ Icon 430 for camera module 143, labeled "Camera". ○ Icon 432 for online video module 155, labeled "Online Video" ○ Icon 434 for stock price widget 149-2, labeled "Stocks (stock price)" ○ Icon 436 for map module 154, labeled "Maps". ○ Icon 438 for weather widget 149-1, labeled "Weather". ○ Icon 440 for alarm clock widget 149-4, labeled "Clock". ○ Icon 442 for training support module 142, labeled "Workout Support". ○ Icon 444 for memo module 153, which is labeled "Notes," and ○ An icon 446 for a configuration application or module that provides access to settings related to device 100 and its various applications 136.

[0174] It should be noted that the icon labels shown in Figure 4A are merely examples. For example, in some embodiments, the icon 422 for the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label for each application icon includes the name of the application to which that application icon corresponds. In some embodiments, the label for a particular application icon is separate from the name of the application to which that particular application icon corresponds.

[0175] Figure 4B shows an exemplary user interface on a device (e.g., device 300, Figure 3) having a touch-sensitive surface 451 (e.g., a tablet or touchpad 355, Figure 3) separate from the display 450. Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357) for detecting the intensity of contact on the touch-sensitive surface 451, and / or one or more tactile output generators 359 for generating tactile output for the user of device 300.

[0176] Many of the following embodiments are given by referring to input on a touchscreen display 112 (when the touch-sensing surface and the display are combined), but in some embodiments, the device detects input on a touch-sensing surface separated from the display, as shown in Figure 4B. In some embodiments, the touch-sensing surface (e.g., 451 in Figure 4B) has a principal axis (e.g., 452 in Figure 4B) corresponding to a principal axis (e.g., 453 in Figure 4B) on the display (e.g., 450). According to those embodiments, the device detects contact (e.g., 460 and 462 in Figure 4B) with the touch-sensing surface 451 at positions corresponding to each of the positions on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in Figure 4B). Thus, when the touch-sensing surface is separated from the display, user input (e.g., contacts 460 and 462, and their movement) detected by the device on the touch-sensing surface (e.g., 451 in Figure 4B) is used by the device to operate the user interface on the display of the multifunction device (e.g., 450 in Figure 4B). Please understand that a similar method may be used, at your discretion, for other user interfaces described herein.

[0177] In addition, while the following embodiments are given primarily with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture, etc.), it should be understood that in some embodiments, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced with a mouse click (e.g., instead of touch), followed by cursor movement along the swipe path (e.g., instead of touch movement). As another example, a tap gesture may optionally be replaced with a mouse click (e.g., instead of touch detection, followed by cessation of touch detection) while the cursor is positioned over the tap gesture location. Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice may optionally be used simultaneously, or a mouse and finger touch may optionally be used simultaneously.

[0178] As used herein, the term “focus selector” refers to an input element that indicates the current portion of the user interface with which the user is interacting. In some implementations, including a cursor or other location marker, the cursor functions as a “focus selector” such that when input (e.g., press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in Figure 3, or touch-sensitive surface 451 in Figure 4B) while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element adjusts according to the detected input. In some implementations, including a touchscreen display (e.g., touch-sensitive display system 112 in Figure 1A, or touchscreen in Figure 4A) that enables direct interaction with user interface elements on the touchscreen display, contact detected on the touchscreen functions as a “focus selector” such that when input (e.g., press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element adjusts according to the detected input. In some implementations, focus is moved from one area of ​​the user interface to another without the movement of a corresponding cursor or touch on the touchscreen display (for example, by moving focus from one button to another using the tab key or arrow keys). In these implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally a user-controlled user interface element (or touch on the touchscreen display) to communicate the user's intended interaction with the user interface (for example, by indicating to the device the user interface element that the user intends to interact with).For example, the position of the focus selector (e.g., cursor, touch, or selection box) above each button while pressure input is detected on a touch-sensitive surface (e.g., touchpad or touchscreen) indicates that the user intends to activate that button (in contrast to other user interface elements shown on the device's display).

[0179] As used herein and in the claims, the term “strength” of contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of contact on the touch-sensitive surface (e.g., finger contact or stylus contact), or a proxy for the force or pressure of contact on the touch-sensitive surface. The strength of contact has a range of values, including at least four distinct values, and more typically hundreds (e.g., at least 256) distinct values. The strength of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average or sum) to determine an estimated force of contact. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or variation of the contact area detected on the touch-sensitive surface, the volume and / or variation of the touch-sensitive surface adjacent to the contact, and / or the resistance and / or variation of the touch-sensitive surface adjacent to the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the substitute measurement for the force or pressure of the contact is used directly to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some implementations, the substitute measurement for the force or pressure of the contact is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of contact as an attribute of user input allows for user access to additional device functions that might otherwise be difficult to access, in reduced-size devices where the area for displaying affordances (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical control such as a knob or button) is limited.

[0180] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (for example, to determine whether a user has "clicked" on an icon). In some embodiments, at least a subset of the intensity thresholds is determined according to software parameters (for example, the intensity thresholds may not be determined by activation thresholds of a particular physical actuator and may be adjusted without changing the physical hardware of device 100). For example, the mouse "click" threshold for a trackpad or touchscreen display may be set to one of a wide range of default thresholds without changing the hardware of the trackpad or touchscreen display. Furthermore, in some implementations, the user of the device is provided with software settings to adjust one or more of the set of intensity thresholds (for example, by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using a system-level click "intensity" parameter).

[0181] As used herein and in the claims, the term “characteristic intensity” of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predetermined number of intensity samples, or on a set of intensity samples collected at predetermined time intervals (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) for a predetermined event (e.g., after detection of contact, before detection of lift-off of contact, before or after detection of the start of movement of contact, before detection of the end of contact, before or after detection of an increase in contact intensity, and / or before or after detection of a decrease in contact intensity). The characteristic intensity of a contact is optional and is based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the top 10% value of the contact intensity, half the maximum value of the contact intensity, 90% of the maximum value of the contact intensity, or a value generated by low-pass filtering of the contact intensity over a predetermined period or at a predetermined time. In some embodiments, the duration of the contact is used when determining the characteristic intensity (for example, when the characteristic intensity is the average of the contact intensity over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether the operation was performed by a user. For example, the set of one or more intensity thresholds may include a first intensity threshold and a second intensity threshold. In this embodiment, a first operation is performed as a result of a contact with a characteristic intensity not exceeding the first threshold, a second operation is performed as a result of a contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold, and a third operation is performed as a result of a contact with a characteristic intensity exceeding the second intensity threshold. In some embodiments, the comparison between the characteristic intensity and one or more intensity thresholds is not used to determine whether to perform a first or second action, but rather to determine whether to perform one or more actions (e.g., to perform each option or to refrain from performing each action).

[0182] In some embodiments, a portion of the gesture is identified for the purpose of determining its characteristic intensity. For example, a touch-sensitive surface may receive a continuous swipe contact (e.g., a drag gesture) that transitions from a start location to an end location, where the intensity of the contact increases. In this embodiment, the characteristic intensity of the contact at the end location may be based on only a portion of the continuous swipe contact (e.g., only the portion of the swipe contact at the end location), rather than the entire continuous swipe contact. In some embodiments, a smoothing algorithm may be applied to the intensity of the swipe contact before determining its characteristic intensity. For example, the smoothing algorithm may optionally include one or more of the following: a non-weighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some situations, these smoothing algorithms exclude small increases or decreases in the intensity of the swipe contact for the purpose of determining its characteristic intensity.

[0183] The user interface diagrams described herein may optionally include one or more intensity thresholds (e.g., contact detection intensity threshold IT0, shallow pressure intensity threshold IT0). L Deep pressure intensity threshold IT D (For example, at least initially IT L (higher than TI), and / or one or more other intensity thresholds (e.g., TI L Lower intensity threshold IT HThis includes a chart of various intensity levels, showing the current intensity of contact on the touch-sensitive surface relative to the touch-sensitive surface. This chart of intensity levels is typically not part of the displayed user interface but is provided to aid in the interpretation of the diagram. In some embodiments, a shallow press intensity threshold corresponds to the intensity at which the device performs an operation typically associated with clicking a physical mouse button or trackpad. In some embodiments, a deep press intensity threshold corresponds to the intensity at which the device performs an operation different from an operation typically associated with clicking a physical mouse button or trackpad. In some embodiments, when contact is detected at a characteristic intensity below the shallow press intensity threshold (for example, above a nominal contact detection intensity threshold IT0 below which contact is no longer detected), the device moves the focus selector in accordance with the movement of contact on the touch-sensitive surface without performing an operation associated with the shallow or deep press intensity threshold. Generally, unless otherwise noted, these intensity thresholds are consistent across different sets of diagrams in the user interface.

[0184] In some embodiments, the device's response to an input detected by the device depends on a criterion based on the contact intensity between inputs. For example, for some “shallow press” inputs, a contact intensity exceeding a first intensity threshold between inputs triggers a first response. In some embodiments, the device's response to an input detected by the device depends on a criterion that includes both a contact intensity and time-based criterion between inputs. For example, for some “deep press” inputs, a contact intensity exceeding a second intensity threshold greater than a first intensity threshold for shallow presses between inputs triggers a second response only if a delay time has elapsed between satisfying the first and second intensity thresholds. This delay time is typically less than 200 ms in duration (e.g., 40 ms, 100 ms, or 120 ms, depending on the magnitude of the second intensity threshold, with the delay time increasing as the second intensity threshold increases). This delay time helps avoid accidental recognition of deep press inputs. In another embodiment, for some “deep presses,” there is a period of reduced sensitivity that occurs after the time the first intensity threshold is satisfied. During the period of reduced sensitivity, the second intensity threshold increases. This temporary increase in the second intensity threshold also helps to avoid accidental deep press inputs. For other deep press inputs, the response to the detection of deep press inputs is independent of time-based criteria.

[0185] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, the application being run, the rate at which intensity is applied, the number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), and the position of the focus selector. Illustrative factors are described in U.S. Patent Applications 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety.

[0186] For example, FIG. 4C shows a dynamic intensity threshold 480 that changes over time based in part on the intensity of the touch input 476 over time. The dynamic intensity threshold 480 is the sum of two components, a first component 474 that decays over time after a predefined delay time p1 from when the touch input 476 is first detected, and a second component 478 that tracks the intensity of the touch input 476 over time. The initial high intensity threshold of the first component 474 reduces the chance of accidentally triggering a "deep press" response and still allows an immediate "deep press" response when the touch input 476 provides sufficient intensity. The second component 478 reduces the chance of accidentally triggering a "deep press" response due to gradual intensity variations in the touch input. In some embodiments, when the touch input 476 meets the dynamic intensity threshold 480 (e.g., at point 481 in FIG. 4C), a "deep press" response is triggered.

[0187] FIG. 4D shows another dynamic intensity threshold 486 (e.g., intensity threshold I D ). FIG. 4D also shows two other intensity thresholds, a first intensity threshold IT H and a second intensity threshold I L . In FIG. 4D, the touch input 484 meets the first intensity threshold IT H and the second intensity threshold IT L before time p2, but no response is provided until the delay time p2 has elapsed at time 482. Also in FIG. 4D, the dynamic intensity threshold 486 decays over time with an attenuation that starts at time 488 after the predefined delay time p1 has elapsed from time 482 (when the response associated with the second intensity threshold IT L is triggered). This type of dynamic intensity threshold reduces the chance of accidentally triggering a response associated with the dynamic intensity threshold I TH or the second intensity threshold I L immediately after or at the same time as triggering a response associated with a lower intensity threshold such as the first intensity threshold I TD .

[0188] FIG. 4E shows yet another dynamic intensity threshold 492 (e.g., intensity threshold I D ). In FIG. 4E, the intensity threshold ITL The associated response is triggered after a delay time p2 has elapsed from the time the touch input 490 is first detected. Simultaneously, the dynamic intensity threshold 492 decays after a predetermined delay time p1 has elapsed from the time the touch input 490 is first detected. Therefore, without releasing the touch input 490, the intensity threshold I L An increase in the intensity of touch input 490 after triggering a response related to touch input 490, followed by a decrease in the intensity of touch input 490, indicates that the intensity of touch input 490 is at another intensity threshold, e.g., intensity threshold I L Even when it falls below (for example, at time 494), the intensity threshold IT D This can trigger a response related to that.

[0189] Shallow pressure intensity threshold IT L From strength below, shallow pressure intensity threshold IT L and deep pressure intensity threshold IT D The increase in characteristic intensity of contact between the intensity and the depth threshold is sometimes referred to as a "shallow pressure" input. D From below a certain intensity to a deep pressure intensity threshold (IT) D The increase in characteristic intensity of contact exceeding a certain intensity is sometimes referred to as a "deep press" input. From an intensity below the contact detection intensity threshold IT0 to the contact detection intensity threshold IT0 and the shallow press intensity threshold IT0 L An increase in the characteristic intensity of contact between a certain intensity and a certain intensity may be referred to as the detection of contact on the touch surface. A decrease in the characteristic intensity of contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 may be referred to as the detection of lift-off of contact from the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some figures, shaded circles or ellipses are used to represent the intensity of contact on the touch-sensing surface. In some figures, unshaded circles or ellipses are used to represent each contact on the touch-sensing surface without specifying the intensity of each contact.

[0190] In some embodiments described herein, one or more operations are performed in response to the detection of a gesture including each press input, or in response to the detection of each press input performed on each (or more) contact, each press input is detected at least in part on the detection of an increase in the intensity of the contact (or more) above a press input intensity threshold. In some embodiments, each operation is performed in response to the detection of an increase in the intensity of each contact above a press input intensity threshold (for example, each operation is performed on the "downstroke" of each press input). In some embodiments, a press input includes an increase in the intensity of each contact above a press input intensity threshold, and a subsequent decrease in the intensity of the contact below the press input intensity threshold, and each operation is performed in response to the detection of a subsequent decrease in the intensity of each contact below the press input threshold (for example, each operation is performed on the "upstroke" of each press input).

[0191] In some embodiments, the device employs intensity hysteresis to avoid accidental inputs, which may be referred to as "jitter," and the device defines or selects a hysteresis intensity threshold that has a predefined relationship with a press input intensity threshold (for example, the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable ratio of the press input intensity threshold). Thus, in some embodiments, the press input includes an increase in the intensity of each contact above the press input intensity threshold, and a subsequent decrease in the intensity of the contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and each operation is performed in response to the detection of a subsequent decrease in the intensity of each contact below the hysteresis intensity threshold (for example, each operation is performed on the "upstroke" of each press input). Similarly, in some embodiments, a press input is detected only when the device detects an increase in the intensity of contact from an intensity below a hysteresis intensity threshold to an intensity above a press input intensity threshold, and optionally a subsequent decrease in the intensity of contact to an intensity below the hysteresis intensity, and each operation is performed in response to the detection of a press input (e.g., a situation-dependent increase in the intensity of contact, or a decrease in the intensity of contact).

[0192] For the sake of clarity, the description of operations performed in response to a press input associated with a press input intensity threshold, or in response to a gesture involving a press input, is optional and is triggered in response to the detection of an increase in contact intensity above the press input intensity threshold, an increase in contact intensity from below the hysteresis intensity threshold to above the press input intensity threshold, a decrease in contact intensity below the press input intensity threshold, or a decrease in contact intensity below the hysteresis intensity threshold corresponding to the press input intensity threshold. In addition, in embodiments described as being performed in response to the detection of a decrease in contact intensity below the press input intensity threshold, the operation is optional and is performed in response to the detection of a decrease in contact intensity below a hysteresis intensity threshold corresponding to and lower than the press input intensity threshold. As described above, in some embodiments, the triggering of these responses also depends on a criterion based on the time it takes to satisfy (for example, a delay time elapses between the satisfaction of a first intensity threshold and the satisfaction of a second intensity threshold). User interface and related processes

[0193] Herein, we focus on embodiments of a user interface ("UI") and associated processing that may be implemented on an electronic device such as a portable multifunction device 100 or device 300, comprising a display, a touch-sensitive surface, one or more tactile output generators for generating tactile output, one or more sensors for detecting contact intensity with a device button (e.g., a virtual or physical home button), and (optionally) one or more sensors for detecting contact intensity with the touch-sensitive surface.

[0194] Figures 5A1 to 5C19 show exemplary user interfaces for providing tactile output and visual feedback in response to multiple types of input on a device button (e.g., a physical home button or a virtual home button) according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 7A to 7G, 9A to 9D, 11A to 11E, 13A to 13D, 15A to 15E, 17A to 17D, and 19A to 19C. For convenience of explanation, some embodiments are discussed in reference to operations performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally the contact of each finger or stylus, a representative point corresponding to the finger or stylus contact (e.g., the center of each contact or a point associated with each contact), or the center of two or more contacts detected on the touch-sensitive display system 112. However, similar operations are optional and performed on devices having the display 450 and another touch-sensing surface 451, in response to the detection of contact on the touch-sensing surface 451 while the user interface shown in the figure is displayed on the display 450, along with the focus selector.

[0195] Figures 5A1 to 5A18 show exemplary user interfaces for providing haptic and visual feedback in response to button interaction, according to some embodiments. Figures 5A1 to 5A18 show opening an application from the home screen and providing visual and haptic feedback in response to multiple types of input on the home button (e.g., button 204).

[0196] Figure 5A1 shows a user interface 510 for an application menu on device 100 (e.g., the primary page of a multi-page home screen) according to one embodiment. User interface 510 includes elements similar to user interface 400 described above with respect to Figure 4A, but for brevity, a comprehensive description of the elements of user interface 510 is not provided here.

[0197] Figures 5A2 to 5A3 show an example of detecting input on the Clock icon (for example, a tap gesture via contact 502 in Figure 5A2) and displaying the user interface 512 (Figure 5A3) of the Clock application.

[0198] Figures 5A4-5A8 show an example of detecting changes in the intensity of contact (e.g., contact 504) on a home button (e.g., button 204) and providing visual feedback regarding the activation of the home button according to that intensity (e.g., indicating the start of a transition from user interface 512 back to user interface 510). In Figure 5A4, the intensity of contact 504-a is still at the "hint" intensity threshold IT. H Since it has not reached the threshold, no visual feedback is displayed. In Figures 5A5-5A7, the intensity of contact 504 is the hint intensity threshold IT. H When the intensity of contact increases beyond a certain point, the visual feedback changes dynamically according to the intensity of contact. For example, as the intensity of contact increases from Figure 5A5 to Figure 5A6, the transition from user interface 512 to user interface 510 (e.g., from user interface 513-1 to user interface 513-2) progresses, and as the intensity of contact decreases from Figure 5A6 to 5A7, the transition from user interface 512 to user interface 510 reverses (e.g., from user interface 513-2 to user interface 513-1). The intensity of contact 504 is a shallow pressure intensity threshold IT L Since the value does not increase beyond a certain point, when the end of input via contact 504 is detected, the user interface 512 of the Clock application is redisplayed on the display (Figure 5A8).

[0199] Figures 5A9-5A14 show an example of detecting changes in the intensity of contact (e.g., contact 508) on a home button (e.g., button 204), providing visual feedback regarding the activation of the home button (e.g., displaying the start of a transition from user interface 512 to user interface 510) according to that intensity, and providing tactile output in response to input on the home button (e.g., button 204). In Figures 5A9-5A11, as the intensity of contact increases (e.g., from contact 508-a to contact 508-b, to contact 508-c), the transition from user interface 512 to user interface 510 (e.g., from user interface 513-1 to user interface 513-2, to user interface 513-3) progresses. In Figure 5A11, the intensity of contact 508-c is the "shallow press" intensity threshold IT on the home button. L When the pressure increases to (for example, a down-click of button 204, or sometimes called a “press event”), device 100 provides a tactile output 503 (e.g., a mini-tap at 270Hz with a gain of 0.5). In Figure 5A13, the intensity of contact 508-e reaches the “shallow press release” intensity threshold IT. LR When the pressure decreases to a certain level (for example, an up-click of button 204, sometimes called a "release event"), device 100 provides a different tactile output, such as a tactile output 505 (e.g., a mini-tap at 270Hz with a gain of 0.3). The intensity of the contact 508 is shallow, and the pressure intensity threshold is low. L The input increases to (for example, Figure 5A11), and the end of the input by contact 508 is detected (for example, by the release event in Figure 5A13), and the transition from user interface 512 to user interface 510 (for example, from user interface 513-3 in Figure 5A11 to user interface 513-4 in Figure 5A12, to user interface 513-5 in Figure 5A13, to user interface 510 in Figure 5A14) continues until completion as shown in Figures 5A11 to 5A14. The tactile outputs shown in Figures 5A11 and 5A13 correspond to a first range of values ​​for the intensity change metric (e.g., "weak" intensity), as will be described below with respect to method 1500.

[0200] Figures 5A15 to 5A16 show that the pressing input by contact 508 reaches a threshold time (e.g., a long pressing time T). LP An alternative sequence from Figure 5A12 is shown for cases where liftoff does not occur until more than ) has elapsed. In this case, the animation transition from user interface 512 to user interface 510 is interrupted (e.g., in user interface 513-6 in Figure 5A16) and user interface 515 (e.g., the Auto Assistant user interface) is displayed. In some embodiments, if the animation transition from user interface 512 to user interface 510 is interrupted, another animation transition is displayed (e.g., from the interruption point to user interface 515), as shown in user interface 514-1 in Figure 5A17. In some embodiments, as shown in Figure 5A16, the intensity of contact 508-f exceeds a threshold time (e.g., T LP During the interval, shallow pressure intensity threshold IT L If the contact is maintained above T, device 100 provides a tactile output 507 (e.g., a double mini-tap 270Hz or micro-tap 270Hz with a gain of 0.9). In some embodiments, once the transition to the automated assistant user interface (e.g., user interface 515 in Figure 5A18) is complete, as shown in Figure 5A18, device 100 provides a tactile output 509 (e.g., a double mini-tap 270Hz or micro-tap 270Hz with a gain of 0.9). In some embodiments, device 100 provides a tactile output 509 (e.g., a double mini-tap 270Hz or micro-tap 270Hz with a gain of 0.9). LP Shallow pressure intensity threshold IT as a function of threshold time L A tactile output is provided either when the pressure is held above a certain level (e.g., tactile output 507 in Figure 5A16) or when the automated user interface is completed (e.g., tactile output 509 in Figure 5A18), but not both.

[0201] In the examples in Figures 5A1 to 5A18, a press event (e.g., shallow contact intensity threshold IT) on the home button (e.g., button 204) is triggered. L (When it increases to) and release events (e.g., when the contact strength is shallow, press release strength IT) LRThe tactile output for when the pressure decreases to a certain level is shown as a 270Hz minitap (for example, when the second tactile output setting is selected, as described below with reference to Figures 5C1-5C9). In some embodiments, when the first tactile output setting is selected (for example, as described below with reference to Figures 5C13-5C19), the tactile output in these examples may be a 230Hz minitap. In some embodiments, when the third tactile output setting is selected (for example, as described below with reference to Figures 5C10-5C12), the tactile output in these examples may be a 300Hz minitap. Similarly, in Figures 5B1-5B75, the tactile outputs for press and release events are shown as a 270Hz minitap (for example, when the second tactile output setting is selected), but when the first or third tactile output setting is selected, the tactile outputs may be a 230Hz or 300Hz minitap, respectively.

[0202] Figures 5B1 to 5B75 show exemplary user interfaces for providing tactile and visual feedback for interacting with buttons, according to some embodiments. Figures 5B1 to 5B75 show opening an application from a folder on a secondary page of a multi-page home screen and providing visual feedback and tactile output in response to multiple types of input on a home button (e.g., button 204).

[0203] Figure 5B1 shows a user interface 520 for an application menu on device 100 (e.g., a secondary page of a multi-page home screen) according to one embodiment. User interface 520 includes elements similar to user interface 400 described above with respect to Figure 4A, but for brevity, a comprehensive description of the elements of user interface 520 is not provided here.

[0204] Figures 5B2-5B3 show an example of detecting input on the photo folder (for example, a tap gesture via contact 530 in Figure 5B2) and displaying the user interface 522 (Figure 5B3) of the photo folder.

[0205] Figures 5B4-5B5 show an example of detecting input on the Pages icon in the user interface 522 (for example, a tap gesture via touch 531 in Figure 5B4) and displaying the user interface 524 of the Pages application (for example, Figure 5B5).

[0206] Figures 5B6 to 5B14 show (for example, a first press event, followed by a first release event, and then a second press event, where the first and second press events occur at a double-click time threshold T) DC This example shows how to detect double-clicks that are temporally close (detected within each other's threshold time quantities), where both the first and second press events occur within the interrupt time threshold T. I These events occur before threshold time quantities such as the first and second press events occur before the interrupt time threshold T. I Because it occurs before (for example, at the first point where the second press event is detected, as shown in Figure 5B10), the animation transition from user interface 524 (e.g., Pages application) to user interface interface 522 (e.g., photo folder) is interrupted.

[0207] Figures 5B6-5B10 show the start of an animation transition from user interface 524 (e.g., page application) to user interface 522 (e.g., photo folder). In Figure 5B6, the intensity of contact 532-a is the shallow pressure threshold IT on the home button. L When the pressure increases to (for example, the first down-click of button 204, or sometimes called the "first press event"), device 100 provides a tactile output 571 (e.g., a mini-tap of 270Hz with a gain of 1). In Figure 5B8, the intensity of contact 532-c is equal to the shallow release intensity threshold IT. LRWhen the intensity decreases to (for example, the first up-click of button 204, or sometimes called the "first release event"), device 100 provides a different tactile output, such as a tactile output 572 (e.g., a mini-tap of 270Hz with a gain of 0.5). In Figure 5B10, the intensity of contact 532-e is reached when the double-click time threshold T is reached. DC During the second period within the period, the shallow pressure intensity threshold on the home button IT L When the intensity increases to (for example, a second down-click of button 204, or sometimes referred to as a "second press event"), device 100 provides another tactile output, such as tactile output 573 (e.g., a mini-tap of 270 Hz with a gain of 0.8). The tactile outputs shown in Figures 5B6, 5B8, and 5B10 correspond to a second range of intensity change metric values ​​(e.g., "normal" intensity), as described below with respect to method 1500.

[0208] In Figure 5B10, when a second press event is detected, the animation transition from user interface 524 (e.g., the Pages application) to user interface 522 (e.g., the photo folder) is interrupted at the first point (e.g., user interface 525-5), and the animation transition to user interface 528 (e.g., the multitasking user interface) starts from the interruption point. Figures 5B11-5B13 show the animation transition from the interruption point to user interface 528 (e.g., from user interface 526-1 in Figure 5B11 to user interface 526-2 in Figure 5B12, and to user interface 526-3 in Figure 5B13). Figure 5B14 shows user interface 528 (e.g., the multitasking user interface) with the Pages application (e.g., user interface 524) on top.

[0209] Figures 5B15-5B16 show an example of detecting input on a home button (e.g., button 204) (e.g., a press input via contact 533 in Figure 5B15) and displaying the user interface 524 of the Pages application (Figure 5B16). Figure 5B16 shows the same starting point as Figure 5B5. The tactile output shown in Figure 5B15 corresponds to a first range of intensity change metric values ​​(e.g., "weak" intensity), as described below with respect to Method 1500.

[0210] Figures 5B17–5B25 show alternative sequences to those in Figures 5B6–5B14, starting from the user interface 524 of the Pages application shown in Figure 5B5. Figures 5B17–5B25 show double-clicks that are further spaced apart in time (compared to Figures 5B5–5B14), for example, a first press event, followed by a first release event, and then a second press event, where the first and second press events occur within a double-click time threshold T DC This example shows how to detect (such as, detected within each other's threshold time quantities), where both the first and second press events are detected within the interrupt time threshold T. I These events occur before threshold time quantities such as the first and second press events occur before the interrupt time threshold T. I Because this occurs before the animation transition from user interface 524 (e.g., Pages application) to user interface interface 522 (e.g., photo folder) is interrupted (for example, at the second point where the second press event is detected, as shown in Figure 5B21).

[0211] Figures 5B17-5B21 show the start of an animation transition from user interface 524 (e.g., page application) to user interface 522 (e.g., photo folder). In Figure 5B17, the intensity of contact 534-a is the shallow pressure threshold IT on the home button. L When the pressure increases to (for example, a first down-click of button 204, or a "first press event"), device 100 provides a tactile output 575 (for example, a mini-tap of 270Hz with a gain of 1). In Figure 5B19, the intensity of contact 534-c is equal to the shallow press release intensity threshold IT.LR When the intensity decreases to (for example, a first up-click of button 204, or a "first release event"), device 100 provides a different tactile output, such as a tactile output 576 (e.g., a mini-tap at 270Hz with a gain of 0.5). In Figure 5B21, the intensity of contact 534-e on the home button reaches the double-click time threshold T. DC During a second time interval (e.g., a second down-click or second press event of button 204), device 100 provides another tactile output, such as tactile output 577 (e.g., a mini-tap of 270Hz with a gain of 0.8). The tactile outputs shown in Figures 5B17, 5B19, and 5B21 correspond to a second range of values ​​for the intensity change metric (e.g., "normal" intensity), as described below with respect to method 1500.

[0212] In Figure 5B21, when a second press event is detected, the animation transition from user interface 524 (e.g., the Pages application) to user interface 522 (e.g., the photo folder) is interrupted at a second point (e.g., user interface 525-6) which is later than the first point (e.g., user interface 525-5 in Figure 5B10), and the animation transition to user interface 528 (e.g., the multitasking user interface) begins from the second interruption point. Figures 5B22-5B24 show the animation transition from the second interruption point to user interface 528 (e.g., from user interface 527-1 in Figure 5B22, to user interface 527-2 in Figure 5B23, and to user interface 527-3 in Figure 5B24). Figure 5B25 shows user interface 528 (e.g., the multitasking user interface) with the Pages application (e.g., user interface 524) on top.

[0213] Figures 5B26-5B27 show an example of detecting input on a home button (e.g., button 204) (e.g., a press input by contact 535 in Figure 5B26) and displaying the user interface 524 of the Pages application (Figure 5B27). Figure 5B27 shows the same starting point as Figure 5B5. The tactile output shown in Figure 5B26 corresponds to a first range of intensity change metric values ​​(e.g., "weak" intensity), as described below with respect to Method 1500.

[0214] Figures 5B28-5B38 show alternative sequences to those in Figures 5B6-5B14 and 5B17-5B25, starting from the user interface 524 of the page application shown in Figure 5B5. Figures 5B28-5B38 show an example of detecting a double-click that is further separated in time (compared to Figures 5B5-5B14 and 5B16-5B25) (for example, a first press event, followed by a first release event, followed by a second press event, where the first and second press events occur within a double-click time threshold T). DC (These indicate that they are detected within the mutual threshold time quantities), and the first press event is interrupted by the interrupt time threshold T I The second press event occurs before the threshold time amount such as T, and the interrupt time threshold T I The second press event occurs after the interrupt time threshold T. I Because it occurs afterward, the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder) is not interrupted before the animation transition to the multitasking user interface (e.g., user interface 529, Figure 5B38).

[0215] Figures 5B28-5B34 show the animation transition from user interface 524 (e.g., page application) to user interface 522 (e.g., photo folder). In Figure 5B28, the intensity of contact 536-a is the shallow pressure threshold IT on the home button. LWhen the pressure increases to (for example, a first down-click or first press event of button 204), device 100 provides a tactile output 579 (e.g., a mini-tap of 270Hz with a gain of 1). In Figure 5B30, the intensity of contact 536-c reaches the shallow release intensity threshold IT. LR When the intensity decreases to (for example, the first up-click of button 204, or the first release event), device 100 provides a different tactile output, such as tactile output 580 (e.g., a mini-tap of 270Hz with a gain of 0.5). In Figure 5B32, the intensity of contact 536-e is reached when the double-click time threshold T is reached. DC During the second period within the period, the shallow pressure intensity threshold on the home button IT L When the intensity increases to (for example, a second down-click or second press event of button 204), device 100 provides another tactile output, such as tactile output 581 (e.g., a mini-tap of 270Hz with a gain of 0.8). The tactile outputs shown in Figures 5B28, 5B30, and 5B32 correspond to a second range of intensity change metric values ​​(e.g., "normal" intensity), as described below with respect to method 1500.

[0216] In Figure 5B32, when a second pressing event is detected, the second pressing event triggers an interrupt time threshold T. I The animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder) is not interrupted because it occurs later. Figures 5B33-5B34 show the continuation of the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder), and Figure 5B35 shows user interface 522. Figures 5B36-5B37 show the animation transition from user interface 522 (e.g., photo folder) to user interface 529. Figure 5B38 shows user interface 529 (e.g., multitasking user interface) with the photo folder (e.g., user interface 522) at the top.

[0217] Figures 5B40-5B48 are alternative sequences to Figures 5B6-5B14, 5B17-5B25, and 5B28-5B38, starting from the user interface 524 of the page application shown in Figure 5B5. Figures 5B40-5B48 show the double-click time threshold T dc Later, an example is shown where two single clicks are detected when a second press event occurs. The second press event is a double-click time threshold T. DC Because they occur after the previous event, the two press events are treated as two single clicks (as opposed to a double click).

[0218] Figures 5B40-5B46 show the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder). In Figure 5B40, the intensity of contact 537-a is below the shallow pressure threshold IT on the home button. L When the pressure increases to (for example, a down-click of button 204, or a first press event), device 100 provides a tactile output 582 (for example, a mini-tap of 270Hz with a gain of 1). In Figure 5B42, the intensity of contact 537-c is equal to the shallow press release intensity threshold IT. LR When the intensity decreases to (for example, an up-click of button 204, or a first release event), device 100 provides a different tactile output, such as tactile output 583 (e.g., a mini-tap with a gain of 0.5 at 270Hz). In Figure 5B45, the intensity of contact 538-a is reduced to the double-click time threshold T. DC After the time has elapsed, the shallow pressure threshold on the home button IT LWhen the intensity increases to (for example, a down-click of button 204, or a second press event), device 100 provides a tactile output 584 (e.g., a mini-tap of 270Hz with a gain of 1). In some embodiments, the tactile output 584 (Figure 5B45) is a down-click of a single-click input and therefore has the same characteristics as the tactile output 582 (Figure 5B50) (e.g., a mini-tap of 270Hz with a gain of 1). The tactile outputs shown in Figures 5B40, 5B42, and 5B45 correspond to a second range of intensity change metric values ​​(e.g., "normal" intensity), as described below with respect to method 1500.

[0219] In Figure 5B45, when a second press event is detected, the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder) is not interrupted because the second press event is a separate single click. Figure 5B46 shows the continuation of the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder), and Figure 5B47 shows user interface 522. Although not shown, in some embodiments, device 100 displays an animation transition from user interface 522 (Figure 5B47) to user interface 520 (Figure 5B48).

[0220] Figure 5B49 shows a comparison of alternative sequences from user interface 524 (e.g., Pages application), as detailed above. The first row of Figure 5B49 has a double-click that is close in time, and both the first and second press events are T I Figures 5B5-5B14 show the sequence that occurs before. The second row in Figure 5B49 has a double-click where the first and second press events are further apart in time, but both the first and second press events are T I Figures 5B16-5B25 show the sequence that occurs before. The third row in Figure 5B49 shows a double-click where the first and second press events are further separated in time, and the first press event is TI Occurs before, and the second pressure event is T I The sequence that follows is shown in Figures 5B27-5B38. The fourth row in Figure 5B49 has two single clicks, and the first press event is T I Occurs before, and the second pressure event is T DC The sequence that follows is shown in Figures 5B39-5B48.

[0221] Figures 5B50–5B59 show yet another alternative sequence of Figures 5B5–5B15. In Figures 5B50–5B59, the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder) does not begin until after the first release event (e.g., Figure 5B53). Although not shown, in some embodiments, the sequences of Figures 5B16–5B25, 5B27–5B38, and / or Figures 5B39–5B48 have alternative sequences (e.g., Figures 5B19, 5B30, and 5B42, respectively) in which the animation transition from user interface 524 (e.g., Pages application) to user interface 522 (e.g., photo folder) does not begin until after the first release event.

[0222] Figures 5B60-5B61 show an example of detecting input on a home button (e.g., button 204) (e.g., a press input by contact 540 in Figure 5B60) and displaying the user interface 524 of the Pages application (Figure 5B61). The tactile output shown in Figure 5B60 corresponds to a first range of intensity change metric values ​​(e.g., "weak" intensity), as described below with respect to Method 1500.

[0223] Figures 5B62-5B65 show "weak" intensity down click and "weak" intensity up click. In Figure 5B63, the "weak" intensity results in a shallow press intensity threshold IT on the home button where the intensity of contact 541-b is low. LWhen the intensity increases to (for example, for a range of intensity change metric values ​​up to a predetermined number of units of intensity per second, such as 1250 grams per second), device 100 provides a tactile output 590 (for example, a mini-tap at 270 Hz with a gain of 0.5). In Figure 5B65, the intensity of contact 541-d is shallow due to the "weak" intensity, which is the release intensity threshold IT. LR When the intensity decreases to a certain number of units of intensity per second (for example, 1250 grams per second, for a range of intensity change metric values), the device 100 provides a tactile output 591 (for example, a mini-tap at 270 Hz with a gain of 0.25). In some embodiments, the tactile output for a "weak" up-click event is 50% of the tactile output for a "weak" down-click event. In some embodiments, the audio output for a "weak" up-click event is 50% of the audio output for a "weak" down-click event.

[0224] Figures 5B66-5B69 show "normal" intensity down click and "normal" intensity up click. In Figure 5B66, the "normal" intensity corresponds to the intensity of contact 542-b on the shallow press intensity threshold IT on the home button. L When the intensity increases to a certain level (for example, for a range of intensity change metric values ​​greater than a predetermined number of units of intensity per second, such as 1250 grams per second), device 100 provides a tactile output 592 (e.g., a mini-tap of 270 Hz with a gain of 1). In Figure 5B69, the intensity of contact 542-d is shallow at the release intensity threshold IT, where the intensity is "normal". LR When the intensity decreases to a certain level (for example, for a range of intensity change metric values ​​greater than a predetermined number of units of intensity per second, such as 1250 grams per second), the device 100 provides a tactile output 593 (e.g., a mini-tap of 270 Hz with a gain of 0.5). In some embodiments, the tactile output for a "normal" up-click event is 50% of the tactile output for a "normal" down-click event. In some embodiments, the audio output for a "normal" up-click event is 50% of the audio output for a "normal" down-click event.

[0225] Figures 5B70 to 5B75 show an example where the home button (e.g., button 204) is used as the "back" button. Figures 5B71 to 5B72 show an example where, while the user interface 522 (e.g., the photo folder) is displayed, input on the home button (e.g., button 204) is detected (e.g., a press input via contact 543 in Figure 5B71) and the user interface 520 (e.g., the secondary page of a multi-page home screen containing the photo folder) is displayed. Figures 5B73 to 5B74 show an example where, while the user interface 520 (e.g., the secondary page of a multi-page home screen) is displayed, input on the home button (e.g., button 204) is detected (e.g., a press input via contact 544 in Figure 5B73) and the user interface 510 (e.g., the primary page of a multi-page home screen) is displayed. In some embodiments, as shown in Figure 5B75, when an input on the home button (e.g., a press input via contact 545) is detected while the primary page of a multi-page home screen (e.g., user interface 510) is displayed, the device 100 provides a tactile output 596 (e.g., a mini-tap of 270Hz with a gain of 1) but does not change the displayed user interface.

[0226] Figures 5C1 to 5C19 show exemplary user interfaces for a home button configuration process according to some embodiments. Figures 5C1 to 5C19 illustrate an example of the home button configuration process, during which the user selects and tries out multiple tactile output settings for the home button before selecting a tactile output setting for the home button on the device.

[0227] Figures 5C1-5C5 show an example of selecting "Settings," then "General," and then "Home Button." Figures 5C6-5C7 show an example of an animation transition to user interface 564 (for example, the home button configuration user interface with option 2 selected). In Figure 5C8, the tactile output setting is currently selected as option 2. In some embodiments, option 2 is selected as the default tactile output setting. In some embodiments, option 2 corresponds to a second tactile output pattern, such as a mini-tap of 270Hz. Figure 5C9 shows the shallow pressure threshold IT of the contact 553 on the home button. L When the frequency increases to that level, device 100 provides a tactile output 561 (e.g., mini-tap 270Hz) corresponding to the selected tactile output setting (e.g., option 2).

[0228] Figures 5C10-5C11 show an example where option 3 is selected (for example, by a press input via contact 554 on option 3, as shown in Figure 5C10) and user interface 566 (for example, the home button configuration user interface with option 3 selected) is displayed. Although not shown, in some embodiments, if option 2 returns to its original position and option 3 moves toward the home button (for example, button 204), an animation transition is displayed between user interface 564 (Figure 5C10) and user interface 566 (Figure 5C11).

[0229] In Figure 5C11, Option 3 is currently selected as the tactile output setting. In some embodiments, Option 3 corresponds to a third tactile output pattern such as MiniTap 300Hz. In Figure 5C12, the pressure intensity threshold IT of the contact 555 on the home button is shallow. L When increased to that level, device 100 provides a tactile output 565 (e.g., mini-tap 300Hz) corresponding to the selected tactile output setting (e.g., option 3).

[0230] Figures 5C13-5C14 show an example where option 1 is selected (for example, by a press input via contact 556 on option 1, as shown in Figure 5C13) and user interface 568 (for example, the home button configuration user interface with option 1 selected) is displayed. Although not shown, in some embodiments, if option 3 returns to its original position and option 1 moves toward the home button (for example, button 204), an animation transition is displayed between user interface 566 (Figure 5C13) and user interface 568 (Figure 5C14).

[0231] In Figure 5C14, option 1 is currently selected as the tactile output setting. In some embodiments, option 1 corresponds to a first tactile output pattern such as a mini-tap of 230Hz. In Figure 5C15, the intensity of the contact 557 on the home button (e.g., button 204) is a shallow pressure intensity threshold IT. L When the frequency increases to that level, device 100 provides a tactile output 567 (e.g., mini-tap 230Hz) corresponding to the selected tactile output setting (e.g., option 1).

[0232] Figures 5C16-5C17 show an example where "Complete" is selected along with option 1 (for example, using a press input via contact 558 on the "Complete" icon, as shown in Figure 5C16), and the user interface 562 (for example, the General Settings user interface, as shown in Figure 5C17) is displayed accordingly.

[0233] In Figure 5C18, the intensity of contact 559 on the home button (e.g., button 204) is shallow, which is the pressure threshold IT. LWhen the input is increased to a certain level, device 100 provides a tactile output 569 (e.g., mini-tap 230Hz) corresponding to the selected tactile output setting (e.g., option 1). In response to input on the home button (e.g., button 204), the user interface 510 (e.g., the primary page of a multi-page home screen) is displayed (Figure 5C19). Note that since Figures 5A1-5A18 and 5B1-5B75 show the tactile output setting on option 2, the tactile outputs in these figures correspond to the selected tactile output setting of option 2 (e.g., mini-tap 270Hz). If the user selects option 1 for the tactile output setting (e.g., as shown in Figure 5C16), the tactile outputs in Figures 5A1-5A18 and 5B1-5B75 correspond to the selected tactile output setting of option 1 (e.g., mini-tap 230Hz). Similarly, if the user selects option 3 for the tactile output setting, the tactile outputs in Figures 5A1-5A18 and 5B1-5B75 will correspond to the selected tactile output setting for option 3 (e.g., mini-tap 300Hz).

[0234] Figures 6A1 to 6B26 show exemplary user interfaces for controlling the generation of user interface haptic feedback and home button haptic feedback according to some embodiments. The user interfaces in those figures are used to illustrate the processes described below, including the processes in Figures 13A to 13D. Some of the following embodiments are described with reference to input on a touchscreen display (combining a touch-sensing surface and a display), but in some embodiments, the device detects input on a touch-sensing surface 451 separate from the display 450, as shown in Figure 4B.

[0235] In some embodiments, the device provides tactile output control settings (e.g., sound and tactile settings) that allow the user to turn on and off user interface tactile output generation on the device.

[0236] Generally, devices generate various types of user interface tactile outputs in response to direct interactions with user interface elements, such as selection, manipulation, drag / drop, and / or activation of user interface elements via a focus selector (e.g., a pointer or touch) positioned close to the user interface element when user input is detected. User interface tactile outputs generally involve visual changes in the user interface. For example, in some embodiments, a device provides a tactile output (e.g., a microtap (150Hz)) in response to input picking up an item in the user interface (e.g., a long press input by sustaining contact at the location corresponding to the item), the user interface indicates the object lifting towards the surface of the display, and the tactile output is timed to coincide with the end of the object's movement. In addition to user interface tactile outputs, devices also generate tactile outputs in response to the activation of persistent buttons on the device (e.g., a virtual home button or non-mechanical home or back buttons). Button activation may or may not involve a corresponding visual change in the user interface. For example, in response to a press on the home button, the device generates a tactile output, discards the currently displayed user interface, and displays the home screen. In response to another press on the home button, the device generates a tactile output for the button press, but continues to display the home screen (for example, if the currently displayed home screen is the sole home screen or the primary page of a multi-page home screen). Therefore, in some contexts, it allows the user to turn off user interface tactile output, for example, to save power or reduce distrust. However, if the user simply wants to turn off user interface tactile output, this prevents the user from accidentally turning off tactile output generation for persistent buttons.

[0237] In some embodiments, the device provides non-visual feedback that includes both audio output components and tactile output components. Certain types of audio output may be paired with or independent of visual changes in the user interface. For example, voice alarms, ringtones, and music clips may be played independently of tactile output and function to alert the user to some change in the user interface or the state of the device. In some embodiments, the device generates specific audio output that is specifically linked to and enhances or complements the tactile output. These types of audio output are referred to here as “tactile audio output.” In some embodiments, the non-visual feedback profile includes tactile output patterns for tactile output and audio output patterns for tactile audio output that evoke certain tactile sensations of the user in conjunction with the tactile output. The interaction between the frequency, amplitude, waveform, and / or timing of the tactile audio output and the corresponding tactile output generates a richer and more subtle tactile sensation for the user, making the non-visual feedback more noticeable to the user. In some situations, the device allows the user to control the generation of haptic audio output (e.g., haptic audio output corresponding to user interface haptic output, and / or haptic audio output corresponding to device / system haptic output (e.g., home button haptic output)) using general volume control and / or mute control, for example, to save power and / or reduce distrust. However, the device also prevents the user from accidentally turning off haptic audio for device / system haptic output (e.g., audio accompanying haptic output for persistent button activation).

[0238] Figures 6A1–6A26 illustrate the generation of tactile outputs when user interface tactile outputs are enabled on the device. Figures 6A1–6A26 show that when user interface tactile outputs are enabled on the device, the device generates both user interface tactile outputs and system tactile outputs (e.g., tactile outputs for activating hardware or persistent buttons on the device). Figures 6B1–6B26 show that when user interface tactile outputs are disabled on the device, the device generates system tactile outputs (e.g., tactile outputs for activating hardware or persistent buttons on the device) but ceases generating at least a portion of the user interface tactile outputs.

[0239] As shown in Figure 6A1, the control user interface (e.g., the sound and haptic control user interface 680) includes several toggle settings for controlling sound and haptic output in the device. For example, the vibration of the ring setting 670 is set to "OFF" via toggle control 671. Vibration in the silent setting 672 is set to "OFF" via toggle control 673, and the user interface haptic output setting 674 is set to "ON" via toggle control 675. The sound and haptic control interface 680 further includes a volume controller 676 for setting the current volume for the ringer and alarm sound output (e.g., by moving the volume indicator 677 along the volume controller 676). Furthermore, changes made by the button setting 678 are set to "OFF" via toggle control 679.

[0240] While the user interface tactile output is turned on in device 100 via the toggle control 675, user interface tactile output and system tactile output are generated. In Figures 6A2-6A4, input by contact (e.g., contact 604) is detected on the touchscreen 112 at a location corresponding to an application launch icon (e.g., icon 606 for launching the Mail application) on the home screen user interface (e.g., home screen 602). As shown in Figures 6A3-6A4, the characteristic intensity of contact 604 is shallow, below the pressure intensity threshold IT. L When the intensity increases above this threshold, the home screen 602 (excluding the icons 606) becomes blurred (as shown by the intensity meter 610 in Figure 6A4), and the menu (e.g., the quick action menu 608) is presented on the blurred home screen 602 (as shown in Figure 6A4). In addition, as shown in Figure 6A4, the device 100 has a shallow pressure intensity threshold IT of the characteristic intensity of the contact 604. L Upon detecting an increase exceeding a certain threshold, the system presents a quick action menu 608 and generates a tactile output 612 (e.g., a microtap with a gain of 1.0 (200Hz)).

[0241] Figures 6A5-6A8 show that while contact 604 is maintained on the touchscreen 112, the device detects the movement of contact 604 from the location corresponding to the application launch icon 606 to the locations corresponding to the respective menu options 614, 616, 618, and 620. When contact 604 moves to the location corresponding to each menu option 614, 616, 618, and 620, device 100 generates the respective tactile output (e.g., tactile outputs 622, 624, 628, and 630) (e.g., microtaps (270Hz) with a gain of 0.4 and a minimum interval of 0.05 seconds) to indicate that contact 604 has moved to a different menu option.

[0242] Figures 6A8-6A9 show that when contact 604 exceeds menu option 620, the lift-off of contact 604 is detected. In response to the detection of the lift-off of contact 604, device 100 launches the Mail application and displays the user interface of the Mail application corresponding to menu option 620 (e.g., the email list user interface 622), as shown in Figure 6A10.

[0243] In Figures 6A11-6A13, input is detected by another touch (e.g., touch 624) on the touchscreen 112 at the location corresponding to an email item (e.g., item 626) in the list of email items. As shown in Figures 6A11-6A13, the characteristic intensity of touch 624 is shallow, which is the pressure threshold IT. L When the intensity increases above (as shown by the intensity meter 610 in Figure 6A13), the email list user interface 622 (except for item 626) becomes blurred, and a preview of item 626 (e.g., preview 628) is presented on the blurred email list user interface 622 (as shown in Figure 6A13). In addition, as shown in Figure 6A13, device 100 has a shallow pressure intensity threshold IT of the characteristic intensity of contact 624. L In response to detecting an increase exceeding the given value, a review 628 is presented and a tactile output 630 (e.g., a microtap with a gain of 1.0 (200Hz)) is generated.

[0244] Figures 6A13–6A16 show that while contact 624 is maintained on the touchscreen 112, the device detects a movement to the left from the point of contact 624. The movement of contact 624 to the left causes preview 628 to be dragged toward the left side of the touchscreen 112. While preview 628 is dragged toward the left side of the touchscreen 112, a hidden menu option 632 (e.g., "Archive") is gradually revealed from behind preview 628. In Figure 6A16, when contact 624 moves across a threshold position in the user interface for triggering an operation related to the hidden menu option 632 (e.g., archive email items and remove them from the email list) (e.g., a threshold position hidden to the left of the centerline of the touchscreen 112), device 100 changes the color of the menu option 632 to indicate that the threshold for triggering the archive operation has been met by the movement of preview 628. Furthermore, device 100 generates a tactile output 634 (e.g., a microtap (270Hz) with a gain of 1.0) along with a visual change in the user interface to indicate that a threshold for triggering an archive operation has been met by the movement of preview 628.

[0245] Figures 6A17–6A19 show that after contact 624 crosses a threshold position (or drags over preview 628) to trigger an archive operation, the lift-off of contact 624 is detected. In response to the detection of the lift-off of contact 624, device 100 performs an archive operation on the email corresponding to item 626, and item 626 is removed from the list of email items as shown in Figure 6A19.

[0246] In Figures 6A20-6A22, input is detected by another touch (e.g., touch 636) on the touchscreen 112 at a location corresponding to another email item (e.g., item 638) in the list of email items. As shown in Figures 6A20-6A22, the characteristic intensity of touch 636 is shallow, which is the pressure threshold IT. LWhen it increases above (as shown by the intensity meter 610 in FIG. 6A22), the email list user interface 622 (excluding item 638) blurs, and a preview of item 638 (e.g., preview 640) is presented on the blurred email list user interface 622 (as shown in FIG. 6A22). Additionally, as shown in FIG. 6A22, when the device 100 detects an increase in the characteristic intensity of contact 636 above the shallow pressing intensity threshold IT of the contact 636 L above, it presents the preview 640 and generates a haptic output 642 (e.g., a micro tap (200 Hz) with a gain of 1.0).

[0247] FIG. 6A23 shows that while contact 636 is maintained on the touch screen 112 (e.g., on preview 640), an increase in the characteristic intensity of contact 636 above the deep pressing intensity threshold IT D is detected. In response to detecting an increase in the characteristic intensity of contact 636 above the deep pressing intensity threshold, the device 100 stops displaying the preview 640 and, instead of the blurred email list user interface, displays a content display user interface 644 (e.g., a user interface that displays the content of email item 638). Further, the device 100 generates a haptic output 646 (e.g., a full tap (150 Hz), gain: 1.0) along with the display of the user interface 644.

[0248] In FIG. 6A24, a lift-off of contact 636 is detected, and after the lift-off of contact 636 is detected, the user interface 644 is maintained.

[0249] FIG. 6A25 shows that while the user interface 644 is displayed on the touch screen 112, a pressing input by contact (e.g., contact 646) is detected on the home button 650 of the device 100. The device 100 detects a pressing input (e.g., a shallow pressing intensity threshold IT of the contact 636 LIn response to the detection of an increase in the characteristic intensity of contact 646 exceeding a certain threshold, a tactile output 648 (e.g., a mini-tap with a gain of 1.0 (230Hz)) is generated to indicate that the home button 650 has been activated by the press input. Figure 6A26 shows that in response to the activation of the home button 650 by the press input, the user interface 644 is discarded and the home screen 602 is displayed on the touchscreen 112 in place of the user interface 644.

[0250] Figures 6B1 to 6B26 illustrate the generation of tactile outputs when user interface tactile outputs are turned off on a device. Specifically, Figures 6B1 to 6B26 show that when user interface tactile outputs are turned off on a device, the device continues to generate system tactile outputs (e.g., tactile outputs for activating hardware or persistent buttons on the device), but stops generating user interface tactile outputs unless the generation of tactile outputs for a specific user interface is excluded from control by the user's tactile output settings.

[0251] As shown in Figure 6B1, the user interface tactile output setting 674 is set to "off" via the toggle control 675 in the voice and tactile control interface 680.

[0252] While the user interface tactile output is turned off in device 100 via the toggle control 675, at least some user interface tactile output is no longer generated. In Figures 6B2-6B4, input by touch (e.g., touch 605) is detected on the touchscreen 112 at the location corresponding to the icon 606 for launching the Mail application on the home screen 602. As shown in Figures 6B3-6B4, the characteristic intensity of the touch 605 is shallow, which is below the pressure threshold IT. LWhen the intensity increases above (as shown by the intensity meter 610 in Figure 6B4), the home screen 602 (excluding the icons 606) becomes blurred, and the quick action menu 608 is presented on the blurred home screen 602 (as shown in Figure 6B4). In addition, as shown in Figure 6B4, device 100 has a shallow pressure intensity threshold IT of the characteristic intensity of contact 605. L In response to detecting an increase exceeding a certain threshold, the quick action menu 608 is presented, and a tactile output 613 (e.g., a microtap with a gain of 1.0 (200Hz)) is generated. In this example, even though the tactile output 613 is a user interface tactile output, the tactile output for presenting the quick action menu in response to a press input is exempt from the control of the user interface tactile output settings. In other words, in some embodiments, the tactile output (e.g., a microtap with a gain of 1.0 (200Hz)) accompanying the presentation of the quick action menu in response to a press input on the application launch icon is always generated regardless of the current tactile output settings for the user interface tactile output.

[0253] Figures 6B5–6B8 show that while contact 605 is maintained on the touchscreen 112, the device detects the movement of contact 605 from the location corresponding to the application launch icon 606 to the locations corresponding to menu options 614, 616, 618, and 620, respectively. When contact 605 moves to the location corresponding to each menu option 614, 616, 618, and 620, the device 100 highlights that menu option to indicate that contact 605 has moved to a different menu option (in contrast to the scenario shown in Figures 6A5–6A8, for example, where tactile outputs 622, 624, 628, and 630 are generated to indicate that contact 604 has moved to a different menu option), but does not generate any tactile output accompanying the visual change occurring within the user interface. In other words, in some embodiments, when the user interface tactile output is turned off, the device 100 stops generating tactile output for a portion of the input (e.g., movement across individual menu options by touch 605), but continues to generate tactile output for other portions of the input (e.g., increasing the intensity of touch 605 above a shallow pressure threshold that displays the quick action menu 608), regardless of whether the user interface tactile output is turned on or off.

[0254] Figures 6B8-6B9 show that when contact 605 exceeds menu option 620, the lift-off of contact 605 is detected. In response to the detection of the lift-off of contact 605, device 100 launches the Mail application and displays the email list user interface 622 corresponding to menu option 620, as shown in Figure 6B10.

[0255] In Figures 6B11-6B13, input is detected from another touch (e.g., touch 625) on the touchscreen 112 at the location corresponding to item 626 in the list of email items. As shown in Figures 6B11-6B13, the characteristic intensity of touch 625 is shallow, which corresponds to the pressure threshold IT. LWhen the intensity increases above (as shown by the intensity meter 610 in Figure 6B13), the email list user interface 622 (except for item 626) becomes blurred, and the preview 628 is presented on the blurred email list user interface 622 (as shown in Figure 6B13). In addition, as shown in Figure 6B13, device 100 has a shallow pressure intensity threshold IT of the characteristic intensity of contact 625. L In response to detecting an increase exceeding a certain threshold, a preview 628 is presented, and a tactile output 631 (e.g., a microtap with a gain of 1.0 (200Hz)) is generated. In other words, in some embodiments, the tactile output (e.g., a microtap with a gain of 1.0 (200Hz)) accompanying the presentation of the preview in response to a press input on an item is always generated, regardless of the current tactile output setting for the user interface tactile output.

[0256] Figures 6B13–6B16 show that while contact 625 is maintained on the touchscreen 112, the device detects a movement to the left from the point of contact 625. The movement of contact 625 to the left causes preview 628 to be dragged toward the left side of the touchscreen 112. While preview 628 is dragged toward the left side of the touchscreen 112, a hidden menu option 632 (e.g., "Archive") is gradually revealed from behind preview 628. In Figure 6B16, when contact 625 moves across a threshold position in the user interface for triggering operations related to the hidden menu option 632 (e.g., archive email items, archive from email list) (e.g., a threshold position hidden to the left of the centerline of the touchscreen 112), device 100 changes the color of the menu option 632 to indicate that the threshold for triggering the archive operation has been met by the movement of preview 628. However, in contrast to the scenario shown in Figure 6A16, device 100 does not generate any tactile output along with any visual changes in the user interface. In other words, device 100 has stopped generating tactile output in the user interface, according to the current tactile output setting, which is in the "off" state.

[0257] Figures 6B17–6B19 show that after contact 625 crosses a threshold position (or drags over preview 628) to trigger an archive operation, the lift-off of contact 625 is detected. In response to the detection of the lift-off of contact 625, device 100 performs an archive operation on the email corresponding to item 626, and item 626 is removed from the list of email items as shown in Figure 6B19.

[0258] In Figures 6B20-6B22, input is detected by another touch (e.g., touch 637) on the touchscreen 112 at the location corresponding to item 638 in the list of email items. As shown in Figures 6B20-6B22, the characteristic intensity of touch 637 is shallow, below the pressure threshold IT. LWhen it increases beyond (as shown by the intensity meter 610 in FIG. 6B22), the email list user interface 622 (excluding item 638) becomes blurred, and the preview 640 is presented on the blurred email list user interface 622 (as shown in FIG. 6B22). Further, as shown in FIG. 6B22, the device 100 has a shallow pressing intensity threshold IT of the characteristic intensity of the contact 637 L In response to detecting an increase beyond, the device presents the preview 640 and generates a haptic output 643 (e.g., a micro tap (200 Hz) with a gain of 1.0) because the haptic output 643 is exempted from the control by the current haptic output setting for turning off the haptic output of the user interface

[0259] FIG. 6B23 shows that the contact 637 detects an increase in the characteristic intensity of the contact 637 on the touch screen 112 (e.g., while maintained on the preview 640) beyond a deep pressing intensity threshold IT D In response to detecting an increase in the characteristic intensity of the contact 637 beyond the deep pressing intensity threshold, the device 100 stops displaying the preview 640 and displays a content display user interface 644 instead of the blurred email list user interface. Further, since the haptic output 647 is exempted from the control by the current haptic output setting for turning off the haptic output of the user interface, the device 100 generates a haptic output 647 (e.g., a full tap (150 Hz) with a gain of 1.0) along with the display of the user interface 644

[0260] In FIG. 6B24, the lift-off of the contact 637 is detected, and after detecting the lift-off of the contact 637, the user interface 644 is maintained

[0261] FIG. 6B25 shows that while the user interface 644 is displayed on the touch screen 112, a pressing input by a contact (e.g., contact 647) is detected on the home button 650 of the device 100. The device 100 has a shallow pressing intensity threshold IT for the pressing input (e.g., LIn response to the detection of an increase in the characteristic intensity of contact 647 exceeding a certain threshold, a tactile output 649 (e.g., a mini-tap with a gain of 1.0 (230Hz)) is generated to indicate that the home button 650 has been activated by a press input. Figure 6B26 shows that in response to the activation of the home button 650 by a press input, the user interface 644 is discarded and the home screen 602 is displayed on the touchscreen 112 in place of the user interface 644.

[0262] Although not shown in Figures 6A1 to 6B26, haptic audio output may be associated with some or all of the haptic output generated in the above examples, depending on the specific prioritization of different considerations in various usage scenarios and power-saving configurations of the device. For example, in some embodiments, the haptic audio output is controlled by general device audio volume control and / or mute control.

[0263] In some embodiments, when a tactile output is generated, a tactile audio output is always generated, regardless of whether the device is muted or not.

[0264] In some embodiments, when a system tactile output is generated, a tactile audio output is always generated, regardless of whether the device is muted or not.

[0265] In some embodiments, when a user interface tactile output is generated, a tactile audio output is always generated, regardless of whether the device is muted or not.

[0266] In some embodiments, when tactile output is generated, tactile audio output is muted when the device is muted, and tactile audio output is generated when the device is not muted.

[0267] In some embodiments, when a system tactile output is generated, the tactile audio output associated with the system tactile output is muted when the device is muted, and the tactile audio output associated with the system tactile output is generated when the device is not muted.

[0268] In some embodiments, when a user interface tactile output is generated, the haptic audio output associated with the user interface tactile output is muted when the device is muted, and the haptic audio output associated with the user interface tactile output is generated when the device is not muted.

[0269] In some embodiments, when a device generates tactile sound with a tactile output, the device further generates tactile sound output according to the characteristics of the generated tactile output. For example, the amplitude of the sound output increases as the amplitude of the corresponding tactile output increases. For example, the frequency of the sound output increases as the frequency of the corresponding tactile output increases. In some embodiments, the amount of change in the characteristics of the sound output (e.g., amplitude or frequency) is greater than the amount of change in the characteristics of the corresponding tactile output.

[0270] In some embodiments, if the device generates haptic audio with a tactile output (e.g., a user interface tactile output controlled by a user interface tactile output setting), the device generates haptic audio output with an amplitude adjusted based on the device's volume setting. In some embodiments, if the device generates haptic audio with a tactile output (e.g., a user interface tactile output exempt from control by a user interface tactile output setting), the device generates haptic audio output with an amplitude that is not adjusted based on the device's volume setting.

[0271] In some embodiments, the device continues to generate tactile audio output when user interface tactile output is turned off, regardless of the device's mute / unmute setting. For example, for tactile output that is not exempt from control by the user interface tactile output setting, when no tactile output is generated, the device may generate an audio output instead to provide non-visual feedback to the user.

[0272] In some embodiments, the device simply turns off the haptic audio output (e.g., the haptic audio output for the user interface haptic output) according to the device's mute setting when the user interface haptic output is turned on.

[0273] Other variations of tactile output control and tactile output control are also possible and are not exhaustively listed herein.

[0274] Figures 7A-7C are flowcharts illustrating a method 700 for providing visual feedback regarding the activation of a user input device, according to some embodiments. The method 700 is performed in an electronic device (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A) having a display, a touch-sensitive surface, one or more sensors for detecting the intensity of contact with the touch-sensitive surface and / or each of the device's buttons (e.g., virtual or physical home buttons), and one or more tactile output generators for generating tactile output. In some embodiments, the display is a touchscreen display, and the touch-sensitive surface is on or incorporated into the display. In some embodiments, the display is separate from the touch-sensitive surface. In some embodiments, the method 700 is stored in a non-temporary computer-readable storage medium (e.g., a non-volatile computer-readable storage medium) and controlled by instructions executed by one or more processors of the electronic device, such as one or more processors 122 of device 100 (Figure 1A). For simplicity of explanation, the method 700 is described below as being performed by device 100. Some operations of method 700 are combined by choice, and / or the order of some operations is changed by choice.

[0275] Method 700 relates to displaying the start of visual feedback regarding the activation of a user input device (e.g., a virtual or physical home button) in response to the detection of a preliminary input (e.g., a light press) that matches an input that activates the user input device (e.g., a press input). In some embodiments, when an input to activate the user input device is detected, the user interface changes corresponding to the activation of the user input device are presented as a continuation of the already presented visual feedback. If no input to activate the user input device is detected before the end of the preliminary input, the visual feedback is not displayed and the original state of the user interface is restored. Displaying the start of visual feedback regarding the activation of a user input device before the actual activation of the user input device makes it possible to provide the user with information about the effect of the input (e.g., activating the home button to return to the home screen) and give the user an opportunity to make a decision on whether to proceed with completing the input based on that information. In this way, visual feedback improves the usability of the device by assisting the user in providing appropriate inputs during device operation / interaction, reducing user errors, and in addition, reduces the power consumption of the device and improves battery life by enabling the user to use the device more quickly and efficiently.

[0276] In some embodiments, the device's home button is a physical home button or a virtual home button (e.g., button 204, Figure 5A1). In some embodiments, the device includes one or more sensors for detecting the intensity of contact with a touch-sensitive surface. In some embodiments, the display is a touch-sensitive display. In some embodiments, the device includes one or more tactile output generators for generating tactile output. In some embodiments, in response to detecting a first type of input on the home button (a press input or press or release input detected via a press of a mechanical switch, or by comparing the intensity of contact on the home button to an activation criterion based on one or more intensity thresholds, as described in more detail with reference to methods 900, 1300, 1500 and 1700), the home button (e.g., permanently displayed in its respective location or located in a permanent location on the device away from the display) is available on the device in several different contexts to discard the currently displayed user interface (e.g., optionally, to redisplay a previously displayed user interface, such as a previous view of an application or system user interface, such as the device's home screen, multitasking user interface, or virtual assistant user interface). Other embodiments of haptic feedback for activating the home button will be described in more detail with reference to methods 1100 and 1900.

[0277] In some embodiments, the home button responds to multiple types of input, including a first type of input (e.g., a single click (e.g., one press input, or one press input followed by a release event)), a second type of input (e.g., a double-click (e.g., a first press event followed by a second press event, or a first press event followed by a second release event, where the first and second press events are detected within a mutual time threshold)), a third type of input (e.g., a long press (e.g., having a first press event that does not lift off until a time exceeding a threshold time amount has elapsed)), and a fourth type of input (e.g., a deep press (e.g., having a first press event that includes an increase in contact intensity exceeding a deep press intensity threshold within a threshold time amount after the initial touchdown of contact)). In some embodiments, each type of input is associated with discarding the currently displayed user interface, performing a corresponding operation, and optionally displaying the corresponding user interface associated with that operation.

[0278] In Method 700, and with reference to Figure 7A, the device (e.g., device 100 in Figure 5A1) displays a first user interface (e.g., a home screen, or the primary page of a multi-home screen, such as user interface 510 in Figure 5A1) on its display (702). While the device is displaying the first user interface (e.g., user interface 510 in Figure 5A1), it detects an input instructed by the first user interface (e.g., a tap gesture by contact 502 on the Clock icon in Figure 5A2) (704). For example, an input instructed by the first user interface is a touch input on a touch-sensitive surface at a location corresponding to an activatable object within or across the first user interface (not on the home button). As another example, input directed to the first user interface could be an application launch icon, a folder icon, a notification, a menu option, a control affordance (e.g., for displaying a control user interface), or a touch input directed to the user interface as a whole (e.g., a swipe input on the home screen, or the primary page of a multi-page home screen).

[0279] In response to detecting input instructed to the first user interface, the device stops displaying the first user interface (for example, user interface 510 in Figure 5A2) and displays a second user interface separate from the first user interface (for example, user interface 512 of the Clock application in Figure 5A3) (for example, the user interface of the first application, an enlarged folder overlaid on a darkened home screen, an enlarged notification overlaid on a darkened home screen, a control panel overlaid on a darkened home screen, a multitasking user interface that simultaneously presents each representation of a multi-user interface, a secondary page of a multi-page home screen, a notification screen, a widget screen, etc.) (706).

[0280] While displaying the second user interface, the device detects a contact on the home button (e.g., contact 504-a shown in Figure 5A4) (708). While continuously detecting contacts on the home button, the device performs at least three operations (710). Firstly, a first intensity threshold (e.g., hint intensity threshold IT) (e.g., as shown in Figure 5A5). H Secondly, in response to detecting an increase in the characteristic intensity of the contact relative to the first intensity threshold (for example, as shown in Figures 5A5-5A6 and 5A9-5A10), the device indicates the start of a transition back from the second user interface to the first user interface. Thirdly, while indicating the start of the transition back from the second user interface to the first user interface, the device indicates the first intensity threshold (for example, the hint intensity threshold IT). H The system detects an input sequence (e.g., input sequences 504-a to 504-e by contact 504 in Figures 5A4 to 5A7, or input sequences 508-a to 508-e by contact 508 in Figures 5A9 to 5A13) which includes detecting a pressing input that includes an increase in the characteristic intensity of the contact that exceeds a certain threshold (e.g., the intensity of the contact continues to increase after reaching a first intensity threshold and is displayed after the start of the transition from the second user interface to the first user interface).

[0281] Referring here to Figure 7B, upon detecting an input sequence, the device determines whether the input sequence satisfies a first criterion and / or whether the input sequence satisfies a second criterion (712). The input sequence satisfies the first criterion (for example, the second intensity threshold being a shallow pressure intensity threshold IT) before the end of the press input is detected (for example, as shown in Figure 5A11). LIn accordance with the determination that the first criterion is met, which requires that the characteristic intensity of the contact increases above the second intensity threshold (for example, detecting the end of a press input includes detecting a release input, such as the lift-off of contact from the contact sensing surface or a decrease in the characteristic intensity of contact below the release intensity threshold (for example, as shown in Figure 5A13) (for example, the release intensity threshold may be pre-configured to be above, below, or equal to a shallow press intensity threshold, or may be dynamically determined based on the input metric of the press input (e.g., the rate of increase in contact intensity)), the device stops displaying the second user interface and redisplays the first user interface on the display (for example, at the end of a transition from the second user interface to the first user interface, the first user interface is redisplayed and the second user interface is no longer displayed). For example, in Figures 5A8 to 5A14, the input sequence satisfies the first criterion, and device 100 stops displaying user interface 512 at the end of the transition from user interface 510 back to user interface 510 (Figure 5A8), and redisplays user interface 510 (e.g., user interfaces 513-1 to 513-5, Figures 5A9 to 5A13) (Figure 5A14).

[0282] In accordance with the determination that the input sequence satisfies a second criterion, which requires that the characteristic intensity of the contact remains below a second intensity threshold before the end of the press input is detected, the device reverses the start of the transition from the second user interface back to the first user interface and redisplays the second user interface on the display. For example, in Figures 5A4-5A8, the input sequence satisfies the second criterion, and device 100 reverses the start of the transition (e.g., from user interface 513-2 to user interface 513-1, Figures 5A6-5A7) and redisplays the second user interface (e.g., user interface 512, Figure 5A8) on the display.

[0283] In some embodiments, stopping the display of the second user interface and redisplaying the first user interface includes displaying the continuation of the start of the transition back from the second user interface to the first user interface (as shown, for example, in Figures 5A9-5A13) (714). In some embodiments, by displaying the continuation of visual feedback displayed in response to audible input, the complete transition from the second user interface to the first user interface is performed seamlessly in accordance with user expectations without confusing the user with unexpected visual changes, making the user interface more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and reducing power consumption of the device and improving battery life by allowing the user to use the device more quickly and efficiently. Furthermore, the transition from the second user interface to the first user interface is not unnecessarily delayed by the display of visual feedback in response to its audible input.

[0284] In some embodiments, indicating the start of a transition back from a second user interface to a first user interface includes displaying at least a portion of the second user interface simultaneously with at least a portion of the first user interface (716). For example, in Figure 5A9, user interface 513-1 includes a portion of user interface 512 and a portion of user interface 510. In some embodiments, as the transition progresses, more of user interface 510 is displayed and less of user interface 512 is displayed (for example, as shown in Figures 5A10-5A13).

[0285] In some embodiments, the first user interface is a home screen (e.g., user interface 510, Figure 5A1) that includes multiple application launch icons corresponding to different applications on the electronic device (718).

[0286] Referring here to Figure 7C, in some embodiments, the first and second user interfaces are sequentially displayed user interfaces for an application (720). For example, the home button functions as a back button in some contexts.

[0287] In some embodiments, indicating the start of a transition back from the second user interface to the first user interface includes reducing the size of the second user interface (for example, as the contact intensity increases) (722). For example, as shown in Figures 5A5-5A6, the display size of the second user interface 512 decreases as the characteristic intensity of the contact 504 increases. In some embodiments, the device reveals the first user interface from behind the second user interface while reducing the size of the second user interface (for example, as the contact intensity increases) (724). For example, the second user interface is displayed at full screen size (for example, as shown in Figure 5A4) before the contact reaches a first intensity threshold, and as the contact intensity increases to exceed the first intensity threshold, the second user interface shrinks (for example, as shown in Figures 5A5-5A6) to reveal the background home screen.

[0288] In some embodiments, the second user interface is the user interface of the first application (e.g., the user interface 512 of the Clock application in Figure 5A8), the first application corresponds to the first application launch icon in the first user interface (e.g., the Clock icon on the user interface 510 in Figure 5A14), and reducing the size of the second user interface includes shrinking the second user interface toward the position of the first application launch icon on the first user interface (e.g., as shown in Figures 5A9-5A13) (726).

[0289] In some embodiments, the second user interface includes an enlarged folder (e.g., user interface 522 in Figure 5B70) (overlaid on, for example, a blurred enlarged version of the first user interface), the enlarged folder corresponding to a first folder icon of the first user interface (e.g., the photo folder icon of user interface 520 in Figure ...

Claims

1. It is a method, An electronic device comprising a display, a button disposed outside the display, and one or more sensors for detecting the intensity of contact between the electronic device and the button, While the first user interface is displayed, a touch on the button is detected, While continuously detecting the contact on the button, To detect an increase in the characteristic intensity of the contact that exceeds a first intensity threshold, In response to detecting the increase in the characteristic intensity of the contact relative to the first intensity threshold, the system displays the start of a transition to the second user interface. While displaying the start of the transition to the second user interface, an input sequence is detected which includes detecting a pressing input that includes an additional increase in the characteristic intensity of the contact, In response to detecting the aforementioned input sequence, The input sequence is determined to satisfy a first criterion, which requires that the characteristic intensity of the contact increases above a second intensity threshold before the end of the pressing input is detected. Displaying the second user interface on the aforementioned display, The input sequence satisfies a second criterion, which requires that the characteristic intensity of the contact remains below the second intensity threshold before the end of the pressing input is detected. Reversing the start of the transition to the second user interface, A method that includes this.

2. The method according to claim 1, wherein displaying the start of the transition to the second user interface includes displaying at least a portion of the second user interface together with at least a portion of the first user interface.

3. The method according to claim 1 or 2, wherein the first user interface and the second user interface are user interfaces for a continuously displayed application.

4. The method according to any one of claims 1 to 3, wherein indicating the start of the transition to the second user interface includes fading the first user interface.

5. Displaying the start of the transition to the second user interface is: The method according to claim 4, comprising shifting at least a portion of the second user interface in a first direction so as to cover at least a portion of the first user interface while fading out the first user interface.

6. The method according to claim 4 or 5, wherein displaying the start of the transition to the second user interface includes changing the virtual depth of the first user interface.

7. Detecting the input sequence includes detecting that the characteristic intensity of the contact changes between the first intensity threshold and the second intensity threshold before the first criterion is satisfied by the input sequence. The aforementioned method, The method according to any one of claims 1 to 6, comprising dynamically advancing and reversing the transition to the second user interface according to the characteristic intensity of the contact, in response to detecting that the characteristic intensity of the contact changes between a first intensity threshold and a second intensity threshold before the first criterion is met by the input sequence.

8. The transition to the second user interface is to be dynamically advanced and reversed according to the characteristic strength of the contact. The method according to claim 7, comprising dynamically revealing at least a portion of the second user interface according to the characteristic intensity of the contact.

9. The transition to the second user interface is to be dynamically advanced and reversed according to the characteristic strength of the contact, The method according to claim 7 or 8, comprising dynamically changing the amount of the second user interface revealed according to the characteristic intensity of the contact.

10. The first user interface described above is the user interface for the first application, The aforementioned method, In response to detecting the aforementioned input sequence, The method according to claim 1, comprising terminating or suspending the first application in accordance with the determination that the input sequence satisfies the first criterion.

11. The electronic device includes one or more tactile output generators, The aforementioned method, In response to detecting the aforementioned input sequence, The method according to any one of claims 1 to 10, comprising generating a first tactile output in accordance with the determination that the input sequence satisfies the first criterion.

12. The method according to any one of claims 1 to 11, comprising generating a tactile output when detecting the increase in the characteristic intensity of the contact that exceeds the first intensity threshold.

13. In response to detecting the input sequence, The method according to any one of claims 1 to 12, comprising interrupting the transition to the second user interface and displaying a transition to a third user interface related to the third criterion, in accordance with the determination that the input sequence includes an increase in the characteristic intensity of the contact exceeding the second intensity threshold and the satisfaction of a third criterion.

14. It is an electronic device, The display and Buttons located on the outside of the aforementioned display, One or more sensors for detecting the intensity of contact between the electronic device and the button, One or more processors, It comprises memory for storing instructions, An electronic device wherein, when the instruction is executed by the one or more processors, the electronic device causes the electronic device to perform the method according to any one of claims 1 to 13.

15. A computer program including instructions, the instructions, when executed on an electronic device, which includes a display, buttons located outside the display, and one or more sensors for detecting the intensity of contact between the electronic device and the buttons, the computer program causes the electronic device to perform the method according to any one of claims 1 to 13.

Images