System and method for activating and using touchpad at electronic device with touch-sensitive display and without force sensor

Abstract

The invention relates to a system and method for activating and using a touchpad at an electronic device having a touch-sensitive display and not having a force sensor. The invention discloses a single finger gesture for activating a touchpad mode on a touch sensitive device without a force sensor. An exemplary method includes displaying a virtual keyboard having a space key and other keys on a touch-sensitive display. A first extended contact over the virtual keyboard is detected, and in accordance with a determination that the first extended contact is over any respective displayed representation of the other keys, the first extended contact is interpreted as a keyboard input and a corresponding extended keyboard function is activated. The method further includes detecting a second extended contact over the virtual keyboard. The method further includes interpreting the second extended contact as a mode switching input that causes the electronic device to operate at least a portion of the touch-sensitive display as a touchpad in accordance with a determination that the second extended contact is above the corresponding display representation of the space key.

Description

[0001] This application is a divisional application of a Chinese invention patent application with an application date of September 28, 2018, application number 201811143385.1, and invention name “System and method for activating and using a touch pad at an electronic device having a touch-sensitive display and no force sensor”. Technical Field

[0002] Embodiments disclosed herein relate generally to electronic devices having touch-sensitive displays and without force sensors, and more specifically to systems and methods for activating and using a trackpad at an electronic device having a touch-sensitive display and without force sensors. Background Art

[0003] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch-sensitive surfaces include trackpads and touch screen displays. Such surfaces are widely used to view and edit electronic documents by manipulating the cursor in the electronic document presented on the display. These electronic documents are viewed or edited in applications with viewing and editing capabilities (e.g., drawing applications, presentation applications (e.g., Apple's KEYNOTE or Microsoft's POWERPOINT), word processing applications (e.g., Apple's PAGES or Microsoft's WORD), website creation applications, spreadsheet applications (e.g., Apple's NUMBERS or Microsoft's EXCEL)).

[0004] Conventional cursor manipulation methods on touch-sensitive devices require the use of force sensors and / or require the user to provide multi-finger gestures (e.g., gestures that require the user to use two of their fingers, such as two-finger taps). For touch-sensitive devices that do not include force sensors, pressure-sensitive / force-sensitive gestures are not available. In addition, such devices may also have a smaller display screen area (e.g., the displays on these devices are typically smaller), so using multi-finger gestures is not only inefficient, but also blocks too much of the already small display screen area.

[0005] As such, it is desirable to provide a single-finger gesture for activating a trackpad mode for use on an electronic device having a touch-sensitive display and without a force sensor. Summary of the invention

[0006] Therefore, there is a need for a single-finger gesture for activating a trackpad mode for use on an electronic device with a touch-sensitive display and without a force sensor. The single-finger gestures discussed herein may also be implemented on devices that do include a force sensor, but detecting these single-finger gestures does not require the use of any force signals that may be generated by such a force sensor (in other words, even if a device does have a force sensor, such a force sensor is not required to detect the single-finger trackpad activation gestures discussed herein, such as the tip-to-groove scroll gesture and an extended contact over a predetermined mode switch region of a virtual keyboard (examples of a mode switch region include the space bar or the region on the virtual keyboard below the space bar)).

[0007] The embodiments described herein address the above shortcomings by providing devices and methods that allow a user to activate a trackpad mode with a single-finger gesture, wherein the trackpad mode is used on an electronic device having a touch-sensitive display and not having a force sensor. On a device that does not include a force sensor, designing a single-finger gesture that allows switching to a trackpad mode while also avoiding unnecessary confusion with other gestures is a very difficult task.

[0008] An exemplary single-finger gesture described below is an extended contact over the space bar on a virtual keyboard (or over another designated area of ​​the virtual keyboard, such as the bottom portion of the virtual keyboard that does not include keys). Allowing a user to provide an extended contact over the space bar (or over another designated area of ​​the virtual keyboard, such as the bottom portion of the virtual keyboard that does not include keys) and then operate the electronic device as a trackpad provides a convenient way for the user to quickly switch from a keyboard input mode to a trackpad mode using only a single finger. Providing this interaction at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more efficient (e.g., by allowing the user to easily switch to a trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate a trackpad mode without using the trackpad mode, or without having to waste time using a complex sequence of user inputs or menu options to activate the trackpad mode).

[0009] Another exemplary single-finger gesture described below is a tip-to-grain (also known as finger roll) gesture over a virtual keyboard. Allowing a user to roll a single finger over a virtual keyboard and then operate the electronic device as a trackpad provides a convenient way for the user to quickly switch from keyboard input mode to trackpad mode using only a single finger. Providing this interaction at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more efficient (e.g., by allowing the user to easily switch to trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate trackpad mode without using trackpad mode, or without having to waste time using a complex sequence of user input or menu options to activate trackpad mode).

[0010] By providing a more efficient human-computer interface, the devices and methods described herein also reduce power usage (e.g., because the user generally provides less input and spends less time performing tasks such as manipulating cursor position) and thereby extend the battery life of the electronic device, and also use limited computing resources in a more efficient manner.

[0011] (A1) According to some embodiments, a method is performed at an electronic device including a touch-sensitive display coupled to a plurality of touch sensors, wherein the touch-sensitive display is not coupled to any force sensor. The method includes: displaying a virtual keyboard on the touch-sensitive display, the virtual keyboard including display representations of a space bar and other keys. The method also includes: detecting a first extended contact above the virtual keyboard; and based on determining that the first extended contact is above any corresponding display representations of other keys, interpreting the extended contact as a keyboard input and activating the corresponding extended keyboard function. The method also includes: detecting a second extended contact above the virtual keyboard; and based on determining that the second extended contact is above a corresponding display representation of the space bar, interpreting the second extended contact as a mode switching input that causes the electronic device to operate at least a portion of the touch-sensitive display as a touchpad.

[0012] On devices that do not include force sensors, designing a single-finger gesture that allows switching to a trackpad mode while also avoiding unnecessary confusion with other gestures is a very difficult task. Allowing a user to provide an extended contact over the space bar (or over another designated area of ​​a virtual keyboard, such as a bottom portion of a virtual keyboard that does not include keys) and then operate the electronic device as a trackpad provides a convenient way for the user to quickly switch from a keyboard input mode to a trackpad mode. Providing this interaction at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more efficient (e.g., by allowing the user to easily switch to a trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate a trackpad mode when the trackpad mode is not in use, or without having to waste time using a complex sequence of user inputs or menu options to activate a trackpad mode).

[0013] (A2) In some embodiments of the method described in A1, the method further includes: while operating as a touch pad: detecting lift-off of the second extended contact from the touch-sensitive display; and in response to detecting lift-off of the second extended contact, determining whether any movement of the second extended contact on the touch-sensitive display before lift-off satisfies movement parameters.

[0014] Using the movement parameters can allow the electronic device to accurately determine when to continue operating in the trackpad mode or when to exit the trackpad mode. By determining whether any movement of the second extended contact on the touch-sensitive display satisfies the movement parameters in response to detecting a lift-off, the device reserves computing resources and uses these computing resources to make the determination only when the determination is immediately needed. In this way, the operability of these devices is enhanced and the human-machine interface on these devices operates more efficiently (e.g., by avoiding wasting resources by making a determination before the determination is needed).

[0015] (A3) In some embodiments of the method according to A2, the method also includes: determining that the movement of the second extended contact on the touch-sensitive display before lift-off does not satisfy the movement parameters, and in response, stopping operating a portion of the touch-sensitive display as a touch pad.

[0016] Once it is determined that the movement before the lift-off does not satisfy the movement parameters, the portion of the touch-sensitive display is stopped from being operated as a trackpad, which ensures that the trackpad mode is invoked only when the user needs it. By ensuring that the trackpad mode is invoked only when the user needs it, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (e.g., by ensuring that the user is presented with the interface controls they need and avoiding wasting time trying to find the interface controls they need).

[0017] (A4) In some embodiments of the method described in A3, determining that movement of the second extended contact on the touch-sensitive display before lift-off does not satisfy movement parameters includes determining that a distance moved by the second extended contact on the touch-sensitive display before lift-off is less than a predetermined distance.

[0018] Once it is determined that the distance moved by the second extended contact on the touch-sensitive display before lifting is less than a predetermined distance, the portion of the touch-sensitive display is stopped from being operated as a touchpad, which ensures that the touchpad mode is invoked only when the user needs the touchpad mode. If the user moves the second extended contact a very short distance (or not at all), this indicates that the user has placed the cursor at the desired location and no longer needs to use the touchpad mode (or when the contact does not move at all, this indicates that the user accidentally invoked the touchpad mode). By ensuring that the touchpad mode is invoked only when the user needs the touchpad mode, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (for example, by ensuring that the interface controls that the user needs are presented to the user and avoiding wasting time on the user trying to find the interface controls that the user needs).

[0019] (A5) In some embodiments of the method described in A3, determining that movement of the second extended contact on the touch-sensitive display before lift-off does not satisfy movement parameters includes determining that a speed at which the second extended contact moves on the touch-sensitive display before lift-off is less than a predetermined speed.

[0020] Once it is determined that the speed at which the second extended contact moves on the touch-sensitive display before lifting is less than a predetermined speed, the portion of the touch-sensitive display is stopped from being operated as a touchpad, which ensures that the touchpad mode is invoked only when the user needs the touchpad mode. If the user moves the second extended contact at a slow speed, this indicates that the user has placed the cursor at the desired position and no longer needs to use the touchpad mode. By ensuring that the touchpad mode is invoked only when the user needs the touchpad mode, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (for example, by ensuring that the interface controls that the user needs are presented to the user and avoiding wasting time on the user trying to find the interface controls that the user needs).

[0021] (A6) In some embodiments of the method described in A2, the method includes: determining that movement of the second extended contact on the touch-sensitive display before lift-off satisfies movement parameters, and in response, continuing to operate a portion of the touch-sensitive display as a touch pad for at least a predetermined amount of time.

[0022] Once it is determined that the second extended contact satisfies the movement parameters, the portion of the touch-sensitive display continues to be operated as a trackpad, which ensures that the trackpad mode is invoked whenever the user needs it. If the user moves the second extended contact in a downward direction or at a faster speed, this indicates to the user that the lift-off of the second extended contact is completed in order to allow the user to reposition his finger to continue moving the cursor (e.g., if the user moves in a downward direction, he may run out of space and will need to lift off to reposition his finger to continue moving the cursor). By ensuring that the trackpad mode is invoked whenever the user needs it and without interrupting the user's movement of the cursor, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (e.g., by ensuring that the user is presented with the interface controls he needs and avoiding wasting time on the user trying to find the interface controls he needs).

[0023] (A7) In some embodiments of the method described in any one of A1 to A6, the method includes: after detecting the lifting of the second extended contact from the touch-sensitive display, continuing to operate a portion of the touch-sensitive display as a touch pad for at least a predetermined amount of time (e.g., 0.5 seconds).

[0024] (A8) In some embodiments of the method described in A7, the touch pad is invoked after the second extended contact has remained in contact with the touch-sensitive display for at least a second predetermined amount of time; and the second predetermined amount of time is less than the predetermined amount of time.

[0025] (A9) In some embodiments of the method described in any one of A7 to A8, the method includes: stopping operating a portion of the touch-sensitive display as a touch pad based on determining that no input is detected above the portion of the touch-sensitive display within a predetermined amount of time after detecting lift-off of the second extended contact.

[0026] (A10) In some embodiments of the method described in any one of A6 or A7, the method includes: detecting a double-click input on the touch-sensitive display when a portion of the touch-sensitive display is operated as a touch pad, detecting the double-click input within a certain amount of time after detecting lift-off of a second extended contact; and, in response to detecting the double-click input, selecting an object displayed on the touch-sensitive display.

[0027] Allowing the user to perform a double-tap input to select an object ensures that while in trackpad mode, the user is able to select an object (e.g., text, images, multiple words, or a combination of both). The operability of the device is enhanced by ensuring that trackpad mode allows selection of objects, which then creates a continuous interaction with the device.

[0028] (A11) In some embodiments of the method according to A10, the method includes: detecting a drag gesture after a double-click input, and in response, selecting more objects as the drag gesture moves.

[0029] Allowing the user to perform a drag gesture after a double-tap input to select more (or fewer) objects ensures that when in trackpad mode, the user is able to select only those parts of the object that they want to select (e.g., a portion of text, an image, or a combination of both). The operability of the device is enhanced by ensuring that trackpad mode allows for fine-grained selection of objects, which then creates a continuous interaction with the device.

[0030] (A12) In some embodiments of the method described in any one of A1 to A11, an extended touch (or momentary touch) above any corresponding display representation of other keys activates the corresponding extended keyboard function and does not cause the electronic device to operate any part of the touch-sensitive display as a touchpad.

[0031] (A13) In some embodiments of the method described in A12, a virtual keyboard is displayed in an area of ​​the touch-sensitive display and each displayed representation of other keys includes a corresponding display symbol, and operating at least a portion of the touch-sensitive display as a touchpad includes (i) operating the area as part of a touchpad, and (ii) stopping displaying at least some of the display symbols of the virtual keyboard.

[0032] The same area of ​​the display used for the virtual keyboard is operated as a trackpad, which allows the user to immediately begin using the trackpad without having to reposition / reposition their fingers after invoking the trackpad, instead, the user simply begins moving their fingers as needed to move the cursor and / or select objects (e.g., using the double-click gesture and double-click and drag gesture discussed above). In addition, ceasing to display at least some of the displayed symbols of the virtual keyboard provides the user with a clear visual indication that the trackpad mode has been invoked. By providing these features, the operability of the device is enhanced, and then a continuous interaction with the device is created (e.g., the user does not need to waste time repositioning his fingers and will not be confused about when to invoke the trackpad, thereby allowing continuous interaction with the device).

[0033] (A14) In some embodiments of the method described in any one of A1 to A13, the method includes: detecting a third extended contact above a portion of a virtual keyboard, the portion of the virtual keyboard being below a displayed representation of a space bar, wherein the portion of the virtual keyboard does not include any other keys; and based on determining that the third extended contact is above the portion of the virtual keyboard, interpreting the third extended contact as a mode switch input that causes the electronic device to operate at least a portion of the touch-sensitive display as a touch pad.

[0034] (A15) In some embodiments of the method described in any of A1 to A14, when operated as a touch pad, any input above at least the portion on the touch-sensitive display controls movement of a cursor displayed in another portion of the touch-sensitive display.

[0035] (A16) In some embodiments of the method according to any one of A1 to A15, at least some of the other keys of the virtual keyboard are respectively associated with corresponding keyboard functions and with corresponding extended keyboard functions, wherein the corresponding keyboard functions are activated in response to a momentary contact above the keys, and the corresponding extended keyboard is activated in response to an extended contact above the keys.

[0036] (A17) In some embodiments of the method according to any one of A1 to A16, the first extended contact and the second extended contact last at least an extended contact detection time threshold.

[0037] (A18) In some embodiments of the method described in any one of A1 to A17, tactile feedback is provided in conjunction with invoking the operation of a portion of the touch-sensitive display as a touchpad.

[0038] (A19) In some embodiments of the method according to any one of A1 to A18, the displayed virtual keyboard does not include a function (fn) key.

[0039] (A20) A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of an electronic device having a touch-sensitive display, cause the electronic device to perform a method according to any one of claims A1 to A19, wherein the electronic device is not coupled to any force sensor.

[0040] (A21) An electronic device comprising one or more processors; a touch-sensitive display; and a memory storing one or more programs, the one or more programs being configured to be executed by the one or more processors, the one or more programs including instructions for executing a method described in any one of A1 to A19, and the electronic device being not coupled to any force sensor.

[0041] (A22) An electronic device having a touch-sensitive secondary display, wherein the electronic device is not coupled to any force sensor and the electronic device includes: a device for performing the method described in any one of A1 to A19.

[0042] (A23) An information processing device used in an electronic device, wherein the electronic device includes a touch-sensitive secondary display, wherein the electronic device is not coupled to any force sensor, and the information processing device includes: an apparatus for executing the method described in any one of A1 to A19.

[0043] (A24) A graphical user interface for an electronic device having one or more processors, a memory, and a touch-sensitive secondary display, wherein the electronic device is not coupled to any force sensor, and the one or more processors execute one or more programs stored in the memory, the graphical user interface including a user interface displayed according to any one of the methods described in A1 to A19.

[0044] (A25) An electronic device, comprising a touch-sensitive display coupled to a plurality of touch sensors, wherein the touch-sensitive display is not coupled to any force sensor, the electronic device further comprising a memory and one or more processors coupled to the plurality of touch sensors and the touch-sensitive display, the memory comprising executable instructions which, when executed by the one or more processors, cause the electronic device to perform the following: display a virtual keyboard on the touch-sensitive display, the virtual keyboard comprising display representations of a space bar and other keys; detect an extended contact at the touch-sensitive display; based on determining that the extended contact is above any corresponding display representations of other keys, interpret the extended contact as a keyboard input and activate a corresponding keyboard function; and based on determining that the extended contact is above a corresponding display representation of the space bar, interpret the extended contact as a mode switching input, the mode switching input causing the electronic device to operate at least a portion of the touch-sensitive display as a touch pad.

[0045] (A26) In some embodiments of the electronic device according to A25, the executable instructions, when executed by one or more processors, also cause the electronic device to perform a method according to any one of A1 to A19.

[0046] In another aspect, methods, devices, and other components for implementing a single-finger scroll / tip-to-groove gesture are also provided.

[0047] (B1) According to some embodiments, a method is performed at an electronic device including a touch-sensitive display coupled to a plurality of touch sensors, and the touch-sensitive display is not coupled to any force sensor. The method includes: displaying a virtual keyboard on the touch-sensitive display; detecting an output from at least one touch sensor of the plurality of touch sensors, the output indicating contact with the touch-sensitive display over at least a portion of the virtual keyboard, wherein the contact includes an area having a center of mass on the touch-sensitive display; detecting a change in the output from at least one of the plurality of touch sensors, the change in the output indicating that the area of ​​contact has expanded and the center of mass has moved; and based on the detected change, operating at least a portion of the touch-sensitive display as a touchpad and ceasing to operate the virtual keyboard.

[0048] Allowing a user to scroll a single finger over a virtual keyboard and then operate the electronic device as a trackpad provides a convenient way for the user to quickly switch from keyboard input mode to trackpad mode using only a single finger; and detecting the scrolling by monitoring the expansion of the area and the movement of the center of mass ensures that the gesture is accurately detected and not confused with other types of gestures. Providing this interaction at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more effective (e.g., by allowing the user to easily switch to trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate trackpad mode without using trackpad mode, or without having to waste time using a complex sequence of user input or menu options to activate trackpad mode).

[0049] (B2) In some embodiments of the method according to B1, the output and the change of the output are detected at least in part based on the capacitance signal stored by at least one touch sensor of the plurality of touch sensors.

[0050] Detecting outputs and changes in outputs based at least in part on capacitive signals helps ensure that gestures are accurately detected and not confused with other types of gestures, and enables devices that do not include force sensors to accurately detect new gesture types, such as the scroll / tip-to-groove gestures discussed herein. Providing such accurate detection at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more effective (e.g., by allowing users to easily switch to trackpad mode using simple gestures, rather than wasting time trying to place a cursor to activate trackpad mode when not in use, or wasting time using a complex sequence of user inputs or menu options to activate trackpad mode).

[0051] (B3) In some embodiments of the method described in any one of B1 to B2, changes in the output are detected based on one or more of: (i) the speed of movement of the center of mass; (ii) changes in the size of the area; (iii) the saturation level of at least one touch sensor among a plurality of touch sensors; and (iv) the calculated angle of a user's finger making contact relative to the touch-sensitive display.

[0052] (B4) In some embodiments of the method according to any one of B1 to B2, the change in the output is further detected based on the moving speed of the center of mass.

[0053] (B5) In some embodiments of the method according to any one of B1 to B2 or B4, the change in the output is further detected based on the change in the size of the area.

[0054] (B6) In some embodiments of the method according to any one of B1 to B2 or B4 to B5, the change in the output is further detected based on the saturation level of at least one touch sensor among the multiple touch sensors.

[0055] (B7) In some embodiments of the method described in any one of B1 to B2 or B4 to B6, changes in the output are further detected based on the calculated angle of the user's finger making contact relative to the touch-sensitive display.

[0056] (B8) In some embodiments of the method described in any one of B1 to B7, the method includes providing tactile feedback by the electronic device in combination with invoking the operation of a portion of the touch-sensitive display as a touch pad.

[0057] Detecting outputs and changes in outputs by monitoring the other data pieces discussed above (in B3 to B8) helps ensure that gestures are accurately detected and not confused with other types of gestures, and enables devices that do not include force sensors to accurately detect new gesture types, such as the roll / tip to texture gesture discussed herein. Adding monitoring of these other data pieces helps improve the accuracy of detecting the new gestures at electronic devices that do not include force sensors, thereby enhancing the operability of these devices and making the human-computer interface on these devices more effective (e.g., by ensuring that gestures are accurately detected, users will not have to waste time reversing the activation of undesired functions).

[0058] (B9) In some embodiments of the method described in any one of B1 to B8, text is displayed in an area of ​​a virtual keyboard displayed adjacent to a touch-sensitive display, a cursor is displayed at a position within the text, and the method further includes: while continuing to operate a portion of the touch-sensitive display as a touch pad: detecting an additional change in output from at least one of the multiple touch sensors, the change in output indicating that the area of ​​contact has decreased and the center of mass has moved; and in response to detecting the additional change in output, maintaining display of the cursor at a position within the text.

[0059] Ensuring that the cursor does not move (e.g., maintains its position) as the user reversely rolls their finger over the touch-sensitive display helps ensure that the cursor is accurately placed based on the user's intent. When the user merely reversely rolls their finger, the user does so before they begin to move the cursor. Thus, by maintaining the cursor in position during reverse scrolling, the operability of the device is enhanced and the human-machine interface operates more efficiently (e.g., by ensuring that the user does not need to waste time reversing unintended cursor movements).

[0060] (B10) In some embodiments of the method described in B9, after additional changes are detected and while continuing to operate a portion of the touch-sensitive display as a touchpad, movement of contact on the touch-sensitive display causes the cursor to move to a different location within the text.

[0061] (B11) In some embodiments of the method described in B9, the method includes: while continuing to operate a portion of the touch-sensitive display as a touch pad: detecting another change in the output from at least one touch sensor of the multiple touch sensors, the further change indicating that the area of ​​contact has expanded and the center of mass has moved; and in response to detecting the further change, selecting at least a portion of the text.

[0062] (B12) In some embodiments of the method according to B11, the method includes: detecting the lifting of contact from the touch-sensitive display, and in response, ceasing to operate a portion of the touch-sensitive display as a touch pad.

[0063] (B13) A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of an electronic device having a touch-sensitive display, cause the electronic device to perform a method according to any one of claims B1 to B12, wherein the electronic device is not coupled to any force sensor.

[0064] (B14) An electronic device comprising one or more processors; a touch-sensitive display; and a memory storing one or more programs, the one or more programs being configured to be executed by the one or more processors, the one or more programs including instructions for executing a method described in any one of B1 to B12, and the electronic device being not coupled to any force sensor.

[0065] (B15) An electronic device having a touch-sensitive secondary display, wherein the electronic device is not coupled to any force sensor, and the electronic device includes: a device for performing the method described in any one of B1 to B12.

[0066] (B16) An information processing device for use in an electronic device including a touch-sensitive secondary display, wherein the electronic device is not coupled to any force sensor, the information processing device comprising: an apparatus for performing the method described in any one of B1 to B12.

[0067] (B17) A graphical user interface for an electronic device having one or more processors, a memory, and a touch-sensitive secondary display, wherein the electronic device is not coupled to any force sensor, and the one or more processors execute one or more programs stored in the memory, the graphical user interface including a user interface displayed according to any one of the methods described in B1 to B12.

[0068] It should be noted that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described in this specification are not comprehensive, and in particular, many additional features and advantages will be apparent to those of ordinary skill in the art based on the drawings, the specification, and the claims. In addition, it should be noted that the language used in this specification is selected in principle for readability and instructional purposes, and may not be selected to describe or define the subject matter of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0070] Figure 1A is a high-level block diagram of a computing device with a touch-sensitive display according to some embodiments.

[0071] Figure 1B is a block diagram of exemplary components for event processing according to some embodiments.

[0072] Figure 2A is a schematic diagram of a portable multifunction device with a touch screen (also interchangeably referred to herein as a "touch-sensitive display") according to some embodiments.

[0073] Figure 2B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0074] Figure 3 is a schematic diagram of a touch screen for showing a user interface for an application menu according to some embodiments.

[0075] Figure 4 is a flowchart illustrating a method of activating a trackpad mode using an extended contact above a virtual keyboard, according to some embodiments.

[0076] Figures 5A-1 to 5K-6 An exemplary user interface is shown for activating trackpad mode using an extended contact above a virtual keyboard (e.g., above a displayed representation of a space bar) in accordance with some embodiments.

[0077] 6A to 6F is a flowchart illustrating a method of activating a trackpad mode using an extended contact above a virtual keyboard, according to some embodiments.

[0078] Figure 7 is a flow chart illustrating a method of activating a trackpad mode using a single-finger scrolling gesture, according to some embodiments.

[0079] Figures 8A-1 to 8F An exemplary user interface is shown for activating touchpad mode using a single-finger scroll gesture in accordance with some embodiments.

[0080] 9A to 9D is a flow chart illustrating a method of activating a touchpad mode using a single-finger scrolling gesture, according to some embodiments. DETAILED DESCRIPTION

[0081] Figure 1A to Figure 1B , FIG. 2A to FIG. 2B as well as Figure 3 Exemplary devices and systems are shown upon which the methods and techniques described herein may be implemented.

[0082] Figures 5A-1 to 5K-6 is a schematic diagram of a display according to some embodiments, which is used to illustrate an exemplary user interface for using an extended contact above a virtual keyboard to activate a touch pad on an electronic device that has a touch-sensitive display and does not have a force sensor. Figure 4 as well as 6A to 6F is a flowchart of a method for activating a touch pad on an electronic device having a touch-sensitive display and no force sensor using an extended contact above a virtual keyboard according to some embodiments. Figures 5A-1 to 5K-6 The user interface in Figure 4as well as 6A to 6F The method and / or process in.

[0083] Figures 8A-1 to 8F An exemplary user interface is shown for activating touchpad mode using a single-finger scroll gesture in accordance with some embodiments. Figure 7 as well as 9A to 9D is a flow chart of a method for activating a touchpad mode using a single-finger scroll gesture according to some embodiments. Figures 8A-1 to 8F The user interface in Figure 7 as well as 9A to 9D The method and / or process in.

[0084] Exemplary Devices and Systems

[0085] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail, so as not to unnecessarily obscure various aspects of the embodiments.

[0086] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish an element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact.

[0087] The terms used in the description of various embodiments described herein are only for the purpose of describing specific embodiments, and are not intended to be limited. As used in the description of various embodiments described and in the appended claims, the singular forms "a", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more items in the items listed in association. It will also be understood that the terms "including" and / or "comprising" are used in this specification to specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their grouping.

[0088] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0089] The disclosure herein interchangeably refers to detecting touch input at, at, above, on top of, or substantially within a particular user interface element or portion of a touch-sensitive display. As used herein, a touch input detected “at” a particular user interface element can also be detected “on,” “over,” “on top of,” or “substantially within” the same user interface element. In some embodiments and as described in more detail below, the desired sensitivity level for detecting touch input is configured by a user of the electronic device (e.g., a user can decide (and configure the electronic device to be operated) that touch input should only be detected when the touch input is completely within a user interface element).

[0090] Embodiments of electronic devices, user interfaces for such devices, and processes associated with using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple iPod from Apple Inc (Cupertino, California). IPOD and Device. Other portable electronic devices, such as laptops or tablets with touch-sensitive surfaces (e.g., touch screen displays and / or trackpads) are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or trackpads).

[0091] In the following discussion, an electronic device is described that includes a display and a touch-sensitive surface. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0092] The device typically supports a variety of applications, such as one or more of the following applications: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a health / fitness application, a photo management application, a digital camera application, a digital camcorder application, a web browsing application, a digital music player application, and / or a digital video player application.

[0093] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and clear to the user.

[0094] Attention is now directed to embodiments of portable electronic devices having touch-sensitive displays. Figure 1A 1 is a block diagram illustrating a portable multifunction device 100 (also interchangeably referred to herein as electronic device 100 or device 100) with a touch-sensitive display 112 according to some embodiments. Touch-sensitive display 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes referred to as or referred to as a touch-sensitive display system. Device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), controller 120, one or more processing units (CPU) 122, peripheral device interface 118, RF circuit 108, audio circuit 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or a touch pad of device 100). These components optionally communicate via one or more communication buses or signal lines 103.

[0095] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when there is no change in the smoothness of the touch-sensitive surface, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. Although such a user's interpretation of touch will be limited by the user's individualized sensory perceptions, many sensory perceptions of touch are common to most users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., "press click," "release click," "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or a component thereof that would generate that sensory perception for a typical (or average) user.

[0096] It should be understood that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings may be implemented in hardware, software, or a combination of software and hardware, including one or more signal processing and / or application specific integrated circuits.

[0097] The memory 102 optionally includes high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, or other random access solid-state memory devices), and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 102 optionally includes one or more storage devices that are located remotely from the one or more processors 122. Access to the memory 102 by other components of the device 100, such as the CPU 122 and the peripheral device interface 118, is optionally controlled by the controller 120.

[0098] Peripheral interface 118 may be used to couple the device's input and output peripherals to CPU 122 and memory 102. One or more processors 122 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0099] In some embodiments, peripherals interface 118, CPU 122, and controller 120 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0100] RF (radio frequency) circuit 108 receives and sends RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, user identity modules (SIM) cards, memory, and the like. RF circuit 108 optionally communicates with networks and other devices through wireless communications, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). Wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n).

[0101] The audio circuit 110, the speaker 111, and the microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into a sound wave audible to humans. The audio circuit 110 also receives an electrical signal converted from a sound wave by the microphone 113. The audio circuit 110 converts the electrical signal into audio data, and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack. The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single-ear headset or a dual-ear headset) and input (e.g., a microphone).

[0102] The I / O subsystem 106 connects input / output peripherals such as a touch screen 112 and other input control devices 116 on the device 100 to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / send electrical signals to the other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any one of the following (or not coupled to any one of the following): a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more buttons may optionally include an increase / decrease button for volume control of a speaker 111 and / or a microphone 113. One or more buttons optionally include a push button.

[0103] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch screen 112 and / or sends electrical signals to the touch screen. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

[0104] The touch screen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on tactile and / or haptic contact. The touch screen 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the area under the user's finger.

[0105] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, or OLED (organic light emitting diode) technology, although other display technologies are used in other embodiments. The touch screen 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in Apple Inc. (Cupertino, California) IPOD and Technology in.

[0106] The touch screen 112 optionally has a video resolution of more than 400 dpi. In some embodiments, the touch screen 112 has a video resolution of at least 600 dpi. In other embodiments, the touch screen 112 has a video resolution of at least 1000 dpi. The user optionally uses any suitable object or finger such as a stylus or finger, etc. to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures. In some embodiments, the device converts finger-based input into precise pointer / cursor positions or commands for performing actions desired by the user.

[0107] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface that is separate from the touch screen 112 or is an extension of the touch-sensitive surface formed by the touch screen.

[0108] The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0109] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor is shown, which is coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the optical sensor 164 optionally captures a static image or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to the touch screen 112 on the front of the device, so that the touch-sensitive display can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device, so that the user can optionally obtain the image of the user for video conferencing while watching other video conference participants on the touch-sensitive display.

[0110] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I / O subsystem 106. In some embodiments, when the multifunction device is placed near a user's ear (e.g., when the user is making a phone call), the proximity sensor turns off and disables touch screen 112.

[0111] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1AA tactile output generator is shown, which is coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices, such as a speaker or other audio component; and / or an electromechanical device that converts energy into linear motion, such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts an electrical signal into a tactile output on the device). In some embodiments, at least one tactile output generator is arranged in juxtaposition or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward to the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on the rear of the device 100 opposite to the touch-sensitive display 112 located on the front of the device 100.

[0112] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown, which is coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch-sensitive display based on an analysis of data received from the one or more accelerometers. The device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to the accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0113] In some embodiments, the software components stored in memory 102 include one or more of an operating system 126, an active module 163 (optionally including a keyboard module 163-1, a touchpad module 163-2, an input switching module 163-3 (e.g., keyboard to touchpad or touchpad to keyboard), a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136). In addition, in some embodiments, memory 102 stores device / global internal state 157, such as Figure 1A. The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display 112; a sensor state, including information obtained from the device's various sensors and input control devices 116; and position information about the device's position and / or posture (e.g., the device's orientation).

[0114] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.

[0115] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., a universal serial bus (USB), FireWire, etc.) is suitable for direct coupling to other devices or indirectly coupling via a network (e.g., the Internet, a wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, or similar to, and / or compatible with the 30-pin connector used in some embodiments of the IPOD device from APPLE Inc. In other embodiments, the external port is a multi-pin (e.g., 8-pin) connector that is the same as, or similar to, and / or compatible with the 8-pin connector used in the LIGHTNING connector from APPLE Inc.

[0116] The contact / motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to the detection of contact, such as determining whether contact has occurred (e.g., detecting a finger press event), determining whether the contact is prolonged (e.g., detecting the duration of a finger press event), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event, a contact break, or a finger lift event). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single contact (e.g., one finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some aspects, contact / motion module 130 and display controller 156 detect contact on a touch pad.

[0117] The contact / motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same position (or substantially the same position) as the finger press event (e.g., at an icon position). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and in some embodiments, also subsequently detecting a finger lift (lift-off) event. In yet another example, detecting a finger scroll gesture on a touch-sensitive surface includes detecting a fingertip press event, then detecting a continuous increase in the contact area between the finger and the touch-sensitive surface, and then further detecting fingerprint contact with the touch-sensitive surface. Similarly, tapping, swiping, dragging, scrolling, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0118] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch screen 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, without limitation, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

[0119] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application program, etc., and also receives coordinate data and other graphic attribute data together if necessary, and then generates screen image data to be output to the display controller 156.

[0120] Haptic feedback module 133 includes various software components for generating instructions used by tactile output generator 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

[0121] The text input module 134, which is optionally a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., the contacts module 137, email client module 140, IM module 141, browser module 147, and any other application requiring text input).

[0122] The GPS module 135 determines the location of the device and provides such information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and maps / navigation widgets).

[0123] Application program ("app") 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0124] Contacts module 137 (sometimes called address book or contact list);

[0125] Telephone module 138;

[0126] Video conferencing module 139;

[0127] Email client module 140;

[0128] Instant messaging (IM) module 141;

[0129] Health module 142;

[0130] A camera module 143 for still images and / or video images;

[0131] Image management module 144;

[0132] Browser module 147;

[0133] Calendar module 148;

[0134] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock market widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets (149-6) acquired or created by the user;

[0135] Search module 151;

[0136] Video and music player module 152, which optionally consists of a video player module and a music player module;

[0137] Notepad module 153;

[0138] Map module 154; and / or

[0139] Online video module 155.

[0140] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, website creation applications, disk editing applications, spreadsheet applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, a widget creator module 149-6 for making user-created widgets, and voice replication.

[0141] In combination with the touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (for example, stored in the contact module 137 in the memory 102 or the memory 370), including: adding one or more names to the address book; deleting names from the address book; associating telephone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing telephone numbers or email addresses to initiate and / or facilitate communications through the telephone module 138, video conferencing module 139, email client module 140, or IM module 141; and the like.

[0142] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 is optionally used to enter a sequence of characters corresponding to a phone number, access one or more phone numbers in address book 137, modify an entered phone number, dial a corresponding phone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0143] In combination with RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137 and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0144] In conjunction with RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0145] In conjunction with the RF circuit 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for inputting a character sequence corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages transmitted using SMS or MMS) and Internet-based messages (e.g., messages transmitted using XMPP, SIMPLE, or IMPS).

[0146] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the video and music player module 146, the health module 142 includes executable instructions for the following operations: creating a fitness (e.g., with a time, distance, and / or calorie burn goal); communicating with a fitness sensor (a sports device such as a watch or a pedometer); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting fitness music and playing it; and displaying, storing, and transmitting fitness data.

[0147] In conjunction with touch screen 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0148] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still images and / or video images.

[0149] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving and displaying web pages or parts thereof and attachments and other files linked to web pages) according to user instructions.

[0150] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0151] In conjunction with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5) or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).

[0152] In combination with the RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134 and browser module 147, a desktop widget creator module (not shown) is optionally used by a user to create a desktop widget (e.g., to transfer a user-specified portion of a web page to a desktop widget).

[0153] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 includes executable instructions for searching memory 102 for text, music, sound, image, video, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions. In some embodiments, search module 151 also includes executable instructions for displaying a search entry portion and a prediction portion. In some embodiments, search module 151, in conjunction with active module 163, also populates the prediction portion with affordances of suggested or predicted people, actions within applications, applications, nearby places, and / or news articles before any user input is received at the search entry portion.

[0154] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions to display, present, or otherwise play back video (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes functionality of an MP3 player, such as the IPOD from APPLE Inc.

[0155] In conjunction with the touch screen 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do lists, etc. according to user instructions.

[0156] In combination with the RF circuit system 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, and the browser module 147, the map module 154 is optionally used to receive, display, modify, and store executable instructions for maps and data associated with the maps (e.g., driving routes; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0157] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on the touch screen or on an external display connected via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0158] like Figure 1A As shown, portable multifunction device 100 also includes other modules 163 for performing certain keyboard functions and touchpad functions and allowing switching between keyboard functions and touchpad functions.

[0159] Other modules 163 optionally include the following modules (or instruction sets) or a subset or superset thereof:

[0160] A keyboard module 163-1 for receiving user input at a displayed virtual keyboard and determining which keyboard function to actuate in response to the user input (which may be a keyboard function or an extended keyboard function, depending on the amount of time the corresponding input is in contact with the touch-sensitive display);

[0161] A touchpad module 163-2 for receiving user input when at least a portion of the touch-sensitive display operates as a touchpad; and

[0162] An input switching module 163-3 for detecting user input that enables switching between touchpad and keyboard modes (e.g., an extended touch discussed below or a tip-to-texture / scroll gesture discussed below, each of which can enable switching from keyboard to touchpad mode).

[0163] Each module and application program in the above-mentioned identified modules and applications corresponds to a set of executable instructions for performing one or more functions and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) do not have to be implemented as independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures determined above. In addition, memory 102 optionally stores other modules and data structures not described above.

[0164] In some embodiments, the device 100 is a device on which operation of a predefined set of functions is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

[0165] A predefined set of functions that are performed exclusively through the touch screen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, when the user touches the touchpad, the touchpad navigates the device 100 from any user interface displayed on the device 100 to a main menu, a home menu, or a root menu. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0166] Figure 1B 1 is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A) includes an event classifier 170 (e.g., in an operating system 126) and a selected from a portable multifunction device 100 ( Figure 1A )'s corresponding application 136-1 among the applications 136 (eg, any of the aforementioned applications stored in the memory 102 together with the application 136).

[0167] Event classifier 170 receives event information and determines the application 136-1 and the application view 191 of application 136-1 to which the event information is to be delivered. Event classifier 170 includes event monitor 171 and event distributor module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0168] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating information being displayed by the application 136-1 or information ready to be displayed by the application 136-1, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0169] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

[0170] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0171] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0172] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining that a sub-event has occurred at a location within one or more views. A view consists of controls and other elements that a user can see on the display.

[0173] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0174] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0175] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, higher views in the hierarchy will still remain actively participating views.

[0176] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers event information to an event recognizer determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores event information in an event queue, which is retrieved by corresponding event receiver 182.

[0177] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or is part of another module stored in memory 102, such as contact / motion module 130.

[0178] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring in a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, the corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more event recognizers in event recognizers 180 are part of an independent module, which is a higher-level object such as a user interface toolkit (not shown) or application 136-1 from which methods and other properties are inherited. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177 or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in the corresponding application view 191.

[0179] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event from the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes at least a subset of the following two: metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0180] Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (e.g., from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0181] Event comparator 184 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes a definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift (touch end) of a predetermined duration. In another example, event 2 (187-2) is defined as dragging a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0182] In some embodiments, event definition 186 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view (where three user interface objects are displayed on touch-sensitive display 112), when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with a sub-event and the object that triggered the hit test.

[0183] In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays the delivery of the event information until it has been determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0184] When a corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which it ignores subsequent sub-events of the touch-based gesture. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0185] In some embodiments, the corresponding event identifier 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event identifiers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event identifiers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0186] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with the tag obtains the tag and performs a predefined process.

[0187] In some embodiments, the event delivery instructions 188 include a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or with an actively participating view receives the event information and performs a predetermined process.

[0188] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137, or stores video files used in video or music player module 145. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 176 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on a touch-sensitive display.

[0189] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0190] It should be understood that the above discussion of event processing of user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate multifunction device 100, and not all user input is initiated on a touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events defining an event to be recognized.

[0191] Figure 2A1 is a schematic diagram of a portable multifunction device (e.g., portable multifunction device 100) with a touch-sensitive display (e.g., touch screen 112) according to some embodiments. In this embodiment, and in other embodiments described below, a user can select one or more of these graphics by, for example, making gestures on the screen with one or more fingers or with one or more styluses. In some embodiments, when the user interrupts contact with one or more graphics (e.g., by lifting a finger from the screen), selection of one or more graphics will occur. In some embodiments, the gesture optionally includes one or more tap gestures (e.g., a series of touches to the screen after lifting), one or more swipe gestures (continuous contact during the gesture along the surface of the screen, e.g., from left to right, from right to left, up and / or down) and / or rolling of a finger that has made contact with the device 100 (e.g., from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with a graphic will not select the graphic. For example, when the gesture used to launch an application is a tap gesture, a swipe gesture that sweeps over an application enable representation (e.g., an icon) may not launch (e.g., open) the corresponding application.

[0192] The device 100 optionally also includes one or more physical buttons, such as a "home" button or a menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112.

[0193] In one embodiment, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for turning the device on and off and locking the device for powering, a volume adjustment button 208, a user identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and keeping the button pressed for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In an alternative embodiment, the device 100 also accepts voice input for activating or deactivating certain functions through a microphone 113. The device 100 also optionally includes one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0194] Figure 2B is used to show that the device (for example, Figure 1A100) having a touch-sensitive surface 251 (e.g., a tablet or touchpad) that is separate from a display 250 (e.g., touch screen 112). In some embodiments, touch-sensitive surface 251 includes one or more tactile output generators 257 that are used to generate tactile output for a user of touch-sensitive surface 251.

[0195] Although some of the examples that follow will be given with reference to input on touch screen 112 (in which a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 2B In some embodiments, the touch-sensitive surface (e.g., Figure 2B 251) has a main axis (e.g., Figure 2B 253) corresponding to the main axis (for example, Figure 2B According to these embodiments, the device detects contact with the touch-sensitive surface 251 at a location corresponding to a corresponding location on the display (e.g., Figure 2B 260 and 262) (e.g., in Figure 2B , 260 corresponds to 268 and 262 corresponds to 270). Thus, when a touch-sensitive surface (e.g., Figure 2B 251) and a display of a multi-function device (e.g., Figure 2B When the user input detected by the device on the touch-sensitive surface (e.g., contacts 260 and 262 and their movement) is separated, the device is used to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0196] In addition, although the following examples are primarily given with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detecting the contact, followed by ceasing to detect the contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and multiple finger contacts are optionally used simultaneously.

[0197] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 2B In the case where an input (e.g., a press input) is detected on the touch-sensitive surface 251 (in some embodiments, the touch-sensitive surface 251 is a touch pad) in the display, the particular user interface element is adjusted according to the detected input. In the case where a touch-sensitive surface 251 (e.g., a touch-sensitive surface 251 is a touch pad) in the display includes a touch-sensitive surface 251 (e.g., a touch pad) in the display, the particular user interface element is adjusted according to the detected input. Figure 1A In some implementations of a touch-sensitive display system 112 or touch screen 112 in a display device, a contact detected on the touch screen acts as a "focus selector" such that when an input (e.g., a press input by the contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted in accordance with the detected input. In some implementations, the focus moves from one area of ​​the user interface to another area of ​​the user interface without corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves in accordance with the movement of the focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user to convey the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch-sensitive display), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).

[0198] Figure 3 is a schematic diagram of a touch screen according to some embodiments, the touch screen being used to display a user interface of a menu of applications. A similar user interface is optionally provided in device 100 ( Figure 1A In some embodiments, the user interface displayed on the touch screen 112 includes the following elements, or a subset or superset thereof:

[0199] One or more signal strength indicators 202 of one or more wireless communications, such as cellular signals and Wi-Fi signals;

[0200] Time 203;

[0201] Bluetooth indicator 205;

[0202] Battery status indicator 206;

[0203] A tray 209 with icons for commonly used applications, such as:

[0204] o An icon 216 labeled "Phone" for the phone module 138, which optionally includes an indicator 214 of the number of missed calls or voice messages;

[0205] o an icon 218 labeled "Mail" for the email client module 140, which optionally includes an indicator 210 of the number of unread emails;

[0206] o An icon 220 labeled "Browser" for the browser module 147; and

[0207] o an icon 222 labeled “iPod” for the video and music player module 152 (also referred to as the iPod (trademark of Apple Inc.) module 152); and

[0208] Icons of other applications, such as:

[0209] o Icon 224 labeled "Messages" for the IM module 141;;

[0210] o Icon 226 labeled "Calendar" for calendar module 148;

[0211] o Icon 228 labeled "Photos" for the image management module 144;

[0212] ○ Icon 230 labeled “Camera” for camera module 143;;

[0213] o An icon 232 labeled "Online Video" for the online video module 155;

[0214] ○ Icon 234 labeled "Stock Market" for the Stock Market widget 149-2;

[0215] o Icon 236 labeled "Map" for the map module 154;

[0216] ○ Icon 238 labeled "Weather" for the Weather widget 149-1;

[0217] ○ Icon 240 labeled “Clock” for the Alarm Clock widget 149-4;

[0218] o An icon 242 labeled "Health" for the health module 142;;

[0219] ○ Icon 244 labeled "Notepad" for the Notepad module 153;;

[0220] o An icon 246 for a settings application or module that provides access to settings for the device 100 and its various applications; and

[0221] o Other icons for additional applications such as App Store, iTunes, Voice Memos, and Utilities.

[0222] It should be noted that the icon labels shown in FIG. 2 are merely exemplary. Other labels are optionally used for various application icons. For example, icon 242 of health module 142 is alternatively labeled "Fitness Support," "Exercise," "Exercise Support," "Sports," "Sports Support," or "Fitness." In some embodiments, the label of a respective application icon includes the name of the application corresponding to the respective application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0223] User interface and associated processes

[0224] Attention now turns to embodiments of the process and associated user interface ("UI") that may be implemented on an electronic device such as portable multifunction device 100 having a display, a touch-sensitive surface, and optionally one or more sensors for detecting intensity of contacts with the touch-sensitive surface.

[0225] Figure 4 4 is a flow chart illustrating a method 400 for activating a touchpad mode using an extended contact over a virtual keyboard on a touch-sensitive display according to some embodiments. Figure 1A A portable multifunction device (e.g., Figure 1A The method 400 is performed at the device 100 in the embodiment of the present invention. As described below, the method 400 does not require the portable multifunction device to include a contact intensity sensor (e.g., a force sensor or a pressure sensor), thereby reducing the cost of designing and manufacturing the portable multifunction device. The method is also intuitive to the user, thereby reducing the amount, degree and / or nature of input from the user when the trackpad mode is activated, and creating a more efficient human-computer interface. A more efficient input mechanism also requires less computing resources, thereby increasing the battery life of the device.

[0226] Figures 5A-1 to 5K-6An exemplary user interface is shown for activating trackpad mode using an extended touch above a virtual keyboard (e.g., above the space bar on a virtual keyboard) in accordance with some embodiments. Figures 5A-1 to 5K-6 The user interface in the Figure 4 as well as 6A to 6F The process described. Figures 5A-1 to 5K-6 Some of the illustrative embodiments also include schematic diagrams showing the passage of time that help illustrate particular time-dependent features, and these representations of the passage of time are merely examples and are not what is displayed on the electronic device during operation of the electronic device.

[0227] exist Figure 5A-1 In the embodiment of the present invention, when an application (e.g., notepad application 153 or message application 141) is in document editing mode, editable content 500 is displayed in content presentation area 502. In some embodiments, content 500 in content presentation area 502 includes text (e.g., plain text, unstructured text, formatted text, or text in a web page). In other embodiments, content includes graphics with or without text. In addition, content can be editable or read-only. In addition to displaying content, when no content is selected, device 100 can display a cursor (e.g., cursor 522) within the electronic document, for example, for text input, text selection, and text manipulation.

[0228] like Figure 5A-1 As shown, in the document editing mode, an on-screen keyboard 521 (e.g., also referred to as a virtual keyboard or soft keyboard) is displayed for text input. The user can input text by performing touch input (also referred to as contact input) on the virtual keyboard 521. The user can also confirm the completion of the editing by performing another touch input (e.g., tapping the "Done" button 504) to exit the text editing mode and stop displaying the virtual keyboard 521.

[0229] Figure 5A-1 Also shown is a display of an insertion marker (e.g., cursor 522) that indicates the current location to which the input set at the virtual keyboard will be directed. Figure 5A-1 As shown, keyboard 521 has a plurality of virtual keys, and each key has a corresponding symbol describing the function of the key. In addition, in some embodiments, each key is displayed in a manner simulating three-dimensional virtual features (e.g., three-dimensional shape, projection, etc.).

[0230] In some embodiments, the application runs on a device 100 having one or more touch sensors (e.g., capacitive sensors) for measuring the proximity, position, displacement, or acceleration of one or more contacts with a touch-sensitive display. In some embodiments, the device 100 continuously monitors the touch input and the continuous movement of the touch input on the touch screen 112. In some embodiments, the device 100 has only a touch sensor and does not include other sensors for detecting the force or pressure of the input.

[0231] See also Figure 4 and Figure 5A-1 In some embodiments, the device 100, while displaying the content of the electronic document, also detects (402) contact input 510 on the touch-sensitive display of the device 100. The contact input may be performed by the user using, for example, his or her finger or by using a stylus.

[0232] In some embodiments, the device 100 determines (404) whether a contact input is detected over a mode switching region of the virtual keyboard 521 associated with activation of the trackpad mode while the virtual keyboard 521 is displayed on at least a portion of the touch screen display. For example, upon detecting an extended contact over the mode switching region (e.g., lasting at least an extended contact duration threshold, such as 0.25 seconds), the device 100 (e.g., using input switching mode 163-3, Figure 1A ) switches the keyboard 521 to a soft touchpad (also called a virtual touchpad, on-screen touchpad).

[0233] In some embodiments, the mode switching area corresponds to a display representation of the space bar. In some embodiments, the mode switching area corresponds to a non-character key (e.g., a language switch key or a microphone activation key) that is not associated with extended keyboard functionality (e.g., an extended contact over a non-character key activates the same function as a momentary contact over the key). In some embodiments, a UI object (e.g., an enable representation or a virtual button) displayed on the touch screen display of device 100 and different from the keys of the virtual keyboard is used as the mode switching area. In some embodiments, an area that does not include any display representation of a key and may be visually different from the display representation of the keys of the virtual keyboard is used as the mode switching area, such as the area 506 between the space bar on the virtual keyboard and the bottom edge of the touch screen display. In some embodiments, when device 100 displays a non-alphanumeric virtual keyboard such as an emoji keyboard, device 100 may use area 506 (or a portion of area 506, such as a portion adjacent to the bottom edge of the touch-sensitive display) as the mode switching area. For simplicity of illustration and explanation only, and without limiting the scope of the present disclosure, the display representation of the space bar of keyboard 521 is Figures 5A-1 to 5K-6600 . Other examples described in this paragraph as mode switching regions may be used in place of or in addition to the illustrated representation of the space bar as a mode switching region (e.g., in conjunction with method 600 discussed in detail below). In some embodiments, the space bar includes a displayed symbol (e.g., "space"), such as Figure 5A-1 In some alternative embodiments, the space bar does not include a displayed symbol.

[0234] In some embodiments, when detecting (402) contact input 510, device 100 determines that contact input 510 is above a virtual key corresponding to the letter "e", which is not part of the mode switch area for activating the trackpad mode (404—No). Figure 5A-1 As shown, in response to detecting contact input 510 on the letter "e" key, device 100 also displays a preview area 512 that extends from the corresponding key to provide an input preview of the letter "e".

[0235] Device 100 also determines (406) whether the contact input is an extended contact. For example, device 100 determines whether contact input 510 has a duration that is longer than or equal to an extended contact duration threshold. In some embodiments, the extended contact duration threshold is selected to be any suitable time period that allows the device to distinguish an extended contact from a transient contact, an example of a suitable time period being 0.25 seconds. Figure 5A-1 As shown, a timer is used to show the duration of contact input 510. In the current example, device 100 detects that contact input 510 has a duration of 0.03 seconds, starting from initial contact 510.

[0236] In some embodiments and as Figure 5A-2 As shown, according to determining that the contact input 510 is not an extended contact (406—No, Figure 4 ), the keyboard input mode is maintained (410). For example, because the duration of contact 510 is shorter than the extended contact duration threshold, contact input 510 is determined to be a momentary contact (e.g., a short single click). In some embodiments, the keyboard input mode includes: when the device detects that the contact input has stopped within a predetermined time threshold (e.g., the finger is lifted off), inputting content corresponding to the contact input. For example, Figure 5A-2 As shown, after contact 510 continues on the display for 0.03 seconds, when device 100 determines that contact input 510 has stopped (eg, the finger is lifted off), device 100 adds the letter “e” to region 502 .

[0237] In some embodiments and as Figure 5B-1As shown, when device 100 detects that the duration of contact input (e.g., contact input 514) on the touch-sensitive display is 0.24 seconds, device 100 determines that contact input 514 is still not an extended contact, and thus displays preview area 512 of the letter of the corresponding touch key (e.g., the letter "e"). When the duration of contact input 514 is greater than or equal to the extended contact duration threshold (e.g., Figure 5B-2 0.25 seconds), the device 100 determines the contact input (eg, contact input 514, Figure 5B-2 ) is to extend the contact (406—Yes, Figure 4 ). The device 100 then proceeds to determine (408) whether the key over which the contact 514 was received is associated with an extended keyboard function. In some embodiments, the extended keyboard function includes displaying a set of accented letters. The extended features may include any other suitable features associated with different keys on the virtual keyboard.

[0238] In some embodiments, such as Figure 5B-2 As shown, according to determining that a key (e.g., the "e" key in the example) is associated with the extended keyboard function (408—Yes, Figure 4 ), the device 100 displays (414) an extended feature bar 526 that lists a set of accent marks associated with the "e" key. The device 100 enters an accented letter selected from the extended feature bar 526 in the keyboard input mode (410). In some embodiments, the user can move the input 514 to select certain accented letters in the extended feature bar 526. Figure 5B-3 As shown, upon detecting a lift-off from the touch-sensitive display (and without requiring the user to move input 514 to select the accented letter option), device 100 actuates the "e" key and stops displaying extended feature bar 526 (as shown in FIG. Figure 5B-3 shown).

[0239] In some embodiments, such as Figure 5C-1 As shown, according to determining that the letter corresponding to the contact input (e.g., contact input 518) does not have an associated extended keyboard function (408—No, Figure 4 ), the device 100 remains in keyboard input mode (410) and does not display any extended keyboard features / functions for the key. For example, although contact input 518 on the virtual keyboard corresponding to the letter "q" continues for a duration of 0.25 seconds, which is equal to the example extended contact duration threshold discussed above, the device 100 still displays the preview area 516 for the letter "q" because the letter "q" is not associated with the extended keyboard functionality in this current example. Figure 5C-2 As shown, upon detecting lift-off of contact 518 from the touch-sensitive display, the device adds “q” to content presentation area 502 .

[0240] In some embodiments, in response to detecting a contact input (e.g., contact input 530, Figure 5D-1 ), the device 100 determines that the contact input 530 is above the mode switching area on the virtual keyboard (eg, above the displayed representation of the space bar) (404—Yes, Figure 4 ). In some embodiments, Figure 5D-1 As shown, upon detecting contact input 530 over the displayed representation of the space bar, the device also highlights the space bar (e.g., by changing the edge of the space bar so that it appears in a thicker line, or by changing the color of the space bar).

[0241] In some embodiments, device 100 determines (418) whether contact input 530 over the mode switch region is an extended contact. In some embodiments, device 100 sets the duration of contact input 530 (e.g., Figure 5D-1 0.03 seconds in (0.03 seconds) is compared with the extended contact duration threshold to determine whether the contact input 530 is an extended contact.

[0242] In some embodiments, such as Figure 5D-2 As shown, the device detects lift-off of contact 530 from the touch-sensitive display. Figure 5D-1 The duration of the contact input 530 in the embodiment is shorter than the extended contact duration threshold, the device 100 determines that the contact input 530 is not an extended contact (418—No, Figure 4 ). Therefore, the device 100 remains in the keyboard input mode (420). For example, Figure 5D-2 As shown, when the device 100 determines that the contact input 530 is a short click on the space bar, the device 100 adds a space to the content presentation area 502 and remains in the keyboard input mode.

[0243] In some embodiments, such as Figure 5E-1 As shown in FIG. 5E-2 , when the device 100 detects that the contact input (e.g., contact input 532) is maintained on the space bar for 0.24 seconds and is not lifted from the touch screen, the device 100 still determines that the contact input 532 is not an extended contact. As shown in FIG. 5E-3 , when the duration of the contact input 532 reaches 0.25 seconds, 0.25 seconds is equal to the extended contact duration threshold, and the device 100 determines that the contact input 532 is an extended contact (418—Yes, Figure 4 In response, device 100 (eg, using input switching module 163-3, Figure 1A ) switches the input mode from the keyboard mode to the touchpad mode (422). Figure 5E-2a As shown, device 100 changes the virtual keyboard (eg, stops displaying symbols on the keyboard) when entering trackpad mode.

[0244] In some embodiments and as Figure 5E-2a As shown, the soft touchpad 523 may still include a key border, but no symbol is displayed to provide a clear visual cue to the user that the device has switched to touchpad mode. Figure 5E-2b As shown, the soft touchpad 524 also removes the borders of the virtual keys on the virtual keyboard to more completely simulate the appearance of a touchpad.

[0245] Furthermore, in some embodiments, Figure 5E-1 The color, hue, saturation, brightness and / or contrast of the virtual keyboard 521 in the application program also changes (e.g., becomes gray and semi-transparent) under the soft touchpad 523 to indicate that the application program has entered a mode different from the keyboard input mode (e.g., a touchpad mode). In the touchpad mode, the soft touchpad 523 does not respond to contact input for text input, but acts as an on-screen touchpad or trackpad for moving a cursor or for selecting content.

[0246] In some embodiments, such as Figure 5E-2a and Figure 5E-2b As shown, once the touchpad mode is activated, the device 100 also displays a ghost cursor 525 offset from the current position of the real cursor 522. In some embodiments, the ghost cursor 525 indicates the position where the cursor 522 will be after the lift-off of the finger contact. In some embodiments, the ghost cursor 525 is a modified version of the original cursor 522 displayed on the screen (for example, the ghost cursor 525 is gray, while the original cursor 522 as shown in Figure 5E-2 is black). In some embodiments, the ghost cursor 525 is linked to the real cursor 522 so that when the real cursor 522 moves around the screen, the ghost cursor 525 also moves around the screen. In some embodiments, when the user moves the real cursor 522 around the screen, the ghost cursor 525 shows the offset position closest to the real cursor 522 (for example, offset to the left or right). When the device 100 detects a lift-off event of the contact input 532, the real cursor 522 replaces the ghost cursor 525 at the position where the ghost cursor 525 was when the lift-off event occurred.

[0247] In some embodiments, after entering the touchpad mode (422), the device 100 detects (423) movement of the contact input. Figure 5E-3 As shown, device 100 detects movement 536 of contact input 532 on touch pad 523 .

[0248] In some embodiments, in response to detecting movement 536 of contact input 532, device 100 determines (424) whether contact input 532 satisfies one or more predetermined movement parameters. In some embodiments, the one or more predetermined movement parameters include a predetermined distance threshold, such as a number of inches of movement on a touch-sensitive display. In some embodiments, the one or more predetermined movement parameters include a predetermined movement speed threshold of the cursor, such as a movement speed of 5 pixels / second on a touch-sensitive display.

[0249] Using these movement parameters allows the device to decide when trackpad mode should be exited and when trackpad mode should be maintained, as discussed in more detail below.

[0250] In some embodiments, device 100 determines that the movement of contact input 532 does not satisfy one or more predetermined movement parameters (424—No, Figure 4 ). For example, Figure 5E-3 As shown, the device 100 detects that the movement 536 of the contact input 532 on the soft touch pad 523 corresponds to a movement of less than 1 inch on the touch-sensitive display and the speed of the movement is detected to be less than 5 pixels / second. Figure 5E-3 Movement 536 of middle contact input 532 does not satisfy predetermined movement parameters.

[0251] In some embodiments, device 100 also continues to determine (426) whether movement 536 of contact input 532 is a precise movement corresponding to the user's intention to precisely place the cursor at a specific location on content presentation area 502. In some embodiments, device 100 determines whether the movement speed of contact input 532 is equal to or less than a threshold, such as 5 pixels per second. If so, device 100 determines that movement 536 of contact input 532 is a precise movement (426—yes, Figure 4 ). For example, device 100 determines that the user intends to place the cursor precisely at a specific location (e.g., between the letters "e" and "d" in the word "created") through movement 536 of contact input 532 on soft touchpad 523. Figure 5E-3 As shown, in response to the finger movement 536 on the soft touch pad, the ghost cursor 525 and the actual cursor 522 move (428) 4 character positions to reach the expected position in the content presentation area 502.

[0252] In some embodiments and as Figure 5E-4 As shown, in response to Figure 5E-3536, the device 100 immediately exits (430) the trackpad mode to display the virtual keyboard 521. There is no delay in switching from the soft trackpad 523 to the soft keyboard 521 because the user has carefully placed the cursor and is unlikely to want to remain in the trackpad mode. In some embodiments, the actual cursor 522 remains at the location where the actual cursor 522 was when the lift-off event occurred and the ghost cursor 525 disappears.

[0253] In some embodiments and as Figure 5F-1 As shown, after device 100 detects that the duration of contact 538 is longer than an extended contact duration threshold (e.g., 0.25 seconds), contact 538 is determined to be an extended contact and the trackpad mode is activated. Device 100 determines whether contact input 538 is static or whether the movement of contact input 538 is a precise movement (e.g., whether the movement of the cursor caused by the contact input is less than a predetermined minimum movement threshold (e.g., 1 inch)). For example, Figure 5F-2 As shown, if no movement is detected, the device 100 determines that the contact input 538 does not have a precise moving portion (426—No, Figure 4 ). For example, a user may accidentally activate trackpad mode by pressing the space bar for an extended period of time without intending to use soft trackpad 523. In response, device 100 locks (432) actual cursor 522 and ghost cursor 525 without causing any movement in content presentation area 502.

[0254] In some embodiments, such as Figure 5F-3 As shown, in response to detecting the lift-off of contact input 538, device 100 immediately exits (434) trackpad mode to display virtual keyboard 521. There is no delay in switching from soft trackpad 523 to soft keyboard 521. In some embodiments, actual cursor 522 remains at the location where actual cursor 522 was when the lift-off event occurred and ghost cursor 525 disappears.

[0255] In a different Figure 5F-3 In an alternative embodiment, in response to detecting a lift-off event of static contact input 538, device 100 remains in trackpad mode (e.g., also referred to as trackpad stick mode) instead of exiting trackpad mode. Static contact input 538 can be used to activate trackpad stick mode so that the soft trackpad can remain for a certain period of time (e.g., 0.5 seconds, 1 second, or 2 seconds, etc.) after lift-off is detected.

[0256] In some embodiments and as Figure 5G-1As shown, after device 100 detects that the duration of contact 540 on the space bar is longer than a predetermined time threshold (eg, 0.25 seconds), contact 540 is determined to be an extended contact and the trackpad mode is activated.

[0257] In some embodiments and as Figure 5G-2 As shown, in response to displaying the touch panel 523 (for example, in the touch panel mode 422, Figure 4 ) after detecting (423) movement of contact input 540, device 100 determines (424) whether the movement of contact input 540 satisfies one or more predetermined movement parameters. In some embodiments, the one or more predetermined movement parameters include a predetermined movement distance threshold and / or a predetermined movement speed threshold, as discussed above. In some embodiments, the one or more predetermined movement parameters include a movement direction parameter that reflects whether the movement is moving in a downward direction.

[0258] For example, Figure 5G-2 As shown, the device 100 determines that the downward movement 542 of the contact input 540 on the space bar satisfies one or more predetermined movement parameters (424—Yes, Figure 4 ). Device 100 moves 436 the cursor in content presentation area 502, including actual cursor 522 and phantom cursor 525, in accordance with movement 542. Since the actual available screen space is running low as movement 542 proceeds, after liftoff of contact 540, the device remains in trackpad mode to give the user an opportunity to reposition their finger in a position that provides them with more space to continue moving the cursor.

[0259] For example, Figure 5G-3 As shown, the device 100 detects (438) a lift-off event of the contact input 540. Instead of exiting the touchpad mode, the device 100 remains (440) in the touchpad mode and maintains the soft touchpad 523. In some embodiments, both the actual cursor 522 and the ghost cursor 525 remain at their respective positions when the lift-off event is detected. In some embodiments, the device 100 continues to detect (442) whether there is a second contact within a predetermined time period after the lift-off is detected (e.g., within 0.5 seconds after the lift-off is detected).

[0260] In some embodiments, the device 100 remains in the touchpad mode for a predetermined period of time (e.g., also referred to as the touchpad stick mode). For purposes of illustration and not intended to be limiting, the present disclosure uses 0.5 seconds as the predetermined period of time. However, one of ordinary skill in the art will appreciate that such a predetermined period of time for remaining in the touchpad mode may be any suitable period of time, such as 1 second, 0.75 seconds, or 0.5 seconds. Such a predetermined period of time may also be pre-set by a manufacturer or designer and subsequently customized by a user using a settings interface on the device.

[0261] In some embodiments, device 100 continuously monitors whether a second contact input is received since the lift-off event was detected (438). Device 100 also tracks a period of time (e.g., elapsed time) since the lift-off event, such as Figure 5G-4 A timer in is visually shown, and the device remains in trackpad mode while monitoring for another contact.

[0262] In some embodiments and as Figure 5G-5 As shown, the device 100 is in a predetermined time period (eg, Figure 5G-5 That is, no second contact is detected during the 0.5 second duration after the lift-off event (442—No, Figure 4 ). At 0.5 seconds, the device 100 exits (444) trackpad mode to display the virtual keyboard 521. In some embodiments, the actual cursor 522 remains at the location where the actual cursor 522 was when the lift-off event occurred and the ghost cursor 525 is no longer displayed.

[0263] In some embodiments, such as Figure 5H-1 As shown, after the device 100 detects that the duration of contact 548 above the mode switching area (e.g., above the space bar) is longer than a predetermined time threshold (e.g., 0.25 seconds), contact 540 is determined to be an extended contact and the trackpad mode is activated. In some embodiments, the device 100 also provides tactile feedback 549 in response to the activation of the trackpad mode to remind the user that the trackpad mode has been entered. For example, once the trackpad mode is activated, the device 100 vibrates, or the device can provide local vibration close to the contact 548.

[0264] In some embodiments, such as Figure 5H-2 As shown, device 100 detects movement 550 of contact input 548. In some embodiments, based on determining that movement 550 satisfies the movement parameters (e.g., movement 550 is performed at a speed greater than a predetermined speed threshold and / or the distance moved during movement 550 is greater than a predetermined distance threshold), movement 550 is determined to satisfy the corresponding movement parameters (424—Yes, Figure 4 ). Device 100 moves ( 436 ) the cursor in content presentation area 502 , including actual cursor 522 and ghost cursor 525 , according to movement 550 .

[0265] In some embodiments, such as Figure 5H-3 As shown, the device 100 detects (438) a lift-off event of the contact input 548. The device 100 remains (440) in the trackpad mode, and the soft trackpad 523 remains on the display. Figure 5G-3 to Figure 5G-5 As discussed, the trackpad mode is maintained for a predetermined period of time, for example, 0.5 seconds, wherein a second contact is not detected on the display.

[0266] Figure 5I-1 It is also shown that, after determining that the movement of the contact input satisfies the corresponding movement parameter (424—Yes, Figure 4 ), the device remains in trackpad mode.

[0267] In some embodiments and as Figure 5I-2 As shown, if the device 100 detects a second contact input 544 (442—Yes) before the trackpad mode ends (e.g., within 0.5 seconds after the self-lift-off event), the device 100 may detect a second contact input 544. Figure 4 ), device 100 continues to determine (446) the touch type of second contact 544. In some embodiments, device 100 determines (456) contact input 544 as a single touch. In response, device 100 remains (458) in trackpad mode.

[0268] For example, Figure 5I-3 As shown, while the device is still in trackpad mode, device 100 detects movement 546 of contact input 544. In response, actual cursor 522 and phantom cursor 525 move in content presentation area 502 according to movement 546 of contact input 544 on soft trackpad 523. In some embodiments and as Figure 5I-4 As shown, upon detecting the lift-off of contact 544 from the touch-sensitive display, device 100 remains in trackpad mode. In some other embodiments, and as described above with reference to Figure 4 , Figure 5E-3 to Figure 5E-4 , Figure 5F-2 to Figure 5F-3 , Figure 5G-2 to Figure 5G-5 as well as Figure 5H-2 to Figure 5H-3 As discussed at step 424, when a lift-off of contact 544 is detected, device 100 can determine whether movement 546 of contact input 544 satisfies one or more movement parameters and then execute a function accordingly. In yet other embodiments (not shown), when a lift-off event is detected after a single touch 544, device 100 immediately exits trackpad mode.

[0269] Figure 5J-1 Also starting with the device remaining in trackpad mode (e.g., the mode may have been activated using an extended contact and movement of that contact satisfies the corresponding movement parameters). Figure 5J-2 As shown, if the device 100 detects a second contact input 556-1 (442—Yes) before the trackpad mode ends (e.g., within 0.5 seconds after the self-lift-off event), the device 100 may detect a second contact input 556-1 (442—Yes). Figure 4 ), device 100 continues to determine (446) the touch type of second contact 556-1. In some embodiments, device 100 detects (460) a quick double-click on the touch screen to trigger (462) text selection.

[0270] For example, in Figure 5J-2 , device 100 may detect that the duration of contact 556-1 is shorter than a predetermined threshold (e.g., 0.05 seconds), followed by a lift-off event of contact 556-1, such as Figure 5J-3 After the lift-off event, if device 100 detects a second contact 556-2 within a predetermined threshold (e.g., 0.05 seconds), Figure 5J-4 As shown, device 100 determines that a quick double-click has been received on the touch-sensitive display. In response, Figure 5J-4 As shown, the device 100 selects a portion of the content 500 (e.g., the word at the cursor position) and displays the selection 562. For example, the word "Liberty" is highlighted when selected. In some embodiments, the device 100 displays a start point object 560 and an end point object 558 at the respective ends of the mark such as "Liberty" to indicate the selection 562.

[0271] In some embodiments, such as Figure 5J-5 As shown, when device 100 detects a lift-off event after a double-click, device 100 remains in trackpad mode and selection 562 remains on the display. In some other embodiments (not shown), when device 100 detects a lift-off event after a double-click, device 100 immediately exits trackpad mode and selection 562 remains on the display.

[0272] Figure 5K-1 A device is shown operating in a trackpad mode (e.g., the mode may have been activated using an extended contact, and movement of the contact satisfies corresponding movement parameters). Figure 5K-2 to Figure 5K-4 As shown, device 100 detects a quick double click (e.g., Figure 5K-2 Tap 570-1 in the Figure 5K-3 The lift-off in the last Figure 5K-4 In response to the tap 570-2 in Figure 5K-4After the quick double-click text selection 562 in , the device 100 may also detect ( 464 ) a hold and drag gesture 574 of contact 570 - 2 , such as Figure 5K-5 As shown. In response, Figure 5K-5 As shown, device 100 selects ( 466 ) more objects (eg, additional words in area 502 ) based on movement 574 on the trackpad.

[0273] In some embodiments, such as Figure 5K-6 As shown, when the device 100 detects a lift-off event after the double-click and hold and drag gesture, the device 100 remains in trackpad mode and the selection 562 of more text remains on the touch-sensitive display. In some other embodiments (not shown), when the device 100 detects a lift-off event after the double-click and hold and drag event, the device 100 immediately exits trackpad mode but maintains the selection 562 on the display.

[0274] 6A to 6F 6 is a flow chart illustrating a method 600 for activating a touchpad mode using an extended contact above a virtual keyboard, according to some embodiments. The method 600 is performed in a computer having a touch-sensitive display (e.g., Figure 1A A portable multifunction device (e.g., a touch screen display 112 in Figure 1A 100). As described below, method 600 provides a mechanism for activating a trackpad mode at a portable multifunction device with a touch-sensitive display. The method does not require the portable multifunction device to include a contact intensity sensor (e.g., a force sensor or a pressure sensor), thereby reducing the cost of designing and manufacturing the portable multifunction device. The method is also intuitive to the user, thereby reducing the amount, degree and / or nature of input from the user when the trackpad mode is activated, and creating a more efficient human-computer interface. A more efficient input mechanism also requires fewer computing resources, thereby increasing the battery life of the device.

[0275] Go to Fig. 6A, method 600 is performed at an electronic device including a touch-sensitive display coupled to a plurality of touch sensors (602). In some embodiments, the touch-sensitive display is not coupled to any force sensor. The electronic device may not be able to directly detect changes in contact intensity because the device does not have any force sensors. In some other embodiments, the touch-sensitive display is coupled to one or more force sensors, but the detection of single-finger gestures as discussed above does not require the use of any force signals that may be generated by such force sensors. In other words, even if the device includes a force sensor, such a force sensor is not required to detect the single-finger touchpad activation gestures discussed herein, such as a tip-to-texture rolling gesture and an extended contact over a predetermined mode switching area of ​​a virtual keyboard. In some embodiments, a touch sensor among the plurality of touch sensors is a capacitive touch sensor.

[0276] In performing method 600, the electronic device displays (604) a virtual keyboard on the touch-sensitive display, the virtual keyboard including display representations of a space bar and other keys. Figure 5A-1 As shown, the displayed virtual keyboard 521 includes a displayed representation of the space bar (in the bottom row of the virtual keyboard 521) as well as displayed representations of numerous other keys.

[0277] The electronic device also detects (606) a first extended contact above the virtual keyboard. An extended contact is also referred to herein as a "long press" and includes a contact above the touch-sensitive display that lasts at least an extended contact detection threshold (e.g., 0.25 seconds). An example of a first extended contact is shown in FIG. Figure 5B-1 514, which is a contact that lasts for 0.25 seconds (eg, Figure 5B-2 As shown, after detecting that the extended contact lasts for 0.25 seconds, the corresponding extended keyboard function of the "e" keyboard is Figure 5B-2 is activated in the

[0278] Based on determining that the first extended contact is over any corresponding displayed representation of other keys (e.g., Figure 5B-1 and Figure 5B-2 ), the electronic device interprets (608) the first extended contact as a keyboard input and activates a corresponding extended keyboard function (e.g., Figure 5B-2In some embodiments, the corresponding extended keyboard function is activated only when it is detected that the first extended contact has lasted for at least an extended contact duration threshold (e.g., 0.25 seconds). Some other keys may be associated with only one keyboard function, or may be associated with both a keyboard function (e.g., a function activated based on a momentary contact) and an extended keyboard function (e.g., a function activated based on an extended contact). In the case where the first extended contact is over a key that is associated with only one keyboard function, the keyboard function is both the keyboard function of the key and the extended keyboard function (e.g., Figure 5C-1 As shown, an extended contact 518 over the "q" key for 0.25 seconds still activates the "q" key because the keyboard function of "q" is the same as the extended keyboard function).

[0279] Next, the electronic device detects (610) a second extended contact (eg, an extended contact different from the first extended contact, such as Figure 5E-1 According to determining that the second extended contact is above the corresponding display representation of the space bar (such as Figure 5E-1 The device interprets (612) the second extended contact as a mode switching input that causes the electronic device to operate at least a portion of the touch-sensitive display as a touch pad. For example, Figure 5E-2a As shown, after the extended contact 532 has been maintained above the space bar for at least the extended contact duration threshold, the device operates at least a portion of the touch-sensitive display as a touchpad. In some embodiments, before the second extended contact is lifted off the touch-sensitive display, the electronic device operates at least a portion of the touch-sensitive display as a touchpad, which allows the user to activate the touchpad and immediately begin using the touchpad in a fluid motion. In some embodiments, a portion of the touch-sensitive display is the entirety of the touch-sensitive display.

[0280] On devices that do not include force sensors, designing a single-finger gesture that allows switching to a trackpad mode while also avoiding unnecessary confusion with other gestures is a very difficult task. Allowing a user to provide an extended contact over the space bar (or over another designated area of ​​a virtual keyboard, such as a bottom portion of a virtual keyboard that does not include keys) and then operate the electronic device as a trackpad provides a convenient way for the user to quickly switch from a keyboard input mode to a trackpad mode. Providing this interaction at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more efficient (e.g., by allowing the user to easily switch to a trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate a trackpad mode when the trackpad mode is not in use, or without having to waste time using a complex sequence of user inputs or menu options to activate a trackpad mode).

[0281] Now go to Figure 6B And continuing from operation 612, while operating a portion of the touch-sensitive display as a touchpad, the device detects (614) a lift-off of the second extended contact from the touch-sensitive display. Examples of such lift-off are shown in the following figures: (1) Figure 5E-4 As shown, the extended contact 532 has been lifted off the touch-sensitive display; (2) as shown Figure 5F-3 As shown, the extended contact 538 has been lifted off the touch-sensitive display; (3) as shown Figure 5G-3 As shown, the extended contact 540 has been lifted off the touch-sensitive display; and (4) as shown Figure 5H-3 As shown, extended contact 548 has been lifted off the touch-sensitive display.

[0282] In response to detecting lift-off of the second extended contact, the device optionally determines ( 614 ) whether any movement of the second extended contact on the touch-sensitive display before lift-off satisfies movement parameters.

[0283] Using the movement parameters can allow the electronic device to accurately determine when to continue operating in the trackpad mode or when to exit the trackpad mode. By determining whether any movement of the second extended contact on the touch-sensitive display satisfies the movement parameters in response to detecting a lift-off, the device reserves computing resources and uses these computing resources to make the determination only when the determination is immediately needed. In this way, the operability of these devices is enhanced and the human-machine interface on these devices operates more efficiently (e.g., by avoiding wasting resources by making a determination before the determination is needed).

[0284] In some embodiments, the device determines (616) that movement of the second extended contact on the touch-sensitive display before lift-off does not satisfy the movement parameters, and in response, stops (616) operating a portion of the touch-sensitive display as a touchpad. An example of this behavior is Figure 5E-3 to Figure 5E-4 (slow movement before lift-off, such as slower than 5 pixels / second) and Figure 5F-2 to Figure 5F-3 (No movement or very small movement before lift-off, such as movement of less than 1 inch). It may stop immediately, such as within 0.01 seconds after lift-off is detected. In some embodiments, the device also redisplays the virtual keyboard and operates the area displaying the virtual keyboard in keyboard input mode again.

[0285] Once it is determined that the movement before the lift-off does not satisfy the movement parameters, the portion of the touch-sensitive display is stopped from being operated as a trackpad, which ensures that the trackpad mode is invoked only when the user needs it. By ensuring that the trackpad mode is invoked only when the user needs it, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (e.g., by ensuring that the user is presented with the interface controls they need and avoiding wasting time trying to find the interface controls they need).

[0286] In some embodiments, determining that the movement of the second extended contact on the touch-sensitive display before lift-off does not satisfy the movement parameters includes determining (618) that the distance moved by the second extended contact on the touch-sensitive display before lift-off is less than a predetermined distance. Figure 5F-2 ) does not move at all, indicating that the trackpad mode was accidentally invoked, and therefore, the user will prefer to switch to the keyboard input mode immediately. In some embodiments, the device also redisplays the virtual keyboard and operates the area displaying the virtual keyboard (such as Figure 5F-3 shown).

[0287] Once it is determined that the distance moved by the second extended contact on the touch-sensitive display before lifting is less than a predetermined distance, the portion of the touch-sensitive display is stopped from being operated as a touchpad, which ensures that the touchpad mode is invoked only when the user needs the touchpad mode. If the user moves the second extended contact a very short distance (or not at all), this indicates that the user has placed the cursor at the desired location and no longer needs to use the touchpad mode (or when the contact does not move at all, this indicates that the user accidentally invoked the touchpad mode). By ensuring that the touchpad mode is invoked only when the user needs the touchpad mode, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (for example, by ensuring that the interface controls that the user needs are presented to the user and avoiding wasting time on the user trying to find the interface controls that the user needs).

[0288] In some embodiments, determining that movement of the second extended contact on the touch-sensitive display before lift-off does not satisfy movement parameters includes determining (620) that the second extended contact moved on the touch-sensitive display before lift-off at a speed less than a predetermined speed (e.g., a speed less than 5 pixels per second or any suitable speed indicating that the user is delicately placing the cursor). Such movement of the second extended contact can indicate that the user is moving the cursor to a particular point and no longer needs the touchpad after they have placed the cursor at the desired location. Figure 5E-3 An example is shown in which contact 532 moves at a low rate 536 before lifting off (which allows the cursor to be finely placed within the text appearing in area 502), and Figure 5E-4 , exited touchpad mode.

[0289] Once it is determined that the speed at which the second extended contact moves on the touch-sensitive display before lifting is less than a predetermined speed, the portion of the touch-sensitive display is stopped from being operated as a touchpad, which ensures that the touchpad mode is invoked only when the user needs the touchpad mode. If the user moves the second extended contact at a slow speed, this indicates that the user has placed the cursor at the desired position and no longer needs to use the touchpad mode. By ensuring that the touchpad mode is invoked only when the user needs the touchpad mode, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (for example, by ensuring that the interface controls that the user needs are presented to the user and avoiding wasting time on the user trying to find the interface controls that the user needs).

[0290] The device can also determine (622) that movement of the second extended contact on the touch-sensitive display before lift-off satisfies movement parameters (e.g., after invoking the trackpad and before lift-off, the second extended contact moves quickly on the display, such as Figure 5H-2 ), and in response, the device continues to operate a portion of the touch-sensitive display as a touchpad for at least a predetermined amount of time (e.g., for at least 5 seconds after detecting lift-off of the second extended contact from the touch-sensitive display).

[0291] Once it is determined that the second extended contact satisfies the movement parameters, a portion of the touch-sensitive display continues to be operated as a trackpad, which ensures that the trackpad mode is invoked whenever the user needs it. If the user moves the second extended contact in a downward direction or at a faster speed, this indicates to the user that the lift-off of the second extended contact is completed in order to allow the user to reposition their finger to continue moving the cursor (e.g., if the user moves in a downward direction, they may run out of space and will need to lift off to reposition their finger to continue moving the cursor). By ensuring that the trackpad mode is invoked whenever the user needs it and without interrupting the user's movement of the cursor, the operability of the device is enhanced and the human-machine interface on the device operates more efficiently (e.g., by ensuring that the user is presented with the interface controls they need and avoiding wasting time on the user trying to find the interface controls they need).

[0292] Reference now Figure 6C (From Fig. 6A), after detecting lift-off of the second extended contact from the touch-sensitive display, the electronic device continues (624) to operate a portion of the touch-sensitive display as a touchpad for at least a predetermined amount of time. In some embodiments, the touchpad remains adhered (e.g., remains in an operational mode) even after lift-off of the second extended contact is detected, and movement parameters are not used to determine whether the touchpad should remain adhered. In some embodiments, the predetermined time period is equal to or less than 1 second, equal to or less than 0.75 seconds, or equal to or less than 0.5 seconds.

[0293] In some embodiments, the trackpad is invoked (626) after the second extended contact has remained in contact with the touch-sensitive display for at least a second predetermined amount of time; and the second predetermined amount of time is less than the predetermined amount of time. For example, the second predetermined amount of time is the extended contact detection time threshold discussed herein, such as 0.25 seconds. Then in some embodiments, the trackpad thus remains adhered (in other words, continues to operate a portion of the display as a trackpad for more than 0.25 seconds) even after lift-off is detected, which allows the user to reposition their finger without having to reactivate the trackpad mode again.

[0294] In some embodiments, the electronic device ceases to operate a portion of the touch-sensitive display as a touchpad based on determining that no input is detected over a portion of the touch-sensitive display within a predetermined amount of time after detecting lift-off of the second extended contact. Figure 5G-5 As shown in the figure, in which the self-detection Figure 5G-2 After a predetermined amount of time (e.g., 5 seconds) has passed after lift-off of extended contact 540, ceasing to operate a portion of the touch-sensitive display as a touchpad.

[0295] Figure 6COperation 630 described in may be after operations 624 / 626, or may be after operation 614. In some embodiments, when a portion of the touch-sensitive display is operated as a touchpad, the electronic device detects (630) a double-click input on the touch-sensitive display, the double-click input is detected within a certain amount of time (e.g., 0.25 seconds or less) after the lift-off of the second extended contact is detected; and, in response to detecting the double-click input, the device selects an object displayed on the touch-sensitive display. For example, the object is positioned near a cursor that is displayed in a content area of ​​the touch-sensitive display (on either side of, under, or adjacent to the cursor), the content area being outside the area on the touch-sensitive display for displaying a virtual keyboard. Selecting the object may include moving the cursor within the content area of ​​the touch-sensitive display to provide a visual indication that the object has been selected. The object may be a word (or some other collection of alphanumeric characters and / or symbols), an image, or a combination of text and an image. Exemplary double-click input in Figure 5J-2 to Figure 5J-4 , where a first tap 556-1 ( Figure 5J-2 ), followed by the liftoff of the tap ( Figure 5J-3 ), and finally the second tap 556-2 ( Figure 5J-4 ). In response to detecting the double click, the device selects an object displayed in area 502, for example, in Figure 5J-5 , select the word "Liberty".

[0296] Allowing the user to perform a double-tap input to select an object ensures that when in trackpad mode, the user is able to select an object (e.g., text, an image, or a combination of both). The operability of the device is enhanced by ensuring that the trackpad mode allows the selection of an object, and then creates a continuous interaction with the device.

[0297] In some embodiments, after the double-click input, the device also detects a drag gesture (e.g., gesture 574, Figure 5K-5 ), and in response, as the drag gesture moves, the device selects more objects (e.g., more displayed text within selection area 502). In other words, the user is able to perform a drag gesture immediately after the double-tap input selection gesture to adjust the number of objects to be selected. Allowing the user to perform a drag gesture after the double-tap input to select more (or fewer) objects ensures that when in trackpad mode, the user is able to select only those portions of the object that the user wants to select (e.g., a portion of text, an image, or a combination of both). The operability of the device is enhanced by ensuring that the trackpad mode allows for fine-grained selection of objects, which then creates a continuous interaction with the device.

[0298] Reference now Fig.6D, the device detects (634) a third extended contact above a portion of the virtual keyboard below the displayed representation of the space bar, the portion of the virtual keyboard not including any other keys (the third extended contact may be received before the second extended contact or after the second extended contact and represents another way to invoke a trackpad). Based on determining that the third extended contact is above a portion of the virtual keyboard below the displayed representation of the space bar and not used to display any representations of other keys, the device interprets the third extended contact as (636) a mode switching input that causes the electronic device to operate at least a portion of the touch-sensitive display as a trackpad. In some embodiments, a gray space portion of the virtual keyboard may also be used to invoke a trackpad mode, such as an area below the space bar (in other words, this is an alternative or supplement to using the displayed representation of the space bar as a mode switching area, and another area such as the area below the space bar may alternatively or additionally be used as a mode switching area). In one example, when the virtual keyboard is in emoji mode, then an extended contact at the bottom below a separate emoji key on the virtual keyboard may be interpreted as a mode switching input.

[0299] Move to Fig. 6E , any extended contact (or momentary contact) over the corresponding display representation of the other keys causes (638) the activation of the corresponding extended keyboard function and does not cause the electronic device to operate any portion of the touch-sensitive display as a touchpad. At least some of the other keys of the virtual keyboard may also be associated (640) with corresponding keyboard functions activated in response to a momentary contact over the key and corresponding extended keyboard functions activated in response to an extended contact over the key, respectively. For example, Figure 5B-2 As shown, an extended touch above the "e" key activates the extended keyboard function of the "e" key (display of accent input options), and a momentary touch above the "e" key ( Figure 5A-1 ) causes the default keyboard function to be activated, such as registering the actuation of the "e" key. For some of the other keys, the corresponding extended keyboard function and the corresponding default keyboard function may be the same, such as for Figure 5C-1 to Figure 5C-2 The "q" key is concerned.

[0300] In some embodiments, when the virtual keyboard is displayed, and before it is operated as a touchpad, the area operates in a keyboard input mode, wherein user input at any other key of the virtual keyboard is interpreted as a keyboard input that activates a keyboard function or extends a keyboard function, depending on the type of input received (momentary contact or extended contact). The area remains in keyboard input mode both before and after the first extended contact. Thus, only an extended contact above a pre-specified mode switch area will cause the touchpad to activate, such as an extended contact above the space bar or an extended contact above the area below the space bar.

[0301] The same area of ​​the display used for the virtual keyboard is operated as a trackpad, which allows the user to immediately begin using the trackpad without having to reposition / reposition their fingers after invoking the trackpad, instead, the user simply begins moving their fingers as needed to move the cursor and / or select objects (e.g., using the double-click gesture and double-click and drag gesture discussed above). In addition, ceasing to display at least some of the display symbols of the virtual keyboard provides the user with a clear visual indication that the trackpad mode has been invoked. By providing these features, the operability of the device is enhanced, and then a continuous interaction with the device is created (e.g., the user does not need to waste time repositioning his fingers, and will not be confused about when to invoke the trackpad, thereby allowing continuous interaction with the device).

[0302] In some embodiments, the displayed virtual keyboard does not include (642) function (fn) keys, such as Figure 5D-1 shown.

[0303] Now go to Fig. 6F , a virtual keyboard is displayed (644) in an area of ​​the touch-sensitive display, and each displayed representation of the other keys includes a corresponding displayed symbol, and operating at least a portion of the touch-sensitive display as a touchpad includes (i) operating the area as part of a touchpad, and (ii) ceasing to display at least some of the displayed symbols of the virtual keyboard. In other words, when operating as a touchpad, an area of ​​the touch-sensitive display that was previously used to display the virtual keyboard is now operated as a touchpad (e.g., Figure 5E-2a As shown, which was previously used to display Figure 5E-1 In some embodiments, the device stops displaying all symbols on the virtual keyboard and may replace the displayed virtual keyboard with an image representing a touchpad. Figure 5E-2b In some embodiments, the device may still show key borders but not keyboard symbols when operating a portion of the touch-sensitive display as a touchpad ( Figure 5E-2a ), and in this way, the device provides a clear visual indication to the user that the area is now being operated as a touchpad.

[0304] Additionally, when operating a portion of the display as a touchpad, any input on at least the portion of the touch-sensitive display controls (646) movement of a cursor displayed in another portion of the touch-sensitive display. For example, when operating as a touchpad, a swipe gesture moved on the touch-sensitive display causes a cursor displayed on the touch-sensitive display to move according to the movement of the swipe gesture.

[0305] In some embodiments, tactile feedback is also provided in conjunction with invoking the operation of a portion of the touch-sensitive display as a trackpad (648). For example, upon detecting an extended contact above the displayed representation of the space bar that lasts for at least an extended contact duration threshold (e.g., 0.25 seconds), the device switches to operating in trackpad mode and also provides tactile / tactile feedback (648). Figure 5H-1 ) to indicate the mode has switched.

[0306] Although a device without any force sensors is used as the primary illustrative example above, the extended contact for activating a trackpad mode in method 600 may also be implemented on a device that does include a force sensor, but detecting these single-finger gestures does not require the use of any force signals that may be generated by such a force sensor (in other words, even if the device does have a force sensor, such a force sensor is not required to detect a single-finger trackpad activation gesture, such as an extended contact over a predetermined mode switch region of the virtual keyboard (examples of mode switch regions include the space bar or an area on the virtual keyboard below the space bar)). In addition, although an extended contact over a displayed representation of the space bar is used as the primary illustrative example of a trackpad activation gesture above, an extended contact may also be received over another mode switch region, such as an area on the virtual keyboard below the displayed representation of the space bar and an area on the virtual keyboard that does not include any displayed representation of a key (e.g., an empty space on the virtual keyboard that does not include any key).

[0307] It should be understood that 6A to 6F The specific order in which the operations have been described is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art will think of many ways to reorder the operations described herein and / or compare the operations with the reference operations. Figure 4 , Figure 7 as well as 9A to 9D The operations described can be combined in various ways.

[0308] Figure 7 700 is a flow chart illustrating a method 700 for activating a touchpad mode using one or more single-finger gestures including a tip-to-texture scroll gesture, according to some embodiments. The method 700 is performed on a touch-sensitive display (e.g., Figure 1A A portable multifunction device (e.g., a touch screen display 112 in Figure 1A100). As described below, method 700 provides a mechanism for activating a touchpad mode (e.g., from a keyboard mode) at a portable multifunction device with a touch-sensitive display. The method does not require the portable multifunction device to include a contact intensity sensor (e.g., a force sensor or a pressure sensor), thereby reducing the cost of designing and manufacturing the portable multifunction device. That is, in some embodiments, the method can still be used on a device with a force sensor. The method is also intuitive to the user, thereby reducing the amount, degree, and / or nature of input from the user when activating the touchpad mode, and creating a more efficient human-computer interface. A more efficient input mechanism also requires fewer computing resources, thereby increasing the battery life of the device.

[0309] Figures 8A-1 to 8F An example of a user interface that activates a trackpad mode using a finger gesture including a tip-to-groove scrolling gesture is shown according to some embodiments. Figures 8A-1 to 8F The user interface in the Figure 7 as well as 9A to 9D The process described.

[0310] exist Figure 8A-1 In the embodiment of the present invention, when an application (e.g., notepad application 153 or message application 141) is in document editing mode, editable content 800 is displayed in content presentation area 802. In some embodiments, content 800 in content presentation area 802 includes text (e.g., plain text, unstructured text, formatted text, or text in a web page). In other embodiments, content includes graphics with or without text. In addition, content can be editable or read-only. In addition to displaying content, when content is not selected, device 100 can display a cursor (e.g., cursor 822) within the electronic document, for example for text input. In addition, a virtual keyboard 821 is displayed for text input.

[0311] In some embodiments, device 100 has a capacitive sensor array (not shown, but described above with respect to Figure 1A The device 100 may include a capacitive sensor array that generates an output signal that is used to measure the proximity, position, displacement, and / or acceleration of one or more contacts with the touch-sensitive display. In some embodiments, the device 100 continuously monitors the touch input on the touch screen 112 and the continuous movement of the touch input.

[0312] In some embodiments, when displaying the content of the electronic document, the device 100 detects changes in the output signal of the sensor array. In some embodiments, the device 100 determines the centroid of the contact input based on the signal registered to one or more sensors (e.g., capacitive electrodes) of the sensor array. The centroid is used to determine the position of the corresponding contact input on the touch screen. In some embodiments, the centroid is the geometric center of the contact area. In some embodiments, the centroid is the geometric center of the area on the touch-sensitive surface that is contacted or touched by the user (e.g., an area with a measured capacitance greater than a predetermined threshold). In some embodiments, the centroid is determined at the position corresponding to the one or more sensors in the sensor array that output the strongest signal (e.g., the geometric center of the one or more sensors that output the strongest signal).

[0313] In some embodiments, device 100 determines what type of finger gesture is made on the touch-sensitive display by detecting one or more parameters selected from: (1) a change in the contact area, (2) a change in the position of the center of mass, (3) a change in the speed of movement of the center of mass, (4) a change in the saturation level of at least one touch sensor in a plurality of touch sensors, and (5) a change in the angle of the user's finger making contact relative to the touch-sensitive display. For example, these situations may occur when a user rolls his finger across the touch-sensitive surface from the tip of the user's finger to a flat portion of the user's finger (e.g., a textured portion or where a user's fingerprint is located).

[0314] refer to Figure 7 and Figure 8A-1 In some embodiments, device 100 first detects contact 830-1 on keyboard 812. Based on the output signals generated by the sensor array of the device in response to contact 830-1 detected on the sensor array, device 100 can determine that contact 830-1 corresponds to contact between the user's fingertip ("tip") and the touch screen.

[0315] In some embodiments, such as Figure 8A-2 As shown, the output signal of contact 830-2 may indicate the contact area compared to Figure 8A-1 830-1 has grown, and as the user rolls their finger further or presses harder, the location of the contact center of mass has moved downward. In some embodiments, the change in the contact area and the location of the contact center of mass is continuous from contact 830-1 to contact 830-2, i.e., there is no interruption in the contact (e.g., a lift-off event). In some embodiments, as Figure 8A-3Further monitoring, the output signal of contact 830-3 indicates that the contact area continues to increase, and as the user further rolls his finger or presses harder, the contact center continues to move downward. Device 100 can determine that contact 830-3 corresponds to contact between the fingerprint portion ("texture portion") of the user's finger and the touch screen.

[0316] In some embodiments, device 100 determines that a tip-to-texture scroll gesture has been detected (702), from Figure 8A-1 to Figure 8A-3 In response to detecting a scroll gesture from the tip portion to the texture portion, as shown in FIG. Figure 8A-3 As shown, the device 100 enters (704) the touchpad mode. For example, the touchpad 823 replaces the keyboard 821, wherein the touchpad 823 does not display the alphanumeric characters on the keys. In some embodiments, the touchpad 823 shows a border of each individual virtual key without showing the alphanumeric characters. In some other embodiments, such as Figure 5E-2b As shown, the touchpad removes the borders of the virtual keys on the virtual keyboard to provide a visual indication that the virtual keyboard has changed to the touchpad mode. In some embodiments, once the soft touchpad mode is activated, the device 100 also displays a ghost cursor 825 offset from the current position of the real cursor 822, such as Figure 8A-3 shown.

[0317] In some embodiments, device 100 detects (705) a second gesture (e.g., a move, a lift-off event, or another scrolling gesture, such as a reverse scrolling gesture). Device 100 continues to determine (706) a gesture type of the second gesture subsequent to the tip-to-texture scrolling gesture detected at step 702.

[0318] In some embodiments, device 100 detects that textured portion contact 830-3 moves on touch pad 823 (712). Figure 8A-4 As shown, based on the output signal of contact 830-3, device 100 can determine the curved movement path 834 of the center of mass of contact 830-3. Therefore, device 100 determines that textured contact 830-3 moves along the same curved path (834). In response, the cursor moves along the same curved path in content presentation area 802 (714). In some embodiments, device 100 continues to detect (715) the next gesture (e.g., continued movement of the contact, a lift-off event, or another scrolling gesture, etc.). The device then determines (706) between the fingerprint contact and the movement of the cursor on the touch screen (e.g., Figure 8A-4 The gesture type of the next gesture after movement 834 in.

[0319] In some embodiments, such as Figure 8BAs shown, the device 100 detects no output signal on the sensor array, and the device 100 determines (708) that a finger lift-off event of the textured portion contact 830-3 has occurred. In response, the device 100 exits (710) the trackpad mode, and the keyboard 821 replaces the trackpad 823 on the touch screen.

[0320] In some embodiments, such as Figure 8C-1 As shown, based on the output signal of the device in response to the texture contact 830-3, the texture contact is in the tip to texture scroll gesture (for example, as shown in reference Figure 8A-1 to Figure 8A-4 After the touch panel 823 is touched (as discussed above) and without any lift-off event occurring, the device 100 determines (705) that the texture contact 830-3 remains on the touch panel 823. In some embodiments, as Figure 8C-2 As shown, the device responds to the contact output signal indicating the contact area compared to Figure 8C-1 Contact 830-3 in has decreased, and as the user further rolls their finger in the opposite direction (from fingerprint to fingertip) or presses more lightly, the location of the contact center of mass has moved upward.

[0321] In some embodiments, such as Figure 8C-3 Further monitoring, the output signal of contact 830-3 indicates that the contact area continues to decrease, and as the user further rolls his finger in the opposite direction or presses the touch screen more lightly, the contact center of mass continues to move upward. Device 100 can determine that contact 830-1 corresponds to contact between the fingertip portion ("tip") of the user's finger and the touch screen.

[0322] In some embodiments, upon detecting the Figure 8C-1 to Figure 8C-3 After the change in one or more parameters (e.g., area size, velocity, center of mass movement, etc.) of the user's finger contact on the touch screen in question, device 100 determines (716) that a reverse scroll gesture from the texture portion to the tip portion has been detected. In response, as Figure 8C-3 As shown, device 100 maintains (718) the trackpad mode. Then, device 100 detects (719) a third gesture (e.g., a lift-off event, a movement of the contact, or another scroll gesture, such as a re-scroll gesture). Device 100 continues to determine (720) the gesture type of the third gesture.

[0323] In some embodiments, device 100 detects fingertip contact 830-1 moving on touchpad 823 (726). For example, Figure 8C-4As shown, based on the output signal of the device in response to contact 830-1, device 100 detects (719) the curved movement path 835 of the center of mass of contact 830-1. Therefore, device 100 determines that the fingertip contact 830-1 moves along the same curved path (835). In response, the cursor moves along the same curved path in content presentation area 802 (728). In some embodiments, device 100 continues to detect (729) the next gesture (e.g., continued movement of the contact, a lift-off event, or another scrolling gesture, etc.). The device then determines (720) the movement of the fingertip contact on the touch screen (e.g., Figure 8C-4 The gesture type of the next gesture after movement 835 in .

[0324] In some embodiments, such as Fig.8D As shown, the device 100 detects no output signal on the sensor array, and therefore determines (722) that a finger lift-off event of the textured portion contact 830-1 has occurred. In response, the device 100 exits (724) the trackpad mode, and the keyboard 821 replaces the trackpad 823 on the touch screen.

[0325] In some embodiments, such as Figure 8E-1 As shown, based on the output signal of the device in response to the fingertip contact 830-1, the fingertip contact is in the reverse scroll gesture from the texture portion to the tip portion (for example, as shown in reference Figure 8C-1 to Figure 8C-4 ) and without any lift-off event occurring, device 100 determines (719) that fingertip contact 830-1 remains on trackpad 823. In some embodiments, as Figure 8E-2 As shown, the device responds to the contact output signal indicating the contact area compared to Figure 8E-1 Contact 830-1 in has grown, and as the user rolls their finger further (from fingertip to fingerprint) or presses harder, the location of the contact's center of mass has moved downward.

[0326] In some embodiments, such as Figure 8E-3 Further monitoring, the output signal of contact 830-3 indicates that the contact area continues to increase, and as the user further rolls his finger or presses the touch screen harder, the contact center continues to move downward. Device 100 can determine that contact 830-3 corresponds to contact between the fingerprint portion ("texture portion") of the user's finger and the touch screen.

[0327] In some embodiments, upon detecting the Figure 8E-1 to Figure 8E-3 After the change in one or more parameters (e.g., area size, velocity, center of mass movement, etc.) of the user's finger contact on the touch screen in question, device 100 determines (730) that a tip-to-texture re-roll gesture has been detected. In response, Figure 8E-3As shown, device 100 selects (732) one or more objects (eg, text, images, etc.) on the touch screen. In some embodiments, device 100 selects an object associated with the location of the cursor on the touch screen.

[0328] Then, device 100 detects (733) a fourth gesture (eg, a lift-off event, movement of the contact, or another scroll gesture, etc.) Device 100 proceeds to determine (734) a type of the fourth gesture.

[0329] In some embodiments, device 100 detects fingerprint contact 830-3 moving on touchpad 823 (740). Figure 8E-4 As shown, based on the output signal of the device in response to contact 830-3, device 100 can determine the curved movement path 860 of the center of mass of contact 830-3. In some embodiments, as shown in FIG. Figure 8E-4 As shown, device 100 selects (742) more objects displayed in content presentation area 802. For example, Figure 8E-4 As shown, the selected further text is positioned proximate to the area through which the curved path 860 passes in the content presentation area 802 .

[0330] In some embodiments, the device 100 continues to detect (743) the next gesture (e.g., continued movement of the contact, a lift-off event, or another scrolling gesture, etc.). The device then determines (734) the movement of the fingerprint contact on the touch screen (e.g., Figure 8E-4 The gesture type of the next gesture after movement 860 in .

[0331] In some embodiments, such as Figure 8F As shown, the device 100 detects no output signal on the sensor array, and therefore determines (736) that a finger lift-off event of the textured portion contact 830-3 has occurred. In response, the device 100 exits (738) the touchpad mode, and the keyboard 821 replaces the touchpad 823 on the touchscreen.

[0332] 9A to 9D 900 is a flow chart illustrating a method 900 for activating a touchpad mode using a single-finger scroll gesture according to some embodiments. The method 900 is performed in a system having a touch screen display (e.g., Figure 1A A portable multifunction device (e.g., a touch screen display 112 in Figure 1A100). As described below, method 900 provides a mechanism for activating a trackpad mode at a portable multifunction device with a touch screen display. The method does not require the portable multifunction device to include a contact intensity sensor (e.g., a force sensor or a pressure sensor), thereby reducing the cost of designing and manufacturing the portable multifunction device. The method is also intuitive to the user, thereby reducing the amount, degree and / or nature of input from the user when the trackpad mode is activated, and creating a more efficient human-computer interface. A more efficient input mechanism also requires fewer computing resources, thereby increasing the battery life of the device.

[0333] like Fig.9A As shown, method 900 is performed at an electronic device including a touch-sensitive display coupled to a plurality of touch sensors, wherein the touch-sensitive display is not coupled to any force sensor. In other words, the electronic device cannot directly detect changes in contact intensity because the device does not have any force sensors. In some embodiments, the touch sensors in the plurality of touch sensors are capacitive touch sensors. Likewise, in some embodiments, force sensors are present in the device but are not used to perform method 900.

[0334] During the execution of method 900, the device displays (904) a virtual keyboard on a touch-sensitive display (e.g., Figure 8A-1 The device also detects (906) output from at least one touch sensor of the plurality of touch sensors, the output indicating contact with the touch-sensitive display over at least a portion of the virtual keyboard, wherein the contact includes an area on the touch-sensitive display having a centroid. A representation of the output is shown at Figure 8A-1 (in the bottom right corner), where the device is able to obtain information about the contact area and is able to determine the center of mass of this area.

[0335] The device also detects (908) a change in output from at least one of the multiple touch sensors, the change in output indicating that the area of ​​contact has expanded and the center of mass has moved. For example, the device may be detecting a scrolling of a user's finger that causes the user's finger to contact a different area of ​​the touch-sensitive display, where the different area (i) is larger than the first area and (ii) has a different center of mass that is different and separate from the center of mass, and the different center of mass is above at least a portion of the user interface element. In some embodiments, the device monitors the output from the touch sensor (e.g., a capacitive signal) to determine multiple parameters, including the area of ​​contact and the location of the center of mass (in addition to those discussed in more detail below). In this way, the device is able to finely distinguish between different types of gestures. Example in Figure 8A-1 to Figure 8A-3 As described in , the user provides Figure 8A-1 Start to Figure 8A-3The scroll gesture ends, and the representation of the touch sensor output in each of these figures shows how the area is enlarged and how the center of mass moves.

[0336] The inventors have discovered that it is important to monitor both the expansion of the area and the movement of the centroid (as well as other input features such as the saturation of the capacitive sensor, the speed of movement of the centroid, and / or the calculated angle of the user's finger making contact relative to the touch-sensitive display) in order to determine that a tip-to-groove scroll gesture has been detected. Monitoring only one of these parameters but not both may not effectively identify these gestures, thereby causing confusion to the user who may accidentally invoke the trackpad mode when it is not needed.

[0337] Based on the detected changes, the device operates at least a portion of the touch-sensitive display as a trackpad (910) and ceases operating the virtual keyboard (e.g., such that input above that portion of the touch-sensitive display controls movement of a cursor displayed in another area outside of that area of ​​the touch-sensitive display).

[0338] Allowing a user to scroll a single finger over a virtual keyboard and then operate the electronic device as a trackpad provides a convenient way for the user to quickly switch from keyboard input mode to trackpad mode using only a single finger; and detecting the scrolling by monitoring the expansion of the area and the movement of the center of mass ensures that the gesture is accurately detected and not confused with other types of gestures. Providing this interaction at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more effective (e.g., by allowing the user to easily switch to trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate trackpad mode without using trackpad mode, or without having to waste time using a complex sequence of user input or menu options to activate trackpad mode).

[0339] Now go to Fig. 9B In some embodiments, the text is displayed in an area of ​​the touch-sensitive display adjacent to the displayed virtual keyboard, and the cursor is displayed at the location within the text. While continuing to operate the portion of the touch-sensitive display as a trackpad, the device optionally detects (912) an additional change in output from at least one of the multiple touch sensors, the additional change in output indicating that the area of ​​contact has decreased and the center of mass has moved (e.g., reverse scrolling of the user's finger, such as Figure 8C-1 to Figure 8C-3 As described above), and in response to detecting an additional change in the output, the device maintains display of the cursor at the position within the text (912) (e.g., the user can scroll in reverse to begin moving the cursor, but this reverse scrolling will not move the cursor around). Figure 8C-3 As shown, after the reverse scroll gesture is completed, the cursor position in area 802 is different from that in Figure 8C-1 The same position as when the reverse scroll gesture started.

[0340] Ensuring that the cursor does not move (e.g., maintains its position) as the user reversely rolls their finger over the touch-sensitive display helps ensure that the cursor is accurately placed based on the user's intent. When the user simply reverses their finger, the user does so before they begin to move the cursor. Alternatively, a reverse scrolling gesture (e.g., a reverse scroll from the texture to the tip) may also occur after the user moves the cursor, but in which the user does not lift their finger off the touch screen. Thus, by maintaining the cursor in position during reverse scrolling, the operability of the device is enhanced and the human-machine interface operates more efficiently (e.g., by ensuring that the user does not need to waste time reversing unintended cursor movements).

[0341] In some embodiments, after detecting additional changes and while continuing to operate a portion of the touch-sensitive display as a touchpad, movement of the contact on the touch-sensitive display causes (914) the cursor to move to a different location within the text. Figure 8C-4 As shown, movement 834 of contact 836 - 3 on touch-sensitive display 112 causes corresponding movement of the cursor within area 802 .

[0342] In some embodiments, while continuing to operate a portion of the touch-sensitive display as a touchpad: the electronic device also detects (916) another change in output from at least one of the multiple touch sensors, the further change indicating that the area of ​​contact has expanded and the center of mass has moved (e.g., another tip-to-texture gesture is detected); and in response to detecting the further change, the device selects (916) at least a portion of the text. For example, Figure 8E-1 to Figure 8E-3 As shown, when the device has been operating a portion of the touch-sensitive display as a touchpad, a new scroll gesture is detected, and thus in response to the new scroll gesture, the text within the selection area 802 (such as Figure 8E-3 shown).

[0343] In some embodiments, the electronic device detects (918) the lifting of the contact from the touch-sensitive display and, in response, stops (918) operating a portion of the touch-sensitive display as a trackpad. Figure 8F As shown, after detecting lift-off of contact 840-3, the device then ceases to operate a portion of the touch-sensitive display as a touchpad (eg, Figure 8F shown).

[0344] Now go to Fig. 9CIn some embodiments, outputs and changes in outputs are detected (920) based at least in part on capacitance signals registered or measured by at least one of the plurality of touch sensors. Detecting outputs and changes in outputs based at least in part on capacitance signals helps ensure that gestures are accurately detected and not confused with other types of gestures, and enables devices that do not include force sensors to accurately detect new gesture types, such as the scroll / tip-to-groove gestures discussed herein. Providing such accurate detection at electronic devices that do not include force sensors enhances the operability of these devices and makes the human-computer interface on these devices more efficient (e.g., by allowing a user to easily switch to a trackpad mode using a simple gesture, without having to waste time trying to place a cursor to activate a trackpad mode when the trackpad mode is not in use, or without having to waste time using a complex sequence of user inputs or menu options to activate a trackpad mode).

[0345] In some embodiments, a change in the output is detected (922) based on one or more of: (i) a speed of movement of the center of mass; (ii) a change in size of the area; (iii) a saturation level of at least one touch sensor among a plurality of touch sensors; and (iv) a calculated angle of a user's finger making contact relative to the touch-sensitive display.

[0346] In some embodiments, a change in the output is further detected (924) based on the speed of movement of the center of mass.

[0347] In some embodiments, a change in the output is further detected (926) based on a change in size of the region.

[0348] In some embodiments, a change in the output is further detected (928) based on a saturation level of at least one touch sensor in the plurality of touch sensors.

[0349] In some embodiments, a change in the output is further detected (930) based on a calculated angle of a user's finger making contact relative to the touch-sensitive display.

[0350] Detecting outputs and changes in outputs by monitoring the other pieces of data discussed above (with respect to operations 922 to 930) helps ensure that gestures are accurately detected and not confused with other types of gestures, and enables devices that do not include force sensors to accurately detect new gesture types, such as the roll / tip-to-texture gesture discussed herein. Monitoring this additional data helps improve the accuracy of detecting the new gestures at electronic devices that do not include force sensors, thereby enhancing the operability of these devices and making the human-machine interface on these devices more effective (e.g., by ensuring that gestures are accurately detected, users will not have to waste time reversing the activation of undesired functions).

[0351] Now go to Fig.9D In some embodiments, tactile / tactile feedback is provided in conjunction with (e.g., simultaneously with) operating a portion of the touch-sensitive display as a touchpad. For example, once a tip-to-texture scrolling gesture is detected (e.g., an expansion of the contact area and a movement of the center of mass are detected), tactile / tactile feedback is provided while switching a portion of the touch-sensitive display to operate as a touchpad.

[0352] In some embodiments, the scroll gesture can be used to supplement or replace the extended contact gesture discussed above. Thus, the features of the extended contact gesture (e.g., method 600) discussed above for invoking trackpad mode can be used to supplement the features of the scroll gesture discussed herein.

[0353] It should be understood that 9A to 9D The specific order in which the operations have been described is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art will think of many ways to reorder the operations described herein and / or compare the operations with the reference operations. Figure 4 , Figure 7 as well as 6A to 6F The operations described can be combined in various ways.

[0354] For the purpose of explanation, the foregoing description is described by reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the disclosed precise form. According to the above teachings, many modifications and variations are possible. The embodiments are selected and described in order to best illustrate the principles of the present invention and its practical application, so that other technical personnel in the field can best use the present invention and the various described embodiments with various modifications suitable for the specific purpose contemplated.

Claims

1. A method comprising: At the electronic device that communicates with the touch screen: while at least a portion of the touch screen is in a trackpad mode and while a corresponding contact is in contact with the at least a portion of the touch screen, detecting a liftoff of the corresponding contact from the touch screen; and In response to detecting the liftoff of the corresponding contact from the touch screen: delaying exiting the at least a portion of the touch screen from the trackpad mode for at least a period of time based on determining that movement of the corresponding contact in the at least a portion of the touch screen prior to the lift-off of the corresponding contact satisfies one or more conditions; as well as Based on determining that the movement of the corresponding contact in the at least a portion of the touch screen prior to the lift-off of the corresponding contact does not satisfy the one or more conditions, causing the at least a portion of the touch screen to exit the touchpad mode without delaying the exit from the touchpad mode.

2. The method of claim 1 , wherein causing the at least a portion of the touch screen to exit the trackpad mode comprises: The at least a portion of the touch screen is caused to enter a soft keyboard mode.

3. The method according to claim 2, wherein entering the soft keyboard mode comprises: A soft keyboard is displayed in the at least a portion of the touch screen.

4. A method according to claim 3, wherein causing at least a portion of the touch screen to operate in the touchpad mode includes displaying the soft keyboard with a modified appearance, and while displaying the soft keyboard with the modified appearance, the corresponding contact is in contact with the at least a portion of the touch screen.

5. The method according to claim 1, further comprising: detecting, before the at least a portion of the touch screen is in the trackpad mode and while the at least a portion of the touch screen is in a soft keyboard mode including displaying a soft keyboard in the at least a portion of the touch screen, an input including a second corresponding contact in the soft keyboard; as well as In response to detecting the input: Based on determining that the input satisfies one or more second conditions, causing the at least a portion of the touch screen to exit the soft keyboard mode, and causing the at least a portion of the touch screen to enter the touchpad mode; as well as Based on determining that the input does not satisfy the one or more second conditions, remaining in the soft keyboard mode and performing a soft keyboard operation based on the input.

6. The method according to claim 1, further comprising: detecting, via the touch screen, an input including a second corresponding contact in the at least a portion of the touch screen after delaying causing the at least a portion of the touch screen to exit the trackpad mode and before expiration of the time period; as well as In response to detecting the input: Based on determining that the input satisfies one or more second conditions, the at least a portion of the touch screen is maintained in the touchpad mode, and a touchpad operation is performed based on the input.

7. The method according to claim 1, further comprising: After causing the at least a portion of the touch screen to exit the trackpad mode without delaying the exit from the trackpad mode, and while the at least a portion of the touch screen is not in the trackpad mode, detecting, via the touch screen, an input of a second corresponding contact included in the at least a portion of the touch screen; as well as In response to detecting the input: Based on determining that the input satisfies one or more second conditions, a soft keyboard operation is performed based on the input.

8. The method of claim 1, wherein the one or more conditions include: A condition is satisfied based on a speed of the movement of the corresponding contact prior to the lift-off of the corresponding contact.

9. A method according to claim 8, wherein the condition is met when the speed of the movement of the corresponding contact before the lift-off of the corresponding contact is greater than a speed threshold, and the condition is not met when the speed of the movement is less than the speed threshold.

10. The method of claim 1, wherein the one or more conditions include: A condition satisfied based on a direction of the movement of the corresponding contact prior to the lift-off of the corresponding contact.

11. The method of claim 1 , wherein the one or more conditions include: A condition is satisfied based on a distance of the movement of the corresponding contact prior to the lift-off of the corresponding contact.

12. The method according to claim 1, further comprising: detecting expiration of the time period after delaying causing the at least a portion of the touch screen to exit the trackpad mode and while the at least a portion of the touch screen remains in the trackpad mode without detecting a second corresponding contact in the at least a portion of the touch screen during the time period; as well as In response to detecting expiration of the time period, the at least a portion of the touch screen is exited from the trackpad mode.

13. An electronic device comprising: touchscreen; one or more processors; and A memory storing instructions, which, when executed by the one or more processors, cause the electronic device to perform the method according to any one of claims 1-12.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of an electronic device in communication with a touch screen, cause the electronic device to perform the method according to any one of claims 1-12.

15. A method comprising: At an electronic device, the electronic device comprising a touch-sensitive display coupled to a plurality of touch sensors, wherein the touch-sensitive display is not coupled to any force sensors: displaying a virtual keyboard on the touch-sensitive display; detecting an output from at least one touch sensor of the plurality of touch sensors, the output indicating contact with the touch-sensitive display over at least a portion of the virtual keyboard, wherein the contact includes an area of ​​the touch-sensitive display having a centroid; detecting a change in the output from at least one touch sensor of the plurality of touch sensors, the change in the output indicating that the area of ​​the contact has expanded and the centroid has moved; as well as Based on detecting the change, at least a portion of the touch-sensitive display is operated as a touchpad and operation of the virtual keyboard is ceased.

16. The method of claim 15, wherein the output and the change in the output are detected based at least in part on a capacitance signal registered by at least one touch sensor of the plurality of touch sensors.

17. A method according to claim 15, wherein the change in the output is detected based on one or more of the following: (i) the speed of movement of the center of mass; (ii) the change in size of the area; (iii) the saturation level of at least one touch sensor among the plurality of touch sensors; and (iv) the calculated angle of a user's finger making the contact relative to the touch-sensitive display.

18. The method of claim 15, wherein the change in the output is further detected based on a speed of movement of the center of mass. The method of claim 15 , wherein the change in the output is further detected based on a change in size of the region.

20. The method of claim 15, wherein the change in the output is further detected based on a saturation level of at least one touch sensor of the plurality of touch sensors.

21. The method of claim 15, wherein the change in the output is further detected based on a calculated angle of a finger of a user making the contact relative to the touch-sensitive display.

22. The method of claim 15, further comprising: Tactile feedback is provided by the electronic device in conjunction with invoking operation of the portion of the touch-sensitive display as the touchpad.

23. The method of claim 15, wherein text is displayed on the touch-sensitive display in an area of ​​the touch-sensitive display adjacent to the displayed virtual keyboard, a cursor is displayed at a location within the text, and the method further comprises, while continuing to operate the portion of the touch-sensitive display as the trackpad: detecting an additional change in the output from at least one touch sensor of the plurality of touch sensors, the additional change in the output indicating that the area of ​​the contact has decreased and the centroid has moved; and In response to detecting the further change in the output, display of the cursor is maintained at the position within the text.

24. The method of claim 23, wherein after detecting the additional change and while continuing to operate the portion of the touch-sensitive display as the touchpad, movement of the contact on the touch-sensitive display causes the cursor to move to a different location within the text.

25. The method of claim 23, further comprising: While continuing to operate the portion of the touch-sensitive display as the touchpad: detecting a further change in the output from at least one touch sensor of the plurality of touch sensors, the further change indicating that the area of ​​the contact has expanded and the centroid has moved; as well as In response to detecting the further change, at least a portion of the text is selected.

26. The method according to claim 25, further comprising: Liftoff of the contact from the touch-sensitive display is detected, and in response, operation of the portion of the touch-sensitive display as the touchpad is ceased.

27. The method of claim 15, wherein the electronic device is not configured to detect changes in intensity of the contact with the touch-sensitive display.

28. An electronic device comprising: touchscreen; one or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, wherein the electronic device is not coupled to any force sensor, and wherein the one or more programs include instructions for a method according to any one of claims 15-27.

29. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of an electronic device communicating with a touch screen, cause the electronic device to perform a method according to any one of claims 15-27, wherein the electronic device is not coupled to any force sensor.

Images