Handwriting keyboard for your screen

Abstract

To provide a faster and more efficient method for handwriting on a small screen and an electronic device that implements the method.SOLUTION: A method for handwriting implemented by an electronic device includes: displaying, on a display, a user input interface including a stroke input area on a touch-sensitive surface that receives a set of strokes and a message area that displays text; receiving a first set of strokes and determining first text; receiving a second set of strokes without displaying the first text; when a time period from receiving the first set of strokes to receiving the second set of strokes does not exceed a threshold, determining and displaying first text revised based on the first set of strokes and the second set of strokes; and when the time period exceeds the threshold, distinguishably displaying the first text and second text based on and the second set of strokes.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present disclosure relates generally to computer user interfaces, and more specifically to techniques for handwriting on small screens. [Background technology]

[0002] Wearable electronic devices often have a display screen for displaying information and for allowing minimal interfacing with the electronic device and the content stored on the device. Summary of the Invention

[0003] Some of these wearable devices include touch-sensitive surfaces, but the screens are small, making it difficult to input text directly into the device. Other methods of data input, such as dictation, are possible but have drawbacks. In the case of messaging, pre-defined messages are only useful when one of a limited number of pre-defined messages applies to the current situation.

[0004] However, some techniques for handwriting on small screens using electronic devices are generally cumbersome and inefficient. For example, some existing techniques use complex and time-consuming user interfaces that may involve multiple key presses or keystrokes. Existing techniques take longer than necessary, wasting the user's time and the device's energy. This latter consideration is particularly important in battery-operated devices.

[0005] Thus, the present technology provides electronic devices with faster and more efficient methods and interfaces for handwriting on small screens. Such methods and interfaces optionally complement or replace other methods for handwriting on small screens. Such methods and interfaces reduce the cognitive burden on the user and create a more efficient human-machine interface. In the case of battery-operated computing devices, such methods and interfaces conserve power and increase the time between battery charges. For example, the technology described below recognizes handwriting entered through a touch-sensitive surface more efficiently by considering both manual and automatic correction of recognized text, intelligently collecting and processing stroke input, and recognizing when stroke input belongs to the current character or the next character.

[0006] According to an embodiment, a portable electronic device having one or more processors, a touch-sensitive surface, a display, and a rotatable input mechanism includes: displaying a user input interface on the display including a message area and a stroke input area; receiving a first set of strokes on the stroke input area of ​​the touch-sensitive surface; determining a first text based on the first set of strokes; displaying the first text in the message area of ​​the display; determining one or more candidates based on the first text, the one or more candidates including one or more modifications to the first text; after determining the one or more candidates, receiving user input via the rotatable input mechanism; displaying at least one of the one or more candidates in response to the user input and displaying a selection indicator indicating a selected one of the one or more candidates; and after displaying the at least one candidate or the one or more candidates and the selection indicator, replacing the display of the first text with a display of the selected candidate.

[0007] An embodiment of a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a portable electronic device having a touch-sensitive surface, a display, and a rotatable input mechanism, the one or more programs including instructions for displaying a user input interface on the display including a message area and a stroke input area, receiving a first set of strokes at the stroke input area of ​​the touch-sensitive surface, determining a first text based on the first set of strokes, displaying the first text in the message area of ​​the display, determining one or more candidates based on the first text, the one or more candidates including one or more modifications to the first text, receiving user input via the rotatable input mechanism after determining the one or more candidates, displaying at least one of the one or more candidates in response to the user input, displaying a selection indicator indicating a selected one of the one or more candidates, and replacing the display of the first text with a display of the selected candidate after displaying the at least one candidate or the one or more candidates and the selection indicator.

[0008] According to one embodiment, an electronic device having one or more processors, a touch-sensitive surface, a display, and a rotatable input mechanism includes displaying a user input interface on the display including a message area and a stroke input area, receiving a first set of strokes in the stroke input area of ​​the touch-sensitive surface, determining first text based on the first set of strokes, displaying the first text in the message area, receiving a second set of strokes in the stroke input area of ​​the touch-sensitive surface after receiving the first set of strokes and displaying the first text, determining modified first text based on the first set of strokes and the second set of strokes, and replacing the display of the first text with the modified first text.

[0009] One embodiment of a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a touch-sensitive surface, a display, and a rotatable input mechanism, the one or more programs including instructions for displaying a user input interface on the display including a message area and a stroke input area, receiving a first set of strokes in the stroke input area of ​​the touch-sensitive surface, determining first text based on the first set of strokes, displaying the first text in the message area, receiving a second set of strokes in the stroke input area of ​​the touch-sensitive surface after receiving the first set of strokes and displaying the first text, determining modified first text based on the first set of strokes and the second set of strokes, and replacing the display of the first text with the modified first text.

[0010] According to one embodiment, an electronic device having one or more processors, a touch-sensitive surface, a display, and a rotatable input mechanism displays a user input interface on the display including a message area and a stroke input area, receives a first set of strokes in the stroke input area of ​​the touch-sensitive surface, initiates determining a first text based on the first set of strokes, after receiving the first set of strokes, receives a second set of strokes in the stroke input area of ​​the touch-sensitive surface without displaying the first text in the message area, determines a modified first text based on the first set of strokes and the second set of strokes, and displays the modified first text in the message area.

[0011] An embodiment of a temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a touch-sensitive surface, a display, and a rotatable input mechanism, the one or more programs including instructions for displaying a user input interface on a display including a message area and a stroke input area, receiving a first set of strokes in the stroke input area of ​​the touch-sensitive surface, initiating determination of first text based on the first set of strokes, receiving a second set of strokes in the stroke input area of ​​the touch-sensitive surface after receiving the first set of strokes without displaying the first text in the message area, determining modified first text based on the first set of strokes and the second set of strokes, and displaying the modified first text in the message area.

[0012] According to an embodiment, an electronic device having one or more processors, a touch-sensitive surface, a display, and a rotatable input mechanism includes displaying a user input interface on the display, the user input interface including a message area and a stroke input area; receiving a first stroke in the stroke input area of ​​the touch-sensitive surface; receiving a second stroke on the touch-sensitive surface at a first time after receiving the first stroke, the second stroke being different from the first stroke; determining whether the first time exceeds a threshold time; determining a first character based on the first stroke but not the second stroke in accordance with a determination that the first time exceeds the threshold time; determining a first character based on the first stroke and the second stroke in accordance with a determination that the first time is less than the threshold time; and displaying the first character in the message area.

[0013] An embodiment of a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a touch-sensitive surface, a display, and a rotatable input mechanism, the one or more programs including instructions for displaying a user input interface on the display including a message area and a stroke input area; receiving a first stroke in the stroke input area of ​​the touch-sensitive surface; receiving a second stroke on the touch-sensitive surface at a first time after receiving the first stroke, the second stroke being different from the first stroke; determining whether the first time exceeds a threshold time; determining a first character based on the first stroke but not the second stroke in accordance with a determination that the first time exceeds the threshold time; determining a first character based on the first stroke and the second stroke in accordance with a determination that the first time is less than the threshold time; and displaying the first character in the message area.

[0014] Executable instructions to perform these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors. Executable instructions to perform these functions are optionally included in a transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0015] This provides devices with faster, more efficient methods and interfaces for handwriting on small screens, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices, which may complement or replace other methods for handwriting on small screens. [Brief description of the drawings]

[0016] For a better understanding of the various embodiments described above, reference should be made to the following description of the embodiments in conjunction with the drawings, in which like reference numerals refer to corresponding parts throughout.

[0017] [Figure 1A] FIG. 1 is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

[0018] [Figure 1B] FIG. 2 is a block diagram illustrating example components for event processing according to some embodiments.

[0019] [Diagram 2] FIG. 1 illustrates a portable multifunction device having a touch screen in accordance with some embodiments.

[0020] [Diagram 3] FIG. 2 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments.

[0021] [Figure 4A] 1 illustrates an exemplary user interface for an application menu of a portable multifunction device in accordance with some embodiments.

[0022] [Figure 4B] 1 illustrates an exemplary user interface for a multifunction device having a touch-sensitive surface separate from a display in accordance with some embodiments.

[0023] [Figure 5A] FIG. 1 illustrates a personal electronic device according to some embodiments.

[0024] [Figure 5B] FIG. 1 is a block diagram illustrating a personal electronic device according to some embodiments.

[0025] [Figure 6A] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6B] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6C] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6D] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6E] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6F] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6G] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6H] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6I] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6J] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6K] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6L] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6M] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6N] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6O] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6P] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6Q] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 6R]FIG. 1 illustrates an exemplary user interface for on-screen handwriting.

[0026] [Figure 7] FIG. 1 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface.

[0027] [Figure 8] FIG. 2 is an exemplary functional block diagram of an electronic device.

[0028] [Figure 9A] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9B] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9C] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9D] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9E] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9F] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9G] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9H] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 9I] FIG. 1 illustrates an exemplary user interface for on-screen handwriting.

[0029] [Figure 10] FIG. 1 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface.

[0030] [Figure 11] FIG. 2 is an exemplary functional block diagram of an electronic device.

[0031] [Figure 12A] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12B] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12C] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12D] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12E] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12F] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12G] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12H] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12I] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12J] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12K] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 12L] FIG. 1 illustrates an exemplary user interface for on-screen handwriting.

[0032] [Figure 13] FIG. 1 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface.

[0033] [Figure 14] FIG. 2 is an exemplary functional block diagram of an electronic device.

[0034] [Figure 15A]FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15B] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15C] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15D] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15E] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15F] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15G] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15H] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15I] FIG. 1 illustrates an exemplary user interface for on-screen handwriting. [Figure 15J] FIG. 1 illustrates an exemplary user interface for on-screen handwriting.

[0035] [Figure 16] FIG. 1 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface.

[0036] [Figure 17] FIG. 2 is an exemplary functional block diagram of an electronic device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] In the following description, example methods, parameters, and the like are set forth. However, it should be recognized that such description is not intended as a limitation on the scope of the present disclosure, but is provided as a description of example embodiments.

[0038] There is a need for electronic devices that provide efficient methods and interfaces for handwriting on small screens. For example, wearable devices with small touch-sensitive surfaces allow for access and navigation to some content and data, but currently these screens are difficult to use when directly entering text data. The techniques described herein result in better handwriting input on touch-sensitive surfaces, especially small touch-sensitive surfaces. Such techniques may reduce the cognitive burden on users who input handwriting on small screens, thereby increasing productivity. Furthermore, such techniques may reduce processor and battery power that is otherwise wasted on redundant user input.

[0039] Below, Figures 1A-1B, 2, 3, 4A-4B, and 5A-5H provide a description of an example device for performing techniques for managing event notifications. Figures 6A-6R are diagrams illustrating example user interfaces for managing event notifications. Figure 7 is a flow diagram illustrating a method for managing event notifications, according to some embodiments. The user interfaces in Figures 6A-6R are used to illustrate processes described below, including the process in Figure 7.

[0040] 9A-9I show example user interfaces for managing event notifications. Figure 10 is a flow diagram showing a method for managing event notifications according to some embodiments. The user interfaces in Figures 9A-9I are used to illustrate the processes described below, including the process in Figure 10B.

[0041] 12A-12L show example user interfaces for managing event notifications. FIG. 13 is a flow diagram showing a method for managing event notifications according to some embodiments. The user interfaces in FIG. 12A-12L are used to illustrate the processes described below, including the process in FIG. 13.

[0042] 15A-15J show example user interfaces for managing event notifications. FIG 16 is a flow diagram showing a method for managing event notifications according to some embodiments. The user interfaces in FIG 15A-15J are used to illustrate the processes described below, including the process in FIG 16.

[0043] In the following description, terms such as "first" and "second" are used to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first touch can be referred to as a second touch, and similarly, a second touch can be referred to as a first touch, without departing from the scope of the various embodiments described. A first touch and a second touch are both touches, but are not the same touch.

[0044] The terms used in the description of the various described embodiments herein are intended to describe only the particular embodiments and are not intended to be limiting. When used in the description of the various described embodiments and in the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is also to be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes", "including", "comprises" and / or "comprising", as used herein, specify the presence of the described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0045] The term "if" is, optionally, interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are, optionally, interpreted to mean "upon determining," or "in response to determining," or "upon detecting (the stated condition or event)," or "in response to detecting (the stated condition or event)," depending on the context.

[0046] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile telephone, that also includes other functions, such as PDA functionality and / or music playback functionality. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers with a touch-sensitive surface (e.g., a touchscreen display and / or a touchpad), can also optionally be used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., a touchscreen display and / or a touchpad).

[0047] In the following discussion, we describe an electronic device that includes a display and a touch-sensitive surface, but it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0048] The device typically supports a variety of applications, such as one or more of drawing applications, presentation applications, word processing applications, website creation applications, disc authoring applications, spreadsheet applications, gaming applications, telephony applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music playback applications, and / or digital video playback applications.

[0049] The various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more features of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or changed for each application and / or within the corresponding application. In this way, the common physical architecture (such as the touch-sensitive surface) of the device optionally supports a variety of applications with user interfaces that are intuitive and transparent to the user.

[0050] Attention now turns to embodiments of portable devices with touch-sensitive displays. FIG. 1A is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112, according to some embodiments. Touch-sensitive display 112 may be conveniently referred to as a "touch screen" and may also be known or referred to as a touch-sensitive display system. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), memory controller 122, one or more processing units (CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input control devices 116, and external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors 165 for detecting the intensity of a contact on device 100 (e.g., a touch-sensitive surface such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0051] As used herein and in the claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or a proxy for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four distinct values, and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using a variety of methods and a variety of sensors, or combinations of sensors. For example, one or more force sensors disposed under or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some implementations, the force measurements of multiple force sensors are combined (e.g., a weighted average) to determine an estimate of the contact force. Similarly, a pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or change in the contact area detected on the touch-sensitive surface, the capacitance and / or change in the capacitance of the touch-sensitive surface proximate the contact, and / or the resistance and / or change in the capacitance of the touch-sensitive surface proximate the contact are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measure of the force or pressure of the contact is used directly to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measure). In some implementations, the surrogate measure of the force or pressure of the contact is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of contact as an attribute of user input allows for user access to additional device functionality that may otherwise be inaccessible by the user on a reduced-size device with a limited area for displaying affordances (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls such as knobs or buttons).

[0052] As used herein and in the claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device that is detected by a user by the user's sense of touch. For example, in a situation where a device or a component of the device is in contact with a touch-sensitive user surface (e.g., the fingers, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical characteristic of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "down-click" or "up-click" of a physical actuator button. In some cases, the user will feel a tactile sensation such as a "down-click" or "up-click" even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's action. As another example, movement of a touch-sensitive surface is optionally interpreted or sensed by a user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. Such interpretation of touch by a user depends on the user's individual sensory perception, but there are many sensory perceptions of touch that are common to a majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an "up click," "down click," "roughness"), unless otherwise noted, the generated tactile output corresponds to a physical displacement of the device, or a component thereof, that produces the described sensory perception of a typical (or average) user.

[0053] It should be noted that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally has two or more components, optionally combines components, or has components in a different configuration or arrangement. The various components shown in FIG. 1A may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and / or application specific integrated circuits.

[0054] Memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. A memory controller 122 can optionally control access to memory 102 by other components of device 100.

[0055] The peripherals interface 118 may be used to couple input and output peripherals of the device to the CPU 120 and memory 102. The one or more processors 120 operate or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions for the device 100 as well as to perform data processing. In some embodiments, the peripherals interface 118, the CPU 120, and the memory controller 122 are optionally implemented on a single chip, such as the chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0056] The RF (Radio Frequency) circuitry 108 receives and transmits RF signals, also referred to as electromagnetic signals. The RF circuitry 108 converts electrical signals to / from electromagnetic signals and communicates with communication networks and other communication devices via the electromagnetic signals. The RF circuitry 108 optionally includes well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and the like. The RF circuitry 108 optionally communicates by wireless communication with networks such as the Internet, also referred to as the World Wide Web (WWW), intranets and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), and other devices. The RF circuitry 108 optionally includes well-known circuits for detecting near-field communication (NFC) fields, such as by short-range communication radio. The wireless communication optionally uses any of a number of communication standards, protocols, and technologies, including Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (WCO), and the like.Wireless technology may include, but is not limited to, wireless LAN, wireless LANs, wireless LAN networks, wireless LAN devices, wireless LAN systems, wireless LAN networks ... Examples of suitable communication protocols include, but are not limited to, Simple Intermediate System (SIMPLE), Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

[0057] The audio circuit 110, speaker 111, and microphone 113 provide an interface between a user and the device 100. The audio circuit 110 receives audio data from the peripherals interface 118, converts the audio data to an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal to sound waves audible to humans. The audio circuit 110 also receives an electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal to audio data and transmits the audio data to the peripherals interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripherals interface 118. In some embodiments, the audio circuit 110 further comprises a headset jack (e.g., 212 in FIG. 2). The headset jack provides an interface between audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset that has both an output (e.g., one- or binaural headphones) and an input (e.g., a microphone).

[0058] The I / O subsystem 106 couples input / output peripherals of the device 100, such as the touch screen 112 and other input control devices 116, to a peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, a light sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals to / from the other input control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) 160 are optionally coupled to any (or none) of a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more buttons (e.g., 208 in FIG. 2) optionally include up / down buttons for adjusting the volume of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button (e.g., 206 in FIG. 2).

[0059] A quick press of the push button may optionally unlock the touch screen 112 or optionally initiate a process of using gestures on the touch screen to unlock the device, as described in U.S. Patent Application Serial No. 11 / 322,549, U.S. Patent No. 7,657,849, filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," which is incorporated herein by reference in its entirety. A longer press of the push button (e.g., 206) optionally powers the device 100 on and off. The function of one or more of the buttons is optionally customizable by the user. The touch screen 112 is used to implement virtual or soft buttons and one or more soft keyboards.

[0060] The touch-sensitive display 112 provides an input and output interface between the device and a user. The display controller 156 receives and sends electrical signals to the touch screen 112. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0061] Touchscreen 112 has a touch-sensitive surface, sensor, or set of sensors that receives input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction sets in memory 102) detects contacts (and any movement or breaking of contact) on touchscreen 112 and translates the detected contacts into interactions with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In one exemplary embodiment, the point of contact between touchscreen 112 and the user corresponds to the user's finger.

[0062] Touch screen 112 optionally uses LCD (liquid crystal display), LPD (light emitting polymer display), or LED (light emitting diode) technology, although other display technologies are used in other embodiments. Touch screen 112 and display controller 156 optionally detect contact and any movement or interruption thereof using any of a number of now known or later developed touch sensing technologies, including, but not limited to, capacitive, resistive, infrared, and surface ultrasonic technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch screen 112. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in the iPhone® and iPod Touch® by Apple Inc. of Cupertino, California.

[0063] The touch-sensitive display in some embodiments of touchscreen 112 is optionally similar to the multi-touch-sensing touchpads described in the following U.S. Patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman), and / or U.S. Patent Application Publication No. 2002 / 0015024 (A1), each of which is incorporated by reference in its entirety herein. However, whereas touchscreen 112 displays visual output from device 100, a touch-sensitive touchpad does not provide visual output.

[0064] Some embodiments of touch-sensitive displays for touch screen 112 are described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, entitled "Multipoint Touch Surface Controller"; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, entitled "Multipoint Touchscreen"; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, entitled "Gestures For Touch Sensitive Input Devices"; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, entitled "Gestures For Touch Sensitive Input Devices"; (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, entitled "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices"; (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, entitled "Virtual Input Device Placement On A Touch Screen User Interface" No. 11 / 228,700, filed Sep. 16, 2005, entitled "Operation Of A Computer With A Touch Screen Interface," (7) No. 11 / 228,700, filed Sep. 16, 2005, entitled "Operation Of A Computer With A Touch Screen Interface," (8) No. 11 / 228,737, filed Sep. 16, 2005, entitled "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard," and (9) No. 11 / 367,749, filed Mar. 3, 2006, entitled "Multi-Functional Hand-Held Device," all of which are incorporated herein by reference in their entirety.

[0065] Touchscreen 112 optionally has a video resolution of 100 dpi or greater. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. A user optionally contacts touchscreen 112 using any suitable object or appendage, such as a stylus, finger, etc. In some embodiments, the user interface is designed to function primarily with finger-based contacts and gestures, which may be less precise than stylus-based input due to the large contact area of ​​a finger on a touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor position or commands to perform the user's desired action.

[0066] In some embodiments, in addition to a touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating certain functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touchscreen 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0067] Device 100 also includes a power system 162 for providing power to the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power within a portable device.

[0068] Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor coupled to light sensor controller 158 in I / O subsystem 106. Light sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. Light sensor 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. Light sensor 164, in conjunction with imaging module 143 (also referred to as a camera module), optionally captures still images or video. In some embodiments, the light sensor is located on the back of device 100, opposite touchscreen display 112 on the front of the device, to enable the touchscreen display as a viewfinder for still image and / or video capture. In some embodiments, the light sensor is located on the front of the device to optionally obtain an image of the user for videoconference at the same time the user sees other videoconference participants on the touchscreen display. In some embodiments, the position of the light sensor 164 can be changed by the user (e.g., by rotating the lens and sensor within the device housing), such that a single light sensor 164 is used in conjunction with a touch screen display for both video conferencing and capturing still and / or video images.

[0069] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 in I / O subsystem 106. Contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor 165 receives contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is located on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite touchscreen display 112 located on the front of device 100.

[0070] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is optionally coupled to an input controller 160 in I / O subsystem 106. The proximity sensor 166 optionally functions as described in U.S. patent application Ser. No. 11 / 241,839, "Proximity Detector In Handheld Device," Ser. No. 11 / 240,788, "Proximity Detector In Handheld Device," Ser. No. 11 / 620,702, "Using Ambient Light Sensor To Augment Proximity Sensor Output," Ser. No. 11 / 586,862, "Automated Response To And Sensing Of User Activity In Portable Devices," and Ser. No. 11 / 638,251, "Methods And Systems For Automatic Configuration Of Peripherals," which are incorporated herein by reference in their entireties. In some embodiments, the proximity sensor turns off and disables the touchscreen 112 when the multifunction device is placed near the user's ear (e.g., when the user is talking on the phone).

[0071] Device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows a tactile output generator coupled to haptic feedback controller 161 in I / O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile output in the device). Contact intensity sensor 165 receives tactile feedback generation instructions from haptic feedback module 133 and generates a tactile output on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is disposed on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112) and generates a tactile output, optionally by moving the touch-sensitive surface vertically (e.g., in / out of the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, the at least one tactile output generator sensor is disposed on the back of device 100, opposite touchscreen display 112, which is disposed on the front of device 100.

[0072] Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows accelerometer 168 coupled to peripherals interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I / O subsystem 106. Accelerometer 168 optionally functions as described in U.S. Patent Application Publication Nos. 20050190059, “Acceleration-based Theft Detection System for Portable Electronic Devices,” and 20060017692, “Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer,” both of which are incorporated herein by reference in their entireties. In some embodiments, information is displayed on the touchscreen display in portrait or landscape orientation based on an analysis of data received from the one or more accelerometers. In addition to accelerometer(s) 168, device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the location and orientation (e.g., portrait or landscape) of device 100.

[0073] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and applications (or instruction sets) 136. Additionally, in some embodiments, as shown in FIGS. 1A and 3, memory 102 (FIG. 1A) or memory 370 (FIG. 3) stores device / global internal state 157. Device / global internal state 157 includes one or more of: an active application state, which indicates which applications, if any, are currently active; a display state, which indicates which applications, views, or other information are occupying various regions of touchscreen display 112; a sensor state, which includes information obtained from the device's various sensors and input control devices 116; and location information relating to the device's position and / or orientation.

[0074] Operating system 126 (e.g., an embedded operating system such as Darwin®, RTXC®, LINUX®, UNIX®, OS X®, iOS®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0075] The communications module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) are adapted to couple to other devices directly or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors the same as or similar to the 30-pin connector used on iPod® (trademark of Apple Inc.) devices and / or compatible multi-pin (e.g., 30-pin) connectors.

[0076] Contact / motion module 130 (in conjunction with display controller 156) optionally detects contact with touch screen 112 and other touch-sensing devices (e.g., a touchpad or a physical click wheel). Contact / motion module 130 includes various software components for performing various operations related to detecting contact, such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking of the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger up event or an interruption of contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by the series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions are, optionally, applied to a single contact (e.g., one finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, contact / motion module 130 and display controller 156 detect contacts on a touchpad.

[0077] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has “clicked” on an icon). In some embodiments, at least a subset of the intensity thresholds are determined according to software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of a particular physical actuator and may be adjusted without modifying the physical hardware of the device 100). For example, the mouse “click” threshold of a trackpad or touchscreen display may be set to any of a wide range of predefined thresholds without modifying the hardware of the trackpad or touchscreen display. Additionally, in some implementations, a user of the device is provided with software settings to adjust one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once via a system-level click “intensity” parameter).

[0078] Contact / motion module 130 optionally detects gesture input by a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timing, and / or intensity of detected contact). Thus, a gesture is optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger down event (e.g., at an icon location) followed by detecting a finger lift off event at the same location (or substantially the same location) as the finger down event. As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger down event followed by detecting one or more finger drag events followed by detecting a finger lift off event.

[0079] Graphics module 132 includes various known software components for rendering and displaying graphics on touchscreen 112 or other display, including components for modifying the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

[0080] In some embodiments, graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. Graphics module 132 receives one or more codes specifying the graphics to be displayed, including, as appropriate, coordinate data and other graphic characteristic data, from an application or the like, and then generates screen image data for output to display controller 156.

[0081] The tactile feedback module 133 includes various software components for generating instructions used by the tactile output generator(s) 167 to generate tactile outputs at one or more locations on the device 100 in response to a user's interaction with the device 100.

[0082] Text input module 134, optionally a component of graphics module 132, provides a soft keyboard for entering text into various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0083] The GPS module 135 determines the location of the device and provides this information for various uses (e.g., to the phone 138 for use in location-based dialing, to the camera 143 as picture / video metadata, and to applications that provide location-based services such as a weather widget, a local yellow pages widget, and a maps / navigation widget).

[0084] Application 136 optionally includes the following modules (or sets of instructions), or a subset or superset thereof: ● a contacts module 137 (sometimes called an address book or contact list); ● Telephone module 138, ●Video conferencing module 139; ● an email client module 140; ● Instant messaging (IM) module 141; ● Training support module 142, A camera module 143 for still and / or video images, ● Image management module 144, ●Video playback module ●Music playback module ● Browser module 147, ● Calendar module 148, A widget module 149, optionally including one or more of a weather widget 149-1, a stock price widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widgets 149-6; ● a widget creation module 150 for creating user-created widgets 149-6; ● Search module 151, ● A video and music playback module 152 that integrates a video playback module and a music playback module; ● Memo module 153, Map module 154, and / or ●Online video module 155.

[0085] Optionally, examples of other applications 136 stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice duplication.

[0086] The contacts module 137, in conjunction with the touch screen 112, the display controller 156, the contact / motion module 130, the graphics module 132, and the text input module 134, is optionally used to manage an address book or contact list (e.g., stored in the memory 102 or in the application internal state 192 of the contacts module 137 in the memory 370), including adding a name(s) to the address book, deleting a name(s) from the address book, associating a phone number(s), email address(es), physical address(es), or other information with a name, associating a picture with a name, categorizing and sorting names, providing phone numbers or email addresses to initiate and / or facilitate communication by telephone 138, videoconferencing module 139, email 140, or IM 141, and the like.

[0087] Telephone module 138, in conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, is optionally used to enter a series of characters corresponding to a telephone number, access one or more telephone numbers in contacts module 137, modify an entered telephone number, dial the corresponding telephone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As mentioned above, the wireless communication optionally uses any of a number of communication standards, protocols, and technologies.

[0088] Videoconferencing module 139, in conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, light sensor 164, light sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, contains executable instructions for initiating, conducting, and terminating a videoconference between a user and one or more other participants in accordance with the user's instructions.

[0089] Email client module 140, in conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, contains executable instructions for creating, sending, receiving, and managing emails in response to user instructions. Email client module 140, in conjunction with image management module 144, greatly facilitates creating and sending emails with attachments of still or video images captured by camera module 143.

[0090] Instant messaging module 141, in conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, includes executable instructions for entering characters corresponding to an instant message, modifying characters already entered, sending the corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), and receiving and displaying the received instant message. In some embodiments, the sent and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0091] The training support module 142, in conjunction with the RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music playback module, includes executable instructions for generating workouts (e.g., having time, distance, and / or calorie burn goals), communicating with training sensors (sports devices), receiving training sensor data, calibrating sensors used to monitor the workouts, selecting and playing music for the workouts, and displaying, storing, and transmitting the workout data.

[0092] Camera module 143, in conjunction with touch screen 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, contains executable instructions for capturing still images or video (including video streams) and storing them in memory 102, changing characteristics of the still images or video, or deleting the still images or video from memory 102.

[0093] Image management module 144, in conjunction with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and camera module 143, contains executable instructions for arranging, modifying (e.g., editing) and otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.

[0094] Browser module 147, in conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, contains executable instructions for browsing the Internet in accordance with a user's instructions, including retrieving, linking, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0095] Calendar module 148, in conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, contains executable instructions for creating, displaying, modifying, and storing calendars and data associated with the calendars (e.g., calendar items, to-do lists, etc.) in accordance with a user's instructions.

[0096] Widget modules 149, in conjunction with RF circuitry 108, touch screen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, are optionally mini-applications that can be downloaded and used by a user (e.g., weather widget 149-1, stocks widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or created by a user (e.g., user-created widget 149-6). In some embodiments, a widget includes an HTML (HyperText Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, a widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0097] Widget creation module 150, in conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, is optionally used by a user when creating a widget (e.g., rotating a user-specified portion of a web page for the widget).

[0098] The search module 151, in conjunction with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, contains executable instructions for searching the memory 102 for text, music, sound, images, video, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with a user's instructions.

[0099] Video and music playback module 152, in conjunction with touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, includes executable instructions that enable a user to download and play pre-recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing videos (on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (a trademark of Apple Inc.).

[0100] The notes module 153, in conjunction with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, contains executable instructions for creating and managing notes, to-do lists, and the like, in accordance with a user's instructions.

[0101] Map module 154, in conjunction with RF circuitry 108, touch screen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, is optionally used to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data about stores and other points of interest at or near a particular location, and other location-specific data) in accordance with a user's instructions.

[0102] Online video module 155, in conjunction with touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, includes instructions that enable a user to access, view, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen or on an external display connected via external port 124), and send and otherwise manage emails containing links to particular online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module 141 is used to send links to particular online videos, rather than email client module 140. Further description of online video applications can be found in U.S. Provisional Patent Application No. 60 / 936,562, entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," filed June 20, 2007, and U.S. Patent Application No. 11 / 968,067, entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," filed December 31, 2007, the contents of which are incorporated by reference in their entireties herein.

[0103] Each of the above identified modules and applications corresponds to an executable instruction set and method described herein (e.g., computer-implemented methods and other information processing methods described herein) that performs one or more of the above functions. These modules (e.g., instruction sets) need not be implemented as separate software programs, procedures, or modules, and thus, in various embodiments, various subsets of these modules are optionally combined or otherwise rearranged. For example, a video playback module is optionally combined with a music playback module into a single module (e.g., video and music playback module 152 of FIG. 1A). In some embodiments, memory 102 optionally stores a subset of the above identified modules and data structures. Additionally, memory 102 optionally stores additional modules and data structures not described above.

[0104] In some embodiments, device 100 is a device in which operation of a default set of functions of the device is performed exclusively via the touchscreen and / or touchpad, optionally reducing the number of physical input control devices (such as push buttons, dials, and the like) of device 100 by using a touchscreen and / or touchpad as the primary input control device for operation of device 100.

[0105] The default set of functions performed exclusively via the touchscreen and / or touchpad, optionally, includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 to a main menu, a home menu, or a root menu from any user interface displayed on device 100. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

[0106] 1B is a block diagram illustrating example components for event processing, according to some embodiments. In some embodiments, memory 102 (FIG. 1A) or memory 370 (FIG. 3) includes an event sorter 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any of applications 137-151, 155, 380-390 described above).

[0107] Event sorter 170 receives the event information and determines which application 136-1 and application view 191 of application 136-1 to deliver the event information to. Event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192 that indicates the current application view(s) that are displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event sorter 170 to determine which application(s) are currently active, and application internal state 192 is used by event sorter 170 to determine which application view 191 to deliver the event information to.

[0108] In some embodiments, application internal state 192 includes additional information, such as one or more of resume information used when application 136-1 resumes execution, user interface state information indicating information being displayed or ready to be displayed by application 136-1, a state queue to allow the user to return to a previous state or view of application 136-1, and a redo / undo queue for actions previously taken by the user.

[0109] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user's touch on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer(s) 168, and / or microphone 113 (via audio circuitry 110). Information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

[0110] In some embodiments, event monitor 171 sends a request to peripherals interface 118 at a predetermined interval. In response, peripherals interface 118 transmits event information. In other embodiments, peripherals interface 118 transmits event information only if there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or for a predetermined duration).

[0111] In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173 .

[0112] Hit view determination module 172 provides software procedures for determining where a sub-event occurred within one or more views when touch-sensitive display 112 displays one or more views. A view consists of controls and other elements that a user can see on the display.

[0113] Another aspect of a user interface associated with an application is the set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and in which touch-based gestures occur. The application view (of the respective application) in which a touch is detected optionally corresponds to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest-level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are recognized as suitable inputs is optionally determined, at least in part, based on the hit view of the initial touch that initiates the touch-based gesture.

[0114] Hit view determination module 172 receives information related to sub-events of a touch-based gesture. If an application has multiple views organized as a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should process the sub-event. In most situations, the hit view is the lowest level view in which the first sub-event occurs (e.g., the first sub-event in a sequence of sub-events that form an event or potential event). Once a hit view is identified by hit view determination module 172, the hit view typically receives all sub-events associated with the same touch or input source for which it was identified as the hit view.

[0115] Active event recognizer determination module 173 determines which view(s) in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that contain the physical location of the sub-event are actively participating views, and therefore all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to the area associated with a particular view, views higher in the hierarchy still remain actively participating views.

[0116] Event dispatcher module 174 sends the event information to an event recognizer (e.g., event identifier 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers the event information to the event recognizer determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information obtained by a corresponding event acceptor 182 in an event queue.

[0117] In some embodiments, operating system 126 includes event sorter 170. Alternatively, application 136-1 includes event sorter 170. In yet other embodiments, event sorter 170 is a stand-alone module or part of another module stored in memory 102, such as contact / motion module 130.

[0118] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each including instructions for handling touch events that occur within a respective view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, each application view 191 includes multiple event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface kit (not shown) or a higher level object from which application 136-1 inherits methods and other properties. In some embodiments, each event handler 190 includes one or more of a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from event sorter 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more respective event handlers 190. In some embodiments, one or more of a data updater 176, an object updater 177, and a GUI updater 178 are also included in the corresponding application view 191.

[0119] Each event recognizer 180 receives event information (e.g., event data 179) from event sorter 170 and identifies an event from the event information. Event recognizer 180 includes an event acceptor 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188 (optionally including sub-event delivery instructions).

[0120] The event acceptor 182 receives event information from the event sorter 170. The event information includes information about a sub-event, e.g., information about a touch or a movement of a touch. Depending on the sub-event, the event information also includes additional information, such as a location of the sub-event. When the sub-event involves a movement of a touch, the event information optionally further includes a speed and a direction of the sub-event. In some embodiments, the event includes a rotation of the device from one orientation to another (e.g., from portrait to landscape orientation or vice versa), and the event information includes corresponding information about the current orientation of the device (also called the device's attitude).

[0121] The event comparator 184 compares the event information to predefined event or sub-event definitions and determines the event or sub-event or determines or updates the state of the event or sub-event based on the comparison. In some embodiments, the event comparator 184 includes an event definition 186. The event definition 186 includes a definition of an event (e.g., a predefined sequence of sub-events), such as, for example, event 1 (187-1), event 2 (187-2), etc. In some embodiments, the sub-events in an event (187) include, for example, touch start, touch end, touch movement, touch stop, and multiple touches. In one example, the definition of event 1 (187-1) is a double tap on a displayed object. The double tap includes, for example, a first touch on a displayed object for a given stage (touch start), a first lift off (touch end) for the given stage, a second touch on a displayed object for a given stage (touch start), and a second lift off (touch end) for the given stage. In another example, a definition of event 2 (187-2) is a drag operation on a displayed object, including, for example, a touch (or contact) on the displayed object for a given step, a movement of the touch across the touch-sensitive display 112, and a lift-off of the touch (end of the touch). In some embodiments, the event also includes information about one or more associated event handlers 190.

[0122] In some embodiments, event definition 187 includes a definition of the event for each user interface object. In some embodiments, event comparator 184 performs a hit test to determine the user interface object associated with the sub-event. For example, in an application view in which three user interface objects are displayed on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (the sub-event). If each displayed object is associated with a corresponding event handler 190, event comparator 184 uses the results of the hit test to determine which event handler 190 to activate. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggers the hit test.

[0123] In some embodiments, the definition of each event (187) also includes a delay action that delays dissemination of the event information until it is determined whether the sequence of sub-events corresponds or does not correspond to the event type of the event recognizer.

[0124] If each event recognizer 180 determines that the sequence of sub-events does not match any of the events in event definition 186, then each event recognizer 180 enters an event disabled, event failed, or event finished state and thereafter ignores the next sub-event of the touch-based gesture. In this situation, any other event recognizers that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0125] In some embodiments, each event recognizer 180 includes a meta list 183 with configurable properties, flags, and / or lists that indicate how the event distribution system performs sub-event distribution to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers may or are enabled to interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to various levels in a view or programmatic hierarchy.

[0126] In some embodiments, each event recognizer 180 activates an event handler 190 associated with an event when one or more particular sub-events of the event are recognized. In some embodiments, each event recognizer 180 delivers event information associated with the event to the event handler 190. Activating an event handler 190 is distinct from sending (and postponing sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the recognized event, and the event handler 190 associated with the flag catches the flag and performs predefined processing.

[0127] In some embodiments, the event distribution instructions 188 include sub-event distribution instructions that distribute event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions send the event information to an event handler associated with a set of sub-events or an actively participating view. The event handler associated with the set of sub-events or an actively participating view receives the event information and performs a predetermined operation.

[0128] In some embodiments, data updater 176 creates and updates data used by application 136-1. For example, data updater 176 updates phone numbers used by contacts module 137 or stores video files used by video playback module. In some embodiments, object updater 177 creates and updates objects used by application 136-1. For example, object updater 177 creates new user interface objects or updates the positions of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on the touch-sensitive display.

[0129] In some embodiments, event handler 190(s) includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of each application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0130] It will be appreciated that the above discussion regarding event processing of user touches on a touch-sensitive display also applies to other forms of user input operating multifunction device 100 having input devices, not all of which are necessarily initiated on a touch screen. For example, mouse movements and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holds, touch movements such as tapping, dragging, scrolling on a touchpad, pen stylus input, device movements, verbal cues, detected eye movements, biometric input, and / or any combination thereof, are optionally utilized as inputs corresponding to sub-events that define the event to be recognized.

[0131] FIG. 2 illustrates portable multifunction device 100 having touch screen 112, according to some embodiments. The touch screen optionally displays one or more graphics in user interface (UI) 200. In this embodiment, as well as in embodiments described below, a user can select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers 202 (not drawn to scale) or one or more styluses 203 (not drawn to scale). In some embodiments, selection of one or more graphics occurs when the user loses contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, upward and / or downward), and / or rolling of a finger in contact with device 100 (right to left, left to right, upward and / or downward). In some implementations or situations, accidental contact with a graphic does not select the graphic. For example, if the gesture corresponding to selection is a tap, a swipe gesture sweeping over an application icon optionally does not select the corresponding application.

[0132] Device 100 also optionally includes one or more physical buttons, such as a “home” or menu button 204. As mentioned above, menu button 204 is optionally used for navigation to any application 136 in a set of applications that optionally execute on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touch screen 112.

[0133] In some embodiments, device 100 includes touch screen 112, menu button 204, push button 206 for turning power to the device on / off and locking the device, volume control button(s) 208, subscriber identity module (SIM) card slot 210, headset jack 212, and external port 124 for docking / charging. Push button 206 is optionally used to power the device on / off by pressing and holding the button down for a predefined time, to lock the device by pressing and releasing the button before the predefined time has elapsed, and / or to unlock the device or initiate an unlocking process. In another embodiment, device 100 also receives verbal input through microphone 113 for activating or deactivating some functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of a contact on touch screen 112 and / or one or more tactile output generators 167 for generating a tactile output to a user of device 100.

[0134] FIG. 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface, according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a child's learning toy), a gaming system, or a control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350, as well as a touchpad 355, a tactile output generator 357 (e.g., similar to tactile output generator(s) 167 described above with reference to FIG. 1A) for generating tactile output on device 300, sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and / or contact intensity sensors similar to contact intensity sensor(s) 165 described above with reference to FIG. 1A). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile semiconductor storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or a subset of, the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (FIG. 1A). Additionally, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, ​​word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, while memory 102 of portable multifunction device 100 (FIG. 1A) optionally does not store these modules.

[0135] Each of the above-identified elements in FIG. 3 is optionally stored in one or more of the memory devices mentioned above. Each of the above-identified modules corresponds to an instruction set that performs the functions described above. The above-identified modules or programs (e.g., instruction sets) need not be implemented as separate software programs, procedures, or modules, and thus, in various embodiments, these modules are optionally combined into various subsets or rearranged in other manners. In some embodiments, memory 370 optionally stores a subset of the above-identified modules and data structures. Additionally, memory 370 optionally stores additional modules and data structures not described above.

[0136] Attention is now directed to embodiments of a user interface, optionally implemented, for example, on portable multifunction device 100.

[0137] 4A shows an exemplary user interface for a menu of applications on portable multifunction device 100, according to some embodiments. A similar user interface is, optionally, implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof: ● signal strength indicator(s) 402 for wireless communication(s), such as cellular and Wi-Fi signals; ●Time 404, ●Bluetooth indicator 405, ● Battery status indicator 406, Tray 408, which contains icons for frequently used applications, such as: ○ An icon 416 for the phone module 138, labeled "Phone", optionally including an indicator 414 of the number of missed calls or voicemail messages; ○ An icon 418 for the email client module 140, labeled "Mail", optionally including an indicator 410 of the number of unread emails; ○ An icon 420 for the browser module 147, labeled "Browser"; and ○ An icon 422 for the video and music playback module 152, also referred to as the iPod (trademark of Apple Inc.) module 152, labeled "iPod"; and ●Icons for other applications, such as: ○ Icon 424 for IM module 141, labeled "Messages"; ○ An icon 426 for the calendar module 148, labeled "Calendar"; ○ Icon 428 for the image management module 144, labeled "Photos"; ○ An icon 430 for the camera module 143, labeled “camera”; ○ Icon 432 for the online video module 155, labeled "Online Video"; ○ Icon 434 for stock price widget 149-2, labeled "Stock Price" ○ Icon 436 for map module 154, labeled "Map"; ○ Icon 438 for weather widget 149-1, labeled "Weather"; ○ Icon 440 for alarm clock widget 149-4, labeled "Clock" ○ Icon 442 for Training Support Module 142, labeled "Training Support"; ○ An icon 444 for the notes module 153, labeled "Notes"; and ○ An icon 446 for a settings application or module, labeled “Settings,” that provides access to settings related to the device 100 and its various applications 136.

[0138] 4A are merely exemplary. For example, icon 422 for video and music playback module 152 is labeled "Music" or "Music Player." Other labels are optionally used for the various application icons. In some embodiments, the label for each application icon includes the name of the application that corresponds to each application icon. In some embodiments, the label of a particular application icon is different from the name of the application that corresponds to the particular application icon.

[0139] 4B shows an exemplary user interface on a device (e.g., device 300 of FIG. 3) that includes a touch-sensitive surface 451 (e.g., tablet or touchpad 355 of FIG. 3) that is separate from display 450 (e.g., touchscreen display 112). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of a contact on touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating a tactile output to a user of device 300.

[0140] Although some of the following examples are described with reference to input on touch screen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as shown in FIG. 4B. In some embodiments, this touch-sensitive surface (e.g., 451 in FIG. 4B) has a major axis (e.g., 452 in FIG. 4B) that corresponds to a major axis (e.g., 453 in FIG. 4B) on the display (e.g., 450). According to these embodiments, the device detects contact with touch-sensitive surface 451 (e.g., 460 and 462 in FIG. 4B) at locations that correspond to their respective locations on the display (e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470). In this manner, when the touch-sensitive surface is separate from the display, user input (e.g., contacts 460 and 462 and their movement) detected by the device on the touch-sensitive surface (e.g., 451 in FIG. 4B ) is used by the device to operate a user interface on the display (e.g., 450 in FIG. 4B ) of the multifunction device. It is to be understood that similar methods are optionally used for the other user interfaces described herein.

[0141] In addition, while the following examples are primarily illustrated with reference to finger input (e.g., finger contact, finger tap gesture, finger swipe gesture), it will be appreciated that in some embodiments, one or more of these finger inputs are replaced with input from another input device (e.g., mouse-based input, or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of a contact) followed by an action of moving a cursor along the path of the swipe (e.g., instead of moving the contact). As another example, a tap gesture is optionally replaced by a mouse click while the cursor is positioned over the location of the tap gesture (e.g., instead of detecting a contact followed by ceasing to detect the contact). Similarly, it will be appreciated that when multiple user inputs are detected simultaneously, multiple computer mice are optionally used simultaneously, or mouse and / or finger contacts are optionally used simultaneously.

[0142] FIG. 5A illustrates an exemplary personal electronic device 500. Device 500 comprises a body 502. In some embodiments, device 500 may include some or all of the features described with respect to devices 100 and 300 (e.g., FIGS. 1A-4B). In some embodiments, device 500 has a touch-sensitive display screen 504, hereafter touchscreen 504. Instead of, or in addition to, touchscreen 504, device 500 has a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally has one or more intensity sensors for detecting the intensity of a contact (e.g., a touch) being applied. The one or more intensity sensors of touchscreen 504 (or touch-sensitive surface) may provide output data representative of the intensity of the touch. A user interface of device 500 may respond to a touch based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface actions on device 500.

[0143] Exemplary techniques for detecting and processing touch intensity can be found, for example, in related applications International Patent Application No. PCT / US2013 / 040061, entitled "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application," filed May 8, 2013, and published as WIPO Publication No. WO / 2013 / 169849, and International Patent Application No. PCT / US2013 / 069483, entitled "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships," filed November 11, 2013, and published as WIPO Publication No. WO / 2014 / 105276, each of which is incorporated by reference herein in its entirety.

[0144] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508, if included, may be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, may allow device 500 to be attached to, for example, hats, eyewear, earrings, necklaces, shirts, jackets, bracelets, watch bands, chains, pants, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0145] FIG. 5B illustrates an exemplary personal electronic device 500. In some embodiments, the device 500 may include some or all of the components described with respect to FIGS. 1A, 1B, and 3. The device 500 has a bus 512 that operably couples an I / O section 514 to one or more computer processors 516 and a memory 518. The I / O section 514 may be connected to a display 504, which may have a touch-sensing component 522 and optionally an intensity sensor 524 (e.g., a contact intensity sensor). Additionally, the I / O section 514 may be connected to a communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular, and / or other wireless communication technologies. The device 500 may include an input mechanism 506 and / or 508. The input mechanism 506 is optionally, for example, a rotatable input device, or a depressible and rotatable input device. In some embodiments, the rotatable input device is a crown, knob, ring, or scroll wheel that is rotatable by interaction with one or more fingers. The rotatable input device may be mounted on the side of device 500 as shown in FIG. 5A, or may be mounted in other locations, such as integrated on or around display 504. Input mechanism 508 is optionally a button in some examples.

[0146] The input mechanism 508 is optionally a microphone in some embodiments. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which may be operatively connected to the I / O section 514.

[0147] The memory 518 of the personal electronic device 500 can include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 516, can cause the computer processors to perform the techniques described below, including, for example, methods 700 (FIG. 7), and 1000 (FIG. 10), 1300 (FIG. 13), and 1600 (FIG. 16). The personal electronic device 500 is not limited to the components and configuration of FIG. 5B and can include other or additional components in multiple configurations.

[0148] As used herein, the term "affordance" refers to a user-interactive graphical user interface object that is optionally displayed on a display screen of device 100, 300, and / or 500 (FIGS. 1, 3, and 5). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each, optionally, constitute an affordance.

[0149] As used herein, the term "focus selector" refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations involving a cursor or other position marker, when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B), the cursor acts as a "focus selector" when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element) and the particular user interface element is adjusted according to the detected input. In some implementations involving a touchscreen display (e.g., touch-sensitive display system 112 in FIG. 1A or touchscreen 112 in FIG. 4A) that allows direct interaction with user interface elements on the touchscreen display, a contact detected on the touchscreen acts as a "focus selector" such that when an input (e.g., a press input by a contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one region of the user interface to another region of the user interface without a corresponding movement of the cursor or movement of a contact on a touchscreen display (e.g., by using the tab or arrow keys to move focus from one button to another), and in these implementations the focus selector moves to follow the movement of focus between various regions of the user interface. Regardless of the specific form that the focus selector takes, the focus selector is generally a user interface element (or a contact on a touchscreen display) that is controlled by the user to communicate the user's intended interaction with the user interface (e.g., by indicating to the device which element of the user interface the user intends to interact with).For example, location of a focus selector (e.g., a cursor, touch, or selection box) over a corresponding button while a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that corresponding button (and not other user interface elements shown on the device's display).

[0150] As used herein and in the claims, the term "characteristic intensity" of a contact refers to a characteristic of that contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on a plurality of intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples, i.e., a set of intensity samples collected during a predefined period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) associated with a predefined event (e.g., after detecting a contact, before detecting lift-off of the contact, before or after detecting the start of movement of the contact, before detecting the end of the contact, before or after detecting an increase in the intensity of the contact, and / or before or after detecting a decrease in the intensity of the contact). The characteristic intensity of a contact is optionally based on one or more of the following: a maximum intensity of the contact, a mean value of the intensity of the contact, an average value of the intensity of the contact, the top 10 percentiles of the intensity of the contact, a half value of the maximum intensity of the contact, a 90th percentile value of the maximum intensity of the contact, etc. In some embodiments, the duration of the contact is used in determining the characteristic intensity (e.g., where the characteristic intensity is an average of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action is performed by the user. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact having a characteristic intensity that does not exceed the first threshold results in a first action being performed, a contact having a characteristic intensity that exceeds the first intensity threshold and does not exceed the second intensity threshold results in a second action being performed, and a contact having a characteristic intensity above the second threshold results in a third action being performed. In some embodiments, the comparison between the characteristic intensity and the one or more thresholds is used to determine whether to perform one or more actions (e.g., to perform each action or to forgo performing each action), rather than to determine whether to perform the first action or the second action.

[0151] In some embodiments, a portion of the gesture is identified for purposes of determining the characteristic intensity. For example, the touch-sensitive surface optionally receives a continuous swipe contact that transitions from a start position to an end position where the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end position is optionally based on only a portion of the swipe contact (e.g., only the portion of the swipe contact at the end position) rather than the entire continuous swipe contact. In some embodiments, a smoothing algorithm is optionally applied to the intensity of the swipe contact prior to determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some circumstances, these smoothing algorithms filter out small increases or decreases in the intensity of the swipe contact for purposes of determining the characteristic intensity.

[0152] The intensity of a contact on the touch-sensitive surface is optionally characterized against one or more intensity thresholds, such as a contact-detection intensity threshold, a light pressure intensity threshold, a deep pressure intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light pressure intensity threshold corresponds to an intensity that will cause the device to perform an action typically associated with clicking a physical mouse button or trackpad. In some embodiments, the deep pressure intensity threshold corresponds to an intensity that will cause the device to perform an action different from the action typically associated with clicking a physical mouse button or trackpad. In some embodiments, if a contact is detected with a characteristic intensity below the light pressure intensity threshold (e.g., and above a nominal contact-detection intensity threshold below which the contact is no longer detected), the device will move a focus selector in accordance with the movement of the contact on the touch-sensitive surface without performing an action associated with the light pressure intensity threshold or the deep pressure intensity threshold. Generally, unless otherwise specified, these intensity thresholds are consistent across different sets of user interface diagrams.

[0153] An increase in the characteristic intensity of the contact from an intensity below the light pressure intensity threshold to an intensity between the light pressure intensity threshold and the deep pressure intensity threshold may be referred to as a "light press" input. An increase in the characteristic intensity of the contact from an intensity below the deep pressure intensity threshold to an intensity above the deep pressure intensity threshold may be referred to as a "deep press" input. An increase in the characteristic intensity of the contact from an intensity below the contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light pressure intensity threshold may be referred to as detecting a contact on the touch surface. A decrease in the characteristic intensity of the contact from an intensity above the contact detection intensity threshold to an intensity below the contact detection intensity threshold may be referred to as detecting a lift-off of the contact from the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0154] In some embodiments described herein, one or more actions are performed in response to detection of a gesture including a respective pressure input or in response to detection of a respective pressure input performed on a respective contact (or multiple contacts), where the respective pressure input is detected based at least in part on detection of an increase in intensity of the contact (or multiple contacts) above a pressure input intensity threshold. In some embodiments, the respective actions are performed in response to detection of an increase in intensity of the corresponding contact (e.g., a "downstroke" of the corresponding pressure input) above a pressure input intensity threshold. In some embodiments, the pressure input includes an increase in intensity of the corresponding contact above a pressure input intensity threshold and a subsequent decrease in intensity of the contact below the pressure input intensity threshold, where the corresponding action is performed in response to detection of a subsequent decrease in intensity of the corresponding contact below the pressure input intensity threshold (e.g., an "upstroke" of the corresponding pressure input).

[0155] In some embodiments, the device employs intensity hysteresis to avoid accidental inputs, sometimes referred to as "jitter," and the device defines or selects a hysteresis intensity threshold that has a predefined relationship to the pressure input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units below the pressure input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the pressure input intensity threshold). Thus, in some embodiments, the pressure input includes an increase in the intensity of a corresponding contact above the pressure input intensity threshold and a subsequent decrease in the intensity of the contact below a hysteresis intensity threshold that corresponds to the pressure input intensity threshold, and a corresponding action is performed in response to detecting a subsequent decrease in the intensity of the corresponding contact below that hysteresis intensity threshold (e.g., an "upstroke" of the corresponding pressure input). Similarly, in some embodiments, a pressure input is detected only when the device detects an increase in the intensity of the contact from an intensity below the hysteresis intensity threshold to an intensity above the pressure input intensity threshold, and optionally a subsequent decrease in the intensity of the contact to an intensity below the hysteresis intensity, and a corresponding action is performed in response to detection of that pressure input (e.g., an increase in the intensity of the contact or a decrease in the intensity of the contact, as the case may be).

[0156] For ease of explanation, descriptions of actions performed in response to a pressure input associated with a pressure input intensity threshold or in response to a gesture including the pressure input are optionally triggered in response to detection of any of an increase in the intensity of the contact above the pressure input intensity threshold, an increase in the intensity of the contact from an intensity below a hysteresis intensity threshold to an intensity above the pressure input intensity threshold, a decrease in the intensity of the contact below the pressure input intensity threshold, and / or a decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to the pressure input intensity threshold. Additionally, in embodiments described as an action being performed in response to detection of a decrease in the intensity of the contact below a pressure input intensity threshold, the action is optionally performed in response to detection of a decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to and lower than the pressure input intensity threshold.

[0157] Attention is now directed to embodiments of user interfaces (“UI”) and associated processes implemented in an electronic device, such as portable multifunction device 100, device 300, or device 500.

[0158] 6A-6R show example user interfaces for handwriting on a small screen according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the process of FIG.

[0159] 6A illustrates electronic device 600, variations of which may include some or all of the components of devices 100, 300, or 500 described above. For example, electronic device 600 includes a display 601, which may correspond to display 340 of device 300 or display 504 of device 500. In some implementations, a touch-sensitive surface is incorporated into display 601. Electronic device 600 also includes a rotatable input device 602, which may correspond to input mechanism 506 of electronic device 500.

[0160] In FIG. 6A, the electronic device 600 displays a handwriting interface 610 on a display 601. The handwriting interface 610 includes a stroke input area 608 (represented by a dotted box) and a message area 609 (represented by a dashed box). User strokes entered into the stroke input area 608 are analyzed by one or more processors of the electronic device 600 to determine the text the strokes represent. The determined text is then displayed in the message area 609. FIG. 6A illustrates the stroke input area 608 and the message area 609 as partially overlapping. In some cases, the stroke input area 608 and the message area 609 may completely overlap, may not overlap but share a common boundary, may be separated by some distance, or may have other spatial configurations.

[0161] Handwriting interface 610 also includes an affordance 603 to indicate that the entered text is complete, an affordance 604 to cancel further text entry in the current instance of handwriting interface 610, an affordance 605 to enter a dictation mode to receive audio input, an affordance 607 to delete the entered text, and an affordance 606 to enter a space in the current text. Selecting affordance 603 may result in the text in the message area being returned to the program that launched handwriting interface 610. For example, if a messaging application launched handwriting interface 610, selecting affordance 603 may result in the text in the message area being returned to the messaging application.

[0162] Figure 6B shows a trajectory 611 that represents the trajectory of a user's finger contacting the display and stroke input area 608 of the touch-sensitive surface. The trajectory 611 includes an arrow that indicates the direction followed by the user's finger. Although Figure 6B shows the trajectory 611 caused by a user's finger, in other cases other objects, such as a stylus or other objects, are used.

[0163] Figure 6C shows the display of stroke representation 612 resulting from the response output of the touch-sensitive surface to a user's finger contacting the touch-sensitive surface along trajectory 611 of Figure 6B. Stroke representation 612 is displayed as a series of dots and partial dots. Other appearances of stroke representation 612 are possible. In some cases, the intensity of the display of stroke representation 612 is based on the time the stroke is detected.

[0164] 6D illustrates the result of the electronic device 600 analyzing the strokes corresponding to the stroke representation 612 to determine the text that represents the strokes. In this case, the strokes have been determined to represent text 614 (the letter "h") that appears in the message area 609. The stroke representation 612 is no longer displayed in the stroke input area 608. In some cases, the stroke representation that appears in the stroke input area 608 is no longer displayed after it has been analyzed to determine the corresponding text. In some cases, the stroke representation that appears in the stroke input area 608 transitions, for example by animation, to the text in the message area 609 as the corresponding strokes are being analyzed.

[0165] Figure 6E is the same as Figure 6D, except that for clarity, the boxes for the message area 609 and stroke input area 608 have been omitted. The remaining views of the electronic device 600 (Figures 6E-6R, 9A-9I, 12A-12L, and 15A-15J) do not include visual boxes indicating the message area 609 and stroke input area 608. However, the electronic device 600 as shown in these views is considered to have the same message area and stroke input area as Figures 6A-6D.

[0166] 6F illustrates handwriting interface 610 after recognizing strokes, determining text 614 based on the recognized strokes, and determining a set of one or more candidates for text 614, as indicated by indicator 616. In some cases, determining the set of one or more candidates for text 614 (or other text in a message area) is based on various factors and techniques, such as an n-gram model, the context of the message, autocorrection of possible text, autocomplete of possible text, the strokes used to generate text 614, the context of the device, or other factors.

[0167] In FIG. 6G, in response to receiving input (e.g., scroll input or rotation input) at the input device 602, the electronic device 600 displays a candidate box 620 that includes one or more candidates 622 for the text in the message area. In some cases, the one or more candidates 622 are part (or all) of a set of candidates for the text in the message area. In this case, as indicated by the text indication 618, the candidates are for the text "h", which is the only text in the message area. In some cases, as described below, the candidate box is displayed for only a portion of the text present in the message area. An indicator 624 indicates the currently selected candidate, where the initially selected candidate is the original text for which the set of one or more candidates 622 was generated. A status bar 626 indicates the position of the currently selected candidate (i.e., the position of the indicator 624) in the set of one or more candidates and is updated as the indicator 624 is moved to select a new candidate.

[0168] 6H illustrates the device 600 responding to receiving further user input (e.g., a scroll input or a rotate input) via the input device 602. The electronic device 600 moves the indicator 624 down several candidates in response to the user input. The message area 618 is updated to indicate the currently selected candidate, "hello." The status bar 626 is updated to reflect the position of the indicator 624 in the set of one or more candidates 622.

[0169] FIG. 6I illustrates the electronic device 600's response to the selection of the candidate "hello." The electronic device 600 updates the message area to include the selected candidate. In some cases, the selection occurs in response to the input device 602 no longer receiving user input (e.g., no longer receiving scrolling or rotating input). In other cases, the selection occurs in response to other user input, such as activation of a button, pressing of the input device 602, a gesture on the touch-sensitive surface, or other user input. There is an indicator 616 that indicates that a set of one or more candidates for the text in the message area is available. In response to receiving more user input (e.g., more scrolling input), the candidate box 620 and set of one or more candidates 622 (FIG. 6G) may be displayed again. In some embodiments, a new set of one or more candidates based on the new text present in the message area is generated and displayed in the candidate box.

[0170] Figure 6J illustrates handwriting interface 610 with stroke representation 628. Handwriting interface 610 is shown without a candidate indicator (e.g., indicator 616 of Figure 6I) after receiving input corresponding to the strokes that generated stroke representation 628, but before analyzing the strokes to determine corresponding text and candidates.

[0171] 6K illustrates the handwriting interface 610 after analyzing the strokes corresponding to the stroke representation 628 and determining the corresponding text. In this case, the corresponding text of the strokes corresponding to the stroke representation 628 is the letter "w" displayed in the message area. A space separates the previous text ("hello") from the new text ("w"). In some cases, the space is present in response to a user input selecting the affordance 606. In other cases, the space is present in response to the electronic device 600 automatically inserting a space after, for example, determining that "hello" is a complete word. An indicator 616 indicates that a set of one or more candidates for the text in the message area is also available.

[0172] 6L, in response to receiving input (e.g., scrolling or rotating input) via input device 602, electronic device 600 displays a candidate box 632 that includes a set of one or more candidates for the text in the message area. In this case, as indicated by text indication 630, the candidates relate to the text "w," which is the portion of the text in the message area that is highlighted by text indication 630.

[0173] 6M shows electronic device 600 displaying handwriting interface 610 after processing a number of additional strokes received on the touch-sensitive surface and determining and displaying corresponding text in the messaging area. Indicator 634 indicates that additional text is available, although not displayed. In some embodiments, the additional text is viewed and accessed in response to a user input, such as a swipe gesture, in the message area.

[0174] 6N illustrates a handwriting interface 610 displayed in response to electronic device 600 receiving user input (e.g., scroll input) via input device 602. Candidate box 636 includes a set of candidates for the text "do," as indicated by text indication 638. In addition to words beginning with "do," the set of candidates in candidate box 636 includes an emoji for a dog.

[0175] 6O shows the handwriting interface 610 after the electronic device 600 receives further user input (e.g., a scroll input) via the input device 602. The user input moves the set of candidates in the candidate box 636, such that the dog emoji becomes the current selection. The handwriting interface 610 correspondingly updates the text in the box indication 638 to the dog emoji.

[0176] 6P illustrates the handwriting interface 610 after the electronic device 600 has selected (e.g., in response to user input) a dog emoji candidate from the candidate box 636 (FIG. 6O). For example, the electronic device 600 selects the dog emoji in response to the selection indicator in the candidate box 636 identifying the dog emoji when the electronic device 600 stopped receiving user input via the input device 602. The message area is updated to include the dog emoji.

[0177] FIG. 6Q illustrates handwriting interface 610 after receiving two more strokes corresponding to stroke representations 640 and 642, respectively. The T stroke corresponding to stroke representation 640 was received earlier, as indicated by its lighter appearance compared to stroke representation 642 (corresponding to the later received stroke). If the time between the input of the strokes corresponding to stroke representations 640 and 642, respectively, is less than a threshold time, the strokes corresponding to stroke representations 640 and 642 are optionally analyzed together as part of the same set. However, the strokes are analyzed separately as part of different sets if they are separated by a certain amount of time. As shown in FIG. 6Q, the strokes corresponding to stroke representations 640 and 642, respectively, were input at substantially the same location within the stroke input area such that they overlap.

[0178] 6R illustrates handwriting interface 610 after the strokes corresponding to stroke representations 640 and 642, respectively, have been analyzed to generate the text "to." In some cases, this text is the same regardless of whether the strokes corresponding to stroke representations 640 and 642, respectively, are analyzed together or separately. In some cases, this text depends on whether the strokes corresponding to stroke representations 640 and 642, respectively, are analyzed together or separately.

[0179] 7 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface using an electronic device according to some embodiments. Method 700 is implemented in a device (e.g., 100, 300, 500) having a display, a touch-sensitive surface, and a rotatable input device. Some operations of method 700 are optionally combined, some operations are optionally reordered, and some operations are optionally omitted.

[0180] As described below, method 700 provides an intuitive method for handwriting on a touch-sensitive surface. This reduces the cognitive burden on a user for handwriting on a touch-sensitive surface and creates a more efficient human-machine interface. For battery-operated computing devices, allowing a user to input handwriting on a touch-sensitive surface faster and more efficiently conserves power and increases the time between battery charges.

[0181] The electronic device displays (702) a user input interface (e.g., 610) on the display including a message area (e.g., 609) and a stroke input area (e.g., 608) (e.g., areas of the display and touch-sensitive surface that detect and display strokes received from a user via a finger, stylus, or other object). In some embodiments, the message area (e.g., 609) and the stroke input area (e.g., 608) partially overlap. In some embodiments, the message area is entirely within the stroke input area (e.g., 608). In some embodiments, the message area and the stroke input area (e.g., 608) do not overlap but share a boundary. In some embodiments, the message area and the stroke input area (e.g., 608) are spaced apart. In some embodiments, the touch-sensitive surface is less than 2 inches by 2 inches, less than 1.5 inches by 1.5 inches, or less than 1 inch by 1 inch.

[0182] The electronic device receives (704) a first set of strokes (e.g., one or more consecutive strokes, each stroke representing a character or part of a character) (e.g., corresponding to stroke representation 612) in a stroke input area (e.g., 608) of the touch-sensitive surface. In some examples, the first set of strokes represents a character, multiple characters, a word, or multiple words. In some examples, the display provides feedback to the user of the position and shape of the strokes in the first set of strokes upon receiving the strokes (e.g., displaying stroke representation 612 in FIG. 6C ).

[0183] The electronic device determines (706) a first text (e.g., a character, multiple characters, a word, or multiple words) based on the first set of strokes (e.g., corresponding to stroke representation 612). In some examples, the determination can be further based on strokes received prior to the first set of strokes, the context of the electronic device (e.g., the time, location, current activity, calendar, person the message is directed to), or other information.

[0184] The electronic device displays (708) the first text (e.g., 614) in a message area of ​​the display (e.g., FIG. 6E). In some embodiments, the electronic device displays the first set of strokes before displaying the first text. In some embodiments, the electronic device transitions the display of the first set of strokes to the display of the first text with an animation. In some embodiments, the animation occurs while the electronic device determines the first text based on the first set of strokes.

[0185] The electronic device determines (710) one or more candidates (e.g., 622) based on the first text (e.g., 614), where the one or more candidates (e.g., 622) include one or more modifications to the first text (e.g., the modifications may realize a proposed completion, correction, or prediction). In some examples, the one or more candidates include a proposed spelling correction for the first text. In some examples, the one or more candidates include a proposed completion of the first text. In some examples, the one or more candidates include a prediction of additional words following the first text.

[0186] After the electronic device has determined one or more candidates (e.g., 622), it receives (712) user input via a rotatable input mechanism (e.g., 602). In some embodiments, the rotatable input mechanism is a crown, knob, ring, or wheel. In some embodiments, the axis of rotation is perpendicular to the plane of the display. In some embodiments, the axis of rotation is parallel to or in the plane of the display. In some embodiments, the rotatable input mechanism is at a side of the display. In some embodiments, the rotatable input mechanism is around the display.

[0187] The electronic device displays (716) at least one of the one or more candidates (e.g., candidate 622 in candidate box 620) in response to user input (714) (e.g., user input received via input device 602) and displays (718) a selection indicator (e.g., 624) indicating a selected one of the one or more candidates (e.g., 622). In some examples, at least one of the one or more candidates is displayed within a surrounding graphic element. In some examples, in response to cessation of user input immediately after displaying at least one of the one or more candidates, the electronic device ceases displaying at least one of the one or more candidates. In some examples, the selection indicator is to a side of the selected candidate. In some examples, the selection indicator surrounds the one or more candidates. In some examples, the selection indicator is integrated with the selected candidate, such as changing a color or otherwise changing an appearance of the selected candidate.

[0188] After displaying at least one of the one or more candidates (e.g., 622) and a selection indicator (e.g., 624), the electronic device replaces the display of the first text in the message area with a display of the selected candidate (720) (e.g., FIG. 6H versus FIG. 6G). In some implementations, the replacement occurs in response to the selection indicator moving to the new candidate.

[0189] According to some embodiments, in response to determining the one or more candidates, the electronic device displays an indication (e.g., 616) that one or more candidates are available. In some examples, the indication is initially displayed simultaneously with the first display of the first text. In some examples, the indication is adjacent to the rotatable input mechanism (e.g., 602), thereby guiding the user to the rotatable input mechanism and even to rotate the rotatable input mechanism in response to the display of the indication adjacent to the rotatable input mechanism (e.g., compare indicator 616 in FIG. 6F with the position and rotational orientation in FIG. 6G).

[0190] According to some embodiments, in response to receiving further user input, the electronic device determines whether a characteristic of the user input (e.g., a time since the user input stopped) satisfies a first criterion. In response to determining that a candidate interface termination criterion is met, the electronic device terminates displaying a graphical interface (e.g., 620) corresponding to one or more candidates (e.g., 622).

[0191] According to some embodiments, in response to continuing to receive user input via the rotatable input mechanism (e.g., input device 602), the electronic device moves a selection indicator (e.g., 624) from a first candidate to a second candidate of the one or more candidates (e.g., FIGS. 6G and 6H).

[0192] According to some embodiments, the display of the one or more candidates includes the first text, one or more candidate uppercase letters above the first text, and one or more candidate lowercase letters below the first text (e.g., Figures 6G, 6L, 6N).

[0193] According to some embodiments, the one or more suggestions include autocomplete suggestions (e.g., complete word suggestions based on the first text) or autocorrect suggestions (e.g., spelling or grammar corrections for the first text).

[0194] According to some embodiments, the one or more candidates include an emoji identified based on the first text (e.g., FIG. 6O). In some examples, the first text is a description of the item the emoji represents (e.g., the first text "dog" generates a dog emoji candidate, or the first text ":)" generates a smiling emoji candidate). In some examples, the first text is a more general description of a characteristic of the emoji (e.g., the first text "happy" generates a smiling emoji candidate).

[0195] According to some embodiments, the one or more candidates are determined using an n-gram model.

[0196] According to some embodiments, the electronic device receives a second set of strokes (e.g., strokes of a previous word or a previous letter of the same word) in the stroke input area before receiving the first set of strokes. The electronic device determines a second text based on the second set of strokes. Determining the one or more candidates is further based on the second text (e.g., a context of the second text is used in analyzing the first set of strokes to determine the first text).

[0197] According to some embodiments, in response to receiving at least a portion of the first set of strokes, the electronic device displays a graphical representation of the first set of strokes in the stroke input area (e.g., FIGS. 6C and 6J).

[0198] According to some embodiments, the electronic device receives a first stroke in the stroke input area. After a period of time has elapsed since receiving the first stroke, the electronic device receives a second stroke in the stroke input area. In response to a determination that the period of time exceeds a threshold, the electronic device includes the first stroke and the second stroke in a first stroke set. In response to a determination that the period of time does not exceed a threshold, the electronic device includes the first stroke in the first stroke set and excludes the second stroke from the first stroke set (e.g., FIG. 6Q).

[0199] According to some embodiments, the first text includes multiple characters (eg, Figures 6Q and 6R).

[0200] According to some embodiments, the first text is a single character (eg, FIG. 6D).

[0201] According to some embodiments, the electronic device receives a third stroke (e.g., 640) at a location within the stroke input area. The electronic device receives a fourth stroke (e.g., 642) at substantially the same location within the stroke input area, where both the third stroke and the fourth stroke are included in the first stroke set (e.g., FIG. 6Q).

[0202] According to some embodiments, the first set of strokes is a single continuous stroke (eg, 612).

[0203] According to some embodiments, the first set of strokes includes a number of individual strokes (eg, 640, 642).

[0204] According to some embodiments, the user input interface includes a plurality of affordances (e.g., 603-607), including an affordance (e.g., 605) corresponding to audio input. In response to receiving user input selecting an affordance (e.g., 605) corresponding to audio input, the electronic device displays the audio input interface for receiving dictation input.

[0205] According to some embodiments, the electronic device displays a scroll indicator (e.g., 626) along with a display of at least one of the one or more candidates (e.g., 622) and a display of a selection indicator (e.g., 624), where the size of the scroll indicator is based on the number of the one or more candidates and the position of the scroll indicator is based on the position of the selection indicator among the number of the one or more candidates (e.g., FIG. 6G versus FIG. 6H).

[0206] According to some embodiments, after replacing the display of the first text with a display of the selected candidate, the electronic device terminates the display of at least one of the one or more candidates (e.g., FIG. 6I). After terminating the display of at least one of the one or more candidates, the electronic device receives further user input (e.g., 602) via the rotatable input mechanism. In response to receiving the further user input, the electronic device displays at least a further one of the one or more candidates and displays a selection indicator (e.g., displaying the same set of candidates if user input is received again).

[0207] It should be noted that the details of the processing described above with respect to method 700 (e.g., FIG. 7) are also applicable in an analogous manner to the methods described below. For example, method 1000 (FIG. 10), method 1300 (FIG. 13), and method 1600 (FIG. 16) optionally include one or more of the features of the various methods described above with reference to method 700. For example, methods 1000, 1300, and 1600 may include method 700 as part of the text input and suggestions to complement the processing of these methods. For the sake of brevity, these details will not be repeated below.

[0208] According to some embodiments, FIG. 8 illustrates an exemplary functional block diagram of an electronic device 800 configured according to the principles of the various embodiments described. According to some embodiments, the functional blocks of the electronic device 800 are configured to execute the techniques described above. The functional blocks of the device 800 are optionally implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various embodiments described. Those skilled in the art will understand that the functional blocks described in FIG. 8 are optionally combined into sub-blocks or separated to realize the principles of the various embodiments described. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0209] 8, electronic device 800 includes a display unit 802, a touch-sensitive surface unit 804, a rotatable input mechanism unit 806, and a processing unit 808 coupled to the display unit 802, the touch-sensitive surface unit 804, and the rotatable input mechanism unit 806. In some embodiments, processing unit 808 includes a display enabling unit 810, a receiving unit 812, a determining unit 814, and an embedding unit 816.

[0210] Processing unit 808 enables (e.g., by display enabling unit 810) display of a user input interface including a message area and a stroke input area on display unit 802, receives (e.g., by accepting unit 812) a first set of strokes on the stroke input area of ​​touch-sensitive surface unit 804, determines (e.g., by determining unit 814) a first text based on the first set of strokes, enables (e.g., by display enabling unit 810) display of the first text in the message area of ​​the display unit, and determines (e.g., by determining unit 814) one or more candidates based on the first text, where the one or more candidates are the display unit 810 to determine one or more candidates, the one or more candidates including one or more modifications to the text; after determining the one or more candidates, receiving user input via the rotatable input mechanism unit 806 (e.g., by the receiving unit 812); in response to the user input, enabling (e.g., by the display enabling unit 810) a display of at least one of the one or more candidates; enabling (e.g., by the display enabling unit 810) a display of a selection indicator indicating a selected one of the one or more candidates; and after enabling the display of the at least one candidate or the one or more candidates and the selection indicator, replacing (e.g., by the display enabling unit 810) the display of the first text with the display of the selected candidate.

[0211] In some embodiments, the processing unit 808 is further configured, in response to determining the one or more candidates, to enable (e.g., by the display enablement unit 810) the display of an indication that the one or more candidates are available.

[0212] In some embodiments, the processing unit 808 is further configured to, in response to receiving the user input, determine (e.g., by the determination unit 814) whether characteristics of the user input satisfy a first criterion, and, in accordance with a determination that a candidate interface termination criterion is satisfied, terminate enabling of the display of the graphical interfaces corresponding to the one or more candidates (e.g., by the display enablement unit 810).

[0213] In some embodiments, the processing unit 808 is further configured to move the selection indicator from a first candidate to a second candidate of the one or more candidates (e.g., by the display enablement unit 810) in response to continuing to receive user input via the rotatable input mechanism unit 806.

[0214] In some embodiments, enabling the display of the one or more candidates (e.g., by display enablement unit 810) includes the first text, one or more candidates for uppercase spelling above the first text, and one or more candidates for lowercase spelling below the first text.

[0215] In some embodiments, the one or more suggestions include autocomplete suggestions or autocorrect suggestions.

[0216] In some embodiments, the one or more candidates include emojis identified based on the first text.

[0217] In some embodiments, the one or more candidates are determined using an n-gram model.

[0218] In some embodiments, the processing unit 808 is further configured to receive a second set of strokes in the stroke input area (e.g., by the receiving unit 812) before receiving the first set of strokes, and determine a second text based on the second set of strokes (e.g., by the determining unit 814), wherein determining the one or more candidates is further based on the second text.

[0219] In some embodiments, the processing unit 808 is further configured to enable (e.g., by the display enablement unit 810) display of a graphical representation of the first stroke set in the stroke input area in response to receiving at least a portion of the first stroke set.

[0220] In some embodiments, the processing unit 808 is further configured to receive (e.g., by the receiving unit 812) a first stroke in the stroke input area, receive (e.g., by the receiving unit 812) a second stroke in the stroke input area after a period of time has elapsed since receiving the first stroke, and include (e.g., by the incorporating unit 816) the first stroke and the second stroke in a first stroke set in accordance with a determination that the period of time exceeds a threshold, and include (e.g., by the incorporating unit 816) the first stroke in the first stroke set and remove the second stroke from the first stroke set in accordance with a determination that the period of time does not exceed the threshold.

[0221] In some embodiments, the first text includes a plurality of characters.

[0222] In some embodiments, the first text is a single character.

[0223] In some embodiments, the processing unit 808 is further configured to receive (e.g., by the receiving unit 812) a third stroke at a position within the stroke input area and receive (e.g., by the receiving unit 812) a fourth stroke at substantially the same position within the stroke input area, where both the third stroke and the fourth stroke are included in the first stroke set.

[0224] In some embodiments, the first set of strokes is a single contiguous stroke.

[0225] In some embodiments, the first set of strokes includes a plurality of individual strokes.

[0226] In some embodiments, the user input interface includes a plurality of affordances including an affordance corresponding to audio input, and the processing unit 808 is further configured to enable (e.g., by the display enablement unit 810) display of the audio input interface for receiving dictation input on the display unit 802 in response to receiving user input selecting the affordance corresponding to audio input.

[0227] In some embodiments, the processing unit 808 is further configured to enable (e.g., by the display enablement unit 810) display of a scroll indicator together with the display of at least one of the one or more candidates and the display of the selection indicator, wherein a size of the scroll indicator is based on the number of the one or more candidates and a position of the scroll indicator is based on a position of the selection indicator among the number of the one or more candidates.

[0228] In some embodiments, the processing unit 808 is further configured to: after replacing the display of the first text with the display of the selected candidate, terminate enabling of the display of at least one of the one or more candidates (e.g., by the display enabling unit 810); after completing the enabling of the display of the at least one of the one or more candidates (e.g., by the display enabling unit 810), receive further user input via the rotatable input mechanism unit (e.g., by the receiving unit 812); and in response to receiving the further user input, enable (e.g., by the display enabling unit 810) the display of at least a further one of the one or more candidates on the display unit 802; and enable (e.g., by the display enabling unit 810) the display of a selection indicator on the display unit 802.

[0229] The operations described above with reference to FIG. 7 are optionally implemented by the components shown in FIGS. 1A-1B or 8. For example, receiving operation 704 is optionally implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 of event sorter 170 detects a contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. Each event recognizer 180 of application 136-1 compares the event information to a respective event definition 186 to determine whether the first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as activation of an affordance on the user interface. Upon detection of the respective predefined event or sub-event, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handling unit 190 optionally utilizes or calls data update unit 176 or object update unit 177 to update application internal state 192. In some embodiments, event handling unit 190 accesses a corresponding GUI update unit 178 to update what is displayed by the application. Similarly, it will be apparent to one skilled in the art how other processes may be implemented based on the components shown in Figures 1A-1B.

[0230] 9A-9I show example user interfaces for on-screen handwriting according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the process of FIG.

[0231] Figure 9A shows an electronic device 600 displaying the handwriting interface 610 described with respect to Figures 6A-6R. The electronic device 600 has received strokes in a stroke input area corresponding to a stroke representation 900. A representation of the strokes has been displayed on the display, but analysis of the strokes corresponding to the stroke representation 900 to determine first text has not been completed.

[0232] 9B illustrates the handwriting interface 610 after the electronic device 600 has received a stroke corresponding to stroke representation 902 in the stroke input area. Stroke representation 900 is faded. In some cases, the electronic device 600 fades the representation of a stroke as a function of time since receiving the stroke, such that stroke representation 902, which corresponds to a more recently received stroke, is darker relative to stroke representation 900. The strokes corresponding to stroke representations 900 and 902 were entered in substantially the same location in the stroke input area.

[0233] Figure 9C shows handwriting interface 610 after electronic device 600 has received a stroke corresponding to stroke representation 904 in the stroke input area. Stroke representation 900 is lighter than its appearance in Figure 9B. Stroke representation 902 is also lighter. Stroke representation 904, which corresponds to the most recently received stroke, is the darkest of the three stroke representations. The strokes corresponding to stroke representations 902 and 904, respectively, were entered in substantially the same location in the stroke input area as stroke 900.

[0234] Figure 9D illustrates the handwriting interface 610 after the electronic device 600 analyzes the first set of strokes (e.g., strokes corresponding to stroke representations 900, 902, and 904, respectively), determines that the corresponding text is "neo," and displays the text in the message area. The handwriting interface 610 of Figure 9D also receives strokes corresponding to stroke representation 906 following the display of the text in the message area. In response to receiving the new strokes, the electronic device determines modified text based on the first set of strokes and the second set of strokes (strokes corresponding to stroke representation 906).

[0235] FIG. 9E illustrates the handwriting interface 610 after the electronic device 600 has analyzed the first and second sets of strokes, determined that the strokes correspond to the text "need," and replaced the previously determined text in the message area with the new corresponding text in the message area. These figures (FIGS. 9A-9E) illustrate how the electronic device can reanalyze the analyzed strokes in light of the newly received strokes. Such processing can help clarify the stroke set based on the next received stroke. The electronic device does not need to consider whether it has received all the strokes for an individual word or whether additional strokes would help it correctly analyze the strokes. FIGS. 9F-9I illustrate this concept as applied to correcting text that is a single character based on the next received stroke.

[0236] 9F illustrates the handwriting interface 610 after the electronic device 600 receives a stroke (e.g., a substantially vertical stroke) corresponding to the stroke representation 908. The electronic device 600 has not yet determined the corresponding text of the stroke corresponding to the stroke representation 908.

[0237] 9G illustrates the handwriting interface 610 after the electronic device 600 has determined a stroke set that includes a stroke that corresponds to the stroke representation 908 that corresponds to an "l" (lowercase "L"), with the corresponding text displayed in the message area.

[0238] 9H illustrates the handwriting interface 610 after the electronic device 600 has received a stroke that corresponds to the stroke representation 910. The electronic device 600 has not yet determined the corresponding text of the stroke.

[0239] 9I illustrates the handwriting interface 610 after the electronic device 600 has determined new text for the message area based on the strokes corresponding to stroke representation 908 and subsequently received the strokes corresponding to stroke representation 910. The strokes corresponding to stroke representation 910 have been determined to be dots that correspond (e.g., paired) to the substantially vertical strokes corresponding to stroke representation 908, with the "l" replaced with an "i."

[0240] The process for determining corrected text, as illustrated in Figures 9A-9E for words and Figures 9F-9I for single characters, may be based on a period of time that elapses between receiving the first set of strokes and receiving the second set of strokes. If the period of time exceeds a threshold, the second set of strokes is analyzed separately from the first set of strokes, and the text corresponding to the first set of strokes is not corrected based on the analysis of the first and second sets of strokes. If the period of time does not exceed a threshold, the second set of strokes may be analyzed together with the first set of strokes, and corrected text may be generated for the first set of strokes.

[0241] The context of the device may also be considered in determining the revised text, for example, the location of the electronic device, the time of day, the mode of the electronic device, and other factors related to the electronic device may be considered in determining the revised text.

[0242] 10 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface using an electronic device according to some embodiments. Method 1000 is implemented in a device (e.g., 100, 300, 500) having a display, a touch-sensitive surface, and a rotatable input device. Some operations of method 1000 are optionally combined, some operations are optionally reordered, and some operations are optionally omitted.

[0243] As described below, method 1000 provides an intuitive method for handwriting on a touch-sensitive surface. This reduces the cognitive burden on a user for handwriting on a touch-sensitive surface and creates a more efficient human-machine interface. For battery-operated computing devices, allowing a user to input handwriting on a touch-sensitive surface faster and more efficiently conserves power and increases the time between battery charges.

[0244] The electronic device displays (1002) a user input interface on a display including a message area (e.g., 609) and a stroke input area (e.g., 608) (e.g., 610) (e.g., areas of the display and touch-sensitive surface that detect and display strokes received from a user via a finger, stylus, or other object). In some embodiments, the message area and the stroke input area partially overlap. In some embodiments, the message area is completely within the stroke input area. In some embodiments, the message area and the stroke input area do not overlap but share a boundary. In some embodiments, the message area and the stroke input area are spaced apart. In some embodiments, the touch-sensitive surface is less than 2 inches by 2 inches, less than 1.5 inches by 1.5 inches, or less than 1 inch by 1 inch.

[0245] The electronic device receives (1004) a first set of strokes (e.g., 900, 902, 904) (e.g., one or more consecutive strokes, each stroke representing a character or part of a character) in a stroke input area (e.g., 608) of the touch-sensitive surface. In some embodiments, the first set of strokes represents a character, multiple characters, a word, or multiple words. In some embodiments, the display provides feedback to the user of the position and shape of the strokes in the first set of strokes as they are received (e.g., displaying strokes 900, 902, 904 in FIG. 9C ). In some embodiments, the display of the first set of strokes is based on the time since each stroke was received, such as by fading the strokes over time (e.g., displaying strokes 900, 902, 904 in FIG. 9C ).

[0246] The electronic device determines (1006) a first text (e.g., a character, multiple characters, a word, or multiple words) based on the first set of strokes (e.g., strokes 900, 902, 904). In some examples, the determination may be further based on strokes received prior to the first set of strokes. According to some embodiments, the electronic device determines the modified first text based on a current context of the electronic device (e.g., time, location, current activity, calendar, person to whom the message is directed) at the time of determining the modified first text.

[0247] The electronic device displays (1008) the first text (e.g., 905) in a message area of ​​the display (e.g., FIG. 9D). In some embodiments, the electronic device displays the first set of strokes before displaying the first text (e.g., FIG. 9C). In some embodiments, the electronic device transitions the display of the first set of strokes to the display of the first text with an animation. In some embodiments, the animation occurs while the electronic device determines the first text based on the first set of strokes.

[0248] The electronic device receives (1010) a second set of strokes (e.g., 906) in a stroke input area of ​​the touch-sensitive surface after receiving a first set of strokes (e.g., strokes 900, 902, 904) and displaying a first text (e.g., 905). In some examples, the second set of strokes represents a portion of a character, a character, multiple characters, a word, or multiple words. According to some embodiments, the electronic device displays the first text before receiving the second set of strokes.

[0249] The electronic device determines (1012) a modified first text (e.g., 907) based on the first set of strokes (e.g., 900, 902, 904) and the second set of strokes (e.g., 906). According to some embodiments, the second set of strokes is received a period of time after receiving the first set of strokes, and determining the modified first text based on the first set of strokes and the second set of strokes is dependent on determining that the period of time is less than a threshold.

[0250] The electronic device replaces the display of the first text (e.g., FIG. 9D ) with the modified first text (1014) (e.g., FIG. 9E ). According to some embodiments, the electronic device receives a third set of strokes on the stroke input area of ​​the touch-sensitive surface after receiving the second set of strokes and after displaying the modified first text. The electronic device determines the second text based on the third set of strokes. The electronic device displays the second text along with the modified first text.

[0251] According to some embodiments, the first set of strokes is a single continuous stroke and the first text is a single character (eg, FIG. 9F or FIG. 9G).

[0252] According to some embodiments, the second set of strokes is a single continuous stroke (eg, FIG. 9D or FIG. 9H).

[0253] According to some embodiments, the corrected first text differs from the first text by only a single character (eg, compare FIG. 9G with FIG. 9I).

[0254] According to some embodiments, the corrected first text differs from the first text only in the last character of the corrected first text or the first text (e.g., compare Figure 9G with Figure 9I).

[0255] According to some embodiments, FIG. 11 illustrates an exemplary functional block diagram of an electronic device 1100 configured according to the principles of the various embodiments described. According to some embodiments, the functional blocks of the electronic device 1100 are configured to execute the techniques described above. The functional blocks of the device 1100 are optionally implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various embodiments described. Those skilled in the art will understand that the functional blocks described in FIG. 11 are optionally combined into sub-blocks or separated to realize the principles of the various embodiments described. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0256] 11, electronic device 1100 includes a display unit 1102, a touch-sensitive surface unit 1104, a rotatable input mechanism unit 1106, and a processing unit 1108 coupled to the display unit 1102, the touch-sensitive surface unit 1104, and the rotatable input mechanism unit 1106. In some embodiments, processing unit 1108 includes a display enabling unit 1110, a receiving unit 1112, and a determining unit 814.

[0257] The processing unit 1108 is configured to enable (e.g., by display enabling) display of a user input interface including a message area and a stroke input area on the display unit 1102, receive (e.g., by the receiving unit 1112) a first set of strokes in the stroke input area of ​​the touch-sensitive surface unit, determine (e.g., by the determining unit 1114) a first text based on the first set of strokes, enable (e.g., by the display enabling unit 1110) display of the first text in the message area, receive (e.g., by the receiving unit 1112) a second set of strokes in the stroke input area of ​​the touch-sensitive surface unit, determine (e.g., by the determining unit 1114) a modified first text based on the first set of strokes and the second set of strokes, and replace (e.g., by the display enabling unit 1110) the display of the first text with the modified first text.

[0258] In some embodiments, the second set of strokes is received a period of time after receiving the first set of strokes, and determining the modified first text based on the first set of strokes and the second set of strokes is dependent on determining that the period of time is less than a threshold.

[0259] In some embodiments, enabling the display of the first text occurs before receiving the second set of strokes.

[0260] In some embodiments, determining the modified first text is further based on a current context of the portable electronic device at the time of determining the modified first text.

[0261] In some embodiments, after receiving the second set of strokes and after enabling display of the modified first text, the processing unit 1108 is further configured to: receive a third set of strokes in the stroke input area of ​​the touch-sensitive surface (e.g., by the receiving unit 1112), determine (e.g., by the determining unit 1114) the second text based on the third set of strokes, and enable display of the second text together with the modified first text (e.g., by the display enabling unit 1110).

[0262] In some embodiments, the first set of strokes is a single contiguous stroke and the first text is a single character.

[0263] In some embodiments, the second set of strokes is a single contiguous stroke.

[0264] In some embodiments, the modified first text differs from the first text by only a single character.

[0265] In some embodiments, the modified first text differs from the first text only in the last character of the modified first text or the first text.

[0266] The operations described above with reference to FIG. 10 are optionally implemented by the components shown in FIGS. 1A-1B or 11. For example, receiving operation 1004 is optionally implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 of event sorter 170 detects a contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. Each event recognizer 180 of application 136-1 compares the event information to a respective event definition 186 to determine whether the first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as activation of an affordance on the user interface. Upon detection of the respective predefined event or sub-event, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally utilizes or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses respective GUI updater 178 to update what is displayed by the application. Similarly, it will be clear to one skilled in the art how other processes may be implemented based on the components shown in Figures 1A-1B.

[0267] 12A-12L show example user interfaces for on-screen handwriting according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the process of FIG.

[0268] 12A-12H illustrate the steps of using electronic device 600 to input the word "going" with multiple strokes onto a touch-sensitive surface, while electronic device 600 processes the strokes completely as they are available. In contrast, FIGS. 12I-12L illustrate the steps of using electronic device 600 to input the word "going" with multiple strokes onto a touch-sensitive surface, while electronic device 600 processes the strokes as they are available, but suspends analysis of the previous stroke and resumes analysis with the new stroke when it becomes available. In some cases, this latter process provides benefits over the former process.

[0269] FIG. 12A illustrates the handwriting interface 610 after the electronic device 600 has received a stroke 1200. FIG. 12B illustrates the result (text 1201) of the electronic device 600 analyzing the strokes corresponding to the stroke representation 1200. FIG. 12C illustrates the handwriting interface 610 after the electronic device 600 has received a stroke corresponding to the stroke representation 1202 and a stroke corresponding to the stroke representation 1204. FIG. 12D illustrates the handwriting interface 610 after the electronic device 600 has started processing the strokes corresponding to the stroke representations 1202 and 1204, respectively, but before determining the text corresponding to the strokes. In FIG. 12D, the fact that the electronic device 600 is processing the strokes corresponding to the stroke representations 1202 and 1204, respectively, is indicated by no longer displaying the stroke representations. However, this behavior is exemplary and does not necessarily apply in all cases. FIG. 12D illustrates that the electronic device 600 has also received a stroke 1206. That is, the electronic device 600 receives the stroke corresponding to stroke representation 1206 after it has begun processing the strokes corresponding to stroke representations 1202 and 1204, respectively, but before it has determined the stroke's corresponding text. FIG. 12E illustrates the electronic device 600 after it has completed analysis of the strokes corresponding to stroke representations 1202 and 1204, respectively, and has begun analysis of the strokes corresponding to stroke representation 1206, but before it has completed analysis of the strokes corresponding to stroke representation 1206. FIG. 12F illustrates the electronic device 600 after it has completed analysis of the strokes corresponding to stroke representation 1206. Finally, in FIG. 12G, the electronic device 600 receives and processes the last stroke (the stroke corresponding to stroke representation 1208). The result is shown in FIG. 12H as text 1201. Because the electronic device 600 completed processing the stroke when it became available, the text 1201 is incorrect because the electronic device 600 did not have all of the complete context of the stroke when it became available.Even if the electronic device 600 had arrived at the correct text in FIG. 12H, it would have performed extra or redundant processing along the way because it determined the text without access to the full context of all of the strokes.

[0270] In contrast, in Figures 12I-12L, the same strokes are entered and processing begins when they are available. However, the electronic device 600 stops processing the current set of strokes each time a new stroke becomes available. For example, Figure 12I illustrates that the electronic device 600 has received two strokes, namely, strokes corresponding to stroke representations 1200 and 1202, respectively. Figure 12J illustrates that the electronic device 600 has begun processing the strokes corresponding to stroke representations 1200 and 1202, respectively (e.g., as indicated by no longer displaying stroke representations 1200 and 1202, although this is not necessarily true in all embodiments), but has not yet determined the corresponding text when it receives strokes corresponding to stroke representations 1204 and 1206, respectively. In this case, in response to receiving the strokes corresponding to stroke representations 1204 and 1206, respectively (or even just the initially entered stroke corresponding to stroke representation 1204), electronic device 600 stops processing the stroke sets corresponding to stroke representations 1200 and 1202, adds the strokes corresponding to stroke representations 1204 and 1206, respectively, to the stroke set, and continues processing the stroke set. FIG. 12K illustrates the last stroke, i.e., the stroke corresponding to stroke representation 1208, being entered. As with the input of strokes corresponding to stroke representations 1204 and 1206, respectively, electronic device 600 is still processing the stroke set when it receives stroke 1208, so stops processing, adds the stroke corresponding to stroke representation 1208 to the stroke set, and continues processing the stroke set. In some cases, electronic device 600 resets / restarts its analysis when it adds a new stroke to the stroke set, but this is not necessarily true in all embodiments.

[0271] 13 is a flow diagram illustrating a method for handwriting on a touch-sensitive surface using an electronic device according to some embodiments. Method 1300 is implemented in a device (e.g., 100, 300, 500) having a display, a touch-sensitive surface, and a rotatable input device. Some operations of method 1300 are optionally combined, some operations are optionally reordered, and some operations are optionally omitted.

[0272] As described below, method 1300 provides an intuitive method for handwriting on a touch-sensitive surface. This reduces the cognitive burden on a user for handwriting on a touch-sensitive surface and creates a more efficient human-machine interface. For battery-operated computing devices, allowing a user to input handwriting on a touch-sensitive surface faster and more efficiently conserves power and increases the time between battery charges.

[0273] The electronic device displays (1302) on the display a user input interface (e.g., 610) including a message area (e.g., 609) and a stroke input area (e.g., 608) (e.g., areas of the display and touch-sensitive surface that detect and display strokes received from a user via a finger, stylus, or other object). In some embodiments, the message area and the stroke input area partially overlap. In some embodiments, the message area is completely within the stroke input area. In some embodiments, the message area and the stroke input area do not overlap but share a boundary. In some embodiments, the message area and the stroke input area are spaced apart. In some embodiments, the touch-sensitive surface is less than 2 inches by 2 inches, less than 1.5 inches by 1.5 inches, or less than 1 inch by 1 inch.

[0274] The electronic device receives (1304) a first set of strokes (e.g., 1200, 1202, 1204) (e.g., one or more consecutive strokes, each stroke representing a character or part of a character) in a stroke input area (e.g., 608) of the touch-sensitive surface. In some examples, the first set of strokes represents a character, multiple characters, a word, or multiple words. In some examples, the display provides feedback to the user of the position and shape of the strokes in the first set of strokes (e.g., a display of strokes 1200, 1202, 1204) upon receiving the strokes.

[0275] The electronic device begins to determine (1306) a first text (e.g., a character, multiple characters, a word, or multiple words) based on the first set of strokes (e.g., strokes 1200, 1202, 1204). In some examples, the determination can be further based on strokes received prior to the first set of strokes, the context of the electronic device (e.g., the time, location, current activity, calendar, person the message is directed to), or other information.

[0276] The electronic device receives (1308) (e.g., FIG. 12J ) a second set of strokes (e.g., 1206) (e.g., one or more consecutive strokes, each stroke representing a character or part of a character) in a stroke input area (e.g., 608) of the touch-sensitive surface without displaying the first text in the message area. In some examples, the second set of strokes represents a part of a character, a character, multiple characters, a word, or multiple words. In some examples, the display provides the user with feedback of the position and shape of the strokes in the first set of strokes (e.g., a display of strokes 1206) upon receiving the strokes. According to some embodiments, the electronic device determines the first text based on the first set of strokes before receiving the second set of strokes.

[0277] The electronic device determines modified first text (eg, text 1208) based on the first set of strokes and the second set of strokes.

[0278] The electronic device displays the modified first text (eg, text 1208) in the message area.

[0279] According to some embodiments, the corrected first text includes a first character, where the first character is not based on the second set of strokes (e.g., the first character is a “G” and is not based on stroke 1206).

[0280] According to some embodiments, the modified first text includes multiple characters, and the multiple characters are based on the first stroke set (e.g., the text includes “Go” based on the first stroke set 1200 and 1202).

[0281] According to some embodiments, the corrected first text includes a second character, where the second character is not based on the first stroke set (e.g., the second character is an "i" that is not based on the first stroke set, strokes 1200 and 1202.

[0282] According to some embodiments, the corrected first text is a single word (eg, text 1208).

[0283] According to some embodiments, the first set of strokes (e.g., 1200 and 1202) are entered into the stroke input area at substantially the same location as the second set of strokes (e.g., 1206 or 1208) were entered.

[0284] According to some embodiments, the corrected first text includes a first portion based on the shape of the first set of strokes and the context provided by the second set of strokes (e.g., the first set of strokes is the shape of an "n" or an "h", but from the shape it is not clear which letter is correct, and the corresponding text of the second set of strokes is "ight", so the context providing the second strokes results in a determination that the corresponding letter is "n").

[0285] According to some embodiments, the corrected first text includes a first portion based on the shape of the second set of strokes and the context provided by the first set of strokes (e.g., the second set of strokes is the shape of an "n" or an "h", but from the shape it is not clear which letter is correct, and the corresponding text of the first set of strokes is "tur", so the context providing the first strokes results in a determination that the corresponding letter is "n").

[0286] According to some embodiments, FIG. 14 illustrates an exemplary functional block diagram of an electronic device 1400 configured according to the principles of the various embodiments described. According to some embodiments, the functional blocks of the electronic device 1400 are configured to execute the techniques described above. The functional blocks of the device 1400 are optionally implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various embodiments described. Those skilled in the art will understand that the functional blocks described in FIG. 14 are optionally combined into sub-blocks or separated to realize the principles of the various embodiments described. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0287] 14, electronic device 1400 includes a display unit 1402, a touch-sensitive surface unit 1404, a rotatable input mechanism unit 1406, and a processing unit 1408 coupled to the display unit 1402, the touch-sensitive surface unit 1404, and the rotatable input mechanism unit 1406. In some embodiments, processing unit 1408 includes a display enabling unit 1410, a receiving unit 1412, and a determining unit 1414.

[0288] The processing unit 1408 is configured to enable display of a user input interface including a message area and a stroke input area on the display 1402 (e.g., by the display enabling unit 1410), receive a first set of strokes in the stroke input area of ​​the touch-sensitive surface (e.g., by the receiving unit 1412), initiate determining a first text based on the first set of strokes (e.g., by the determining unit 1414), without enabling display of the first text in the message area after receiving the first set of strokes, receive a second set of strokes in the stroke input area of ​​the touch-sensitive surface (e.g., by the receiving unit 1412), determine a modified first text based on the first set of strokes and the second set of strokes (e.g., by the determining unit 1414), and enable display of the modified first text in the message area (e.g., by the display enabling unit 1410).

[0289] In some embodiments, the modified first text includes a first character, where the first character is not based on the second set of strokes.

[0290] In some embodiments, the modified first text includes a plurality of characters, the plurality of characters being based on the first set of strokes.

[0291] In some embodiments, the modified first text includes a second character, where the second character is not based on the first stroke set.

[0292] In some embodiments, the modified first text is a single character.

[0293] In some embodiments, the corrected first text is a single word.

[0294] In some embodiments, the first set of strokes is entered into the stroke input area at substantially the same location as the second set of strokes is entered.

[0295] In some embodiments, the processing unit 1408 is further configured to determine (eg, by the determining unit 1414) the first text based on the first set of strokes before receiving the second set of strokes.

[0296] In some embodiments, the modified first text includes a first portion based on the shape of the first set of strokes and the context provided by the second set of strokes.

[0297] In some embodiments, the modified first text includes a first portion based on the shape of the second set of strokes and the context provided by the first set of strokes.

[0298] The operations described above with reference to FIG. 13 are optionally implemented by the components shown in FIGS. 1A-1B or 14. For example, receiving operation 1304 is optionally implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 of event sorter 170 detects a contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. Each event recognizer 180 of application 136-1 compares the event information to a respective event definition 186 to determine whether the first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as activation of an affordance on the user interface. Upon detection of the respective predefined event or sub-event, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handling unit 190 optionally utilizes or calls data update unit 176 or object update unit 177 to update application internal state 192. In some embodiments, event handling unit 190 accesses a corresponding GUI update unit 178 to update what is displayed by the application. Similarly, it will be apparent to one skilled in the art how other processes may be implemented based on the components shown in Figures 1A-1B.

[0299] 15A-15J show example user interfaces for on-screen handwriting according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the process of FIG.

[0300] Figure 15A illustrates handwriting interface 610 after electronic device 600 has received a stroke set corresponding to stroke set representation 1502, including strokes corresponding to stroke representation 1503 and stroke representation 1504, which are the last two strokes received in Figure 15A. Stroke set representation 1502 represents a logographic character (e.g., a simplified Chinese character). Because overlapping logographic characters include many strokes and are difficult to analyze (e.g., see Figure 15E for an example of only two overlapping logographic characters), electronic device 600 determines the corresponding logographic character based on the time between the received strokes when all strokes for the logographic character have been received.

[0301] For the strokes corresponding to stroke representations 1503 and 1504, respectively, it is determined whether the time between the strokes (e.g., from when the stroke corresponding to stroke representation 1503 is completed to when the stroke corresponding to stroke representation 1504 begins) exceeds a threshold. If the time does not exceed the threshold, the strokes corresponding to stroke representations 1503 and 1504, respectively, are included together in the same set, as shown in FIG.

[0302] If the time between the next stroke (i.e., the stroke corresponding to stroke representation 1506 in FIG. 15B) and the stroke corresponding to stroke representation 1504 exceeds a threshold, then the stroke set corresponding to stroke set representation 1502 is analyzed together as a set, without the stroke corresponding to stroke representation 1506, to determine a corresponding logographic character 1508. The stroke corresponding to stroke representation 1506 is added to the stroke set (the stroke set corresponding to stroke set representation 1510 in FIG. 15C) for the next character shown in FIG. 15C. This method, which is also described with respect to FIG. 16, enables the electronic device 600 to conserve battery power by avoiding simultaneous processing of multiple logographic characters.

[0303] This technique can also be used to disambiguate strokes that represent phonetic as well as logographic characters. For example, FIG. 15F shows handwriting interface 610 after electronic device 600 has received text 1514 and strokes corresponding to stroke representation 1516. In FIG. 15G, the next stroke is received, the stroke corresponding to stroke representation 1518. Based on the time between the stroke corresponding to stroke representation 1516 and the stroke corresponding to stroke representation 1518, electronic device 600 can determine whether the strokes corresponding to stroke representations 1516 and 1518, respectively, are intended to be the letter "k" to spell the word "cook" (text 1516 of FIG. 15H), or whether the strokes corresponding to stroke representations 1516 and 1518, respectively, are two separate characters and are intended to spell "cool" (text 1520 of FIG. 15J).

[0304] If the time between strokes exceeds a threshold, the electronic device 600 treats the strokes corresponding to stroke representations 1516 and 1518 as parts of different characters, as shown in Figure 15I. The strokes corresponding to stroke representation 1518 may be analyzed separately to generate, for example, the letter "c" as part of the word "car" (text 1518 in Figure 15H). If the time between strokes does not exceed a threshold, strokes 1516 and 1518 are treated as parts of the same character, as shown in Figure 15J.

[0305] 16 is a flow diagram illustrating a method for writing on a touch-sensitive surface using an electronic device according to some embodiments. Method 1600 is implemented in a device (e.g., 100, 300, 500) having a display, a touch-sensitive surface, and a rotatable input device. Some operations of method 1600 are optionally combined, some operations are optionally reordered, and some operations are optionally omitted.

[0306] As described below, method 1600 provides an intuitive method for handwriting on a touch-sensitive surface. This reduces the cognitive burden on a user for handwriting on a touch-sensitive surface and creates a more efficient human-machine interface. For battery-operated computing devices, allowing a user to input handwriting on a touch-sensitive surface faster and more efficiently conserves power and increases the time between battery charges.

[0307] The electronic device displays (1602) on the display a user input interface (e.g., 610) including a message area (e.g., 609) and a stroke input area (e.g., 608) (e.g., areas of the display and touch-sensitive surface that detect and display strokes received from a user via a finger, stylus, or other object). In some embodiments, the message area and the stroke input area partially overlap. In some embodiments, the message area is completely within the stroke input area. In some embodiments, the message area and the stroke input area do not overlap but share a boundary. In some embodiments, the message area and the stroke input area are spaced apart. In some embodiments, the touch-sensitive surface is less than 2 inches by 2 inches, less than 1.5 inches by 1.5 inches, or less than 1 inch by 1 inch.

[0308] The electronic device receives 1604 a first stroke in a stroke input area of ​​the touch-sensitive surface, such as stroke 1504. In some embodiments, the first stroke is the most recent of the multiple strokes.

[0309] The electronic device receives (1606) a second stroke (e.g., 1506) on the touch-sensitive surface at a first time after receiving the first stroke, the second stroke being different from the first stroke (e.g., the first stroke and the second stroke are not the same or different portions of the same stroke).

[0310] The electronic device determines (1608) whether the first time exceeds a threshold time (eg, whether the time from the end of stroke 1504 to the beginning of stroke 1506 exceeds a threshold).

[0311] In response to determining that the first time exceeds the threshold time, the electronic device determines (1610) a first character (e.g., character 1508) based on the first stroke (e.g., 1504) but not the second stroke (e.g., 1506).

[0312] In response to the electronic device determining that the first time (e.g., using stroke 1503 as a first stroke and stroke 1504 as a second stroke) is less than a threshold time, the electronic device determines (1612) a first character (e.g., character 1508) based on the first stroke (e.g., 1503) and the second stroke (e.g., 1504) (e.g., in addition to other strokes in stroke set 1502).

[0313] The electronic device displays (1614) the first character (eg, character 1508) in the message area.

[0314] According to some embodiments, the electronic device further determines a second character (e.g., character 1512) based on the second stroke (e.g., 1506) in accordance with determining that the first time exceeds a threshold time. The electronic device displays the second character (e.g., character 1512) in the message area along with the first character.

[0315] According to some embodiments, the respective representations of the first character and the second character are separated by a space (eg, FIG. 15I).

[0316] According to some embodiments, the first character is a phonetic character (eg, Figures 15H and 15J).

[0317] According to some embodiments, the first character is an ideographic character (eg, character 1508).

[0318] According to some embodiments, FIG. 17 illustrates an exemplary functional block diagram of an electronic device 1700 configured according to the principles of the various embodiments described. According to some embodiments, the functional blocks of the electronic device 1700 are configured to execute the techniques described above. The functional blocks of the device 1700 are optionally implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various embodiments described. Those skilled in the art will understand that the functional blocks described in FIG. 17 are optionally combined into sub-blocks or separated to realize the principles of the various embodiments described. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0319] 17, electronic device 1700 includes a display unit 1702, a touch-sensitive surface unit 1704, a rotatable input mechanism 1706, and a processing unit 1708 coupled to the display unit 1702, the touch-sensitive surface unit 1704, and the rotatable input mechanism 1706. In some embodiments, processing unit 1708 includes a display enabling unit 1710, a receiving unit 1712, and a determining unit 1714.

[0320] Processing unit 1708 enables (e.g., by display enablement unit 1710) display of a user input interface including a message area and a stroke input area on display unit 1702; receives (e.g., by receiving unit 1712) a first stroke on the stroke input area of ​​touch-sensitive surface unit 1704; receives (e.g., by receiving unit 1712) a second stroke on touch-sensitive surface unit 1704 at a first time after receiving the first stroke, where the second stroke is different from the first stroke; The display device is configured to: determine (e.g., by the determining unit 1714) whether the first time exceeds a threshold time; determine (e.g., by the determining unit 1714) a first character based on the first stroke but not the second stroke according to a determination that the first time exceeds the threshold time; determine (e.g., by the determining unit 1714) a first character based on the first stroke and the second stroke according to a determination that the first time is less than the threshold time; and enable display of the first character in the message area (e.g., by the display enabling unit 1710).

[0321] In some embodiments, the processing unit 1708 is further configured to, in accordance with determining that the first time exceeds a threshold time, determine (e.g., by the determining unit 1714) a second character based on the second stroke, and enable (e.g., by the display enabling unit 1710) display of the second character together with the first character in the message area.

[0322] In some embodiments, each appearance of the first character and the second character is separated by a space.

[0323] In some embodiments, the processing unit 1708 is further configured to determine (e.g., by the determining unit 1714) a context associated with the first character and to determine (e.g., by the determining unit 1714) a second character further based on the context.

[0324] In some embodiments the first character is a phonetic character.

[0325] In some embodiments, the first character is an ideographic character.

[0326] The operations described above with reference to FIG. 16 are optionally performed by the components shown in FIGS. 1A-1B or 17. For example, receiving operation 1604 is optionally performed by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 of event sorter 170 detects a contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. Each event recognizer 180 of application 136-1 compares the event information to a respective event definition 186 to determine whether the first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as activation of an affordance on the user interface. Upon detection of the respective predefined event or sub-event, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handling unit 190 optionally utilizes or calls data update unit 176 or object update unit 177 to update application internal state 192. In some embodiments, event handling unit 190 accesses a corresponding GUI update unit 178 to update what is displayed by the application. Similarly, it will be apparent to one skilled in the art how other processes may be implemented based on the components shown in Figures 1A-1B.

[0327] The above description has been set forth in terms of specific embodiments for purposes of illustration. However, the illustrative description above is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of the technology and their practical application so as to enable others skilled in the art to best utilize the technology and various embodiments with various modifications suited to the particular use contemplated.

[0328] Although the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art, and such changes and modifications should be understood to be included within the scope of the present disclosure and examples as defined by the claims.

Claims

1. 1. An electronic device having one or more processors, a touch-sensitive surface, and a display, comprising: displaying a user input interface on the display, the user input interface including a message area and a stroke input area; receiving a first set of strokes in the stroke input area on the touch-sensitive surface; determining a first text based on the first set of strokes; receiving a second set of strokes in the stroke input area on the touch-sensitive surface a predetermined time after receiving the first set of strokes without displaying the first text in the message area; in response to a determination that a time since receipt of the first stroke set is below a threshold; determining a modified first text based on the first set of strokes and the second set of strokes; displaying the modified first text in the message area; in response to a determination that a time since receiving the first set of strokes exceeds a threshold; determining a second text based on the second set of strokes independent of the first set of strokes; displaying the first text in the message area distinct from the second text; The method includes:

2. The method of claim 1 , wherein the modified first text includes a first character, the first character not based on the second stroke set.

3. The method of claim 1 , wherein the modified first text includes a plurality of characters, the plurality of characters being based on the first set of strokes.

4. The method of claim 1 , wherein the modified first text includes a second character, the second character not based on the first stroke set.

5. The method of claim 1 , 2 or 4 , wherein the modified first text is a single character.

6. The method of claim 1 or 3, wherein the corrected first text is a single word.

7. The method of claim 1 , wherein the first set of strokes is input at a position on the stroke input area that is substantially the same as a position on the stroke input area at which the second set of strokes is input.

8. The method of claim 1 , further comprising determining a first text based on the first set of strokes prior to accepting the second set of strokes.

9. 9. The method of claim 1, wherein the modified first text includes a first portion based on a shape of the first set of strokes and a context provided by the second set of strokes.

10. 9. The method of claim 1, wherein the modified first text includes a first portion based on a shape of the second set of strokes and a context provided by the first set of strokes.

11. A computer program comprising instructions that, when executed by one or more processors of an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform the method of any one of claims 1 to 10.

12. 1. An electronic device comprising: A display and a touch-sensitive surface; and one or more processors; 11. An electronic device, comprising: a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 1 to 10.

13. A display and a touch-sensitive surface; and Means for carrying out the method according to any one of claims 1 to 10; 1. An electronic device comprising:

Images