Device, method, and graphical user interface for annotating content

Abstract

The present disclosure relates to devices, methods, and graphical user interfaces for annotating content. According to some embodiments, a method is disclosed herein as being performed on a device having one or more processors, non-transitory memory, a display, and an input device. The method includes displaying, on the display, a drawing user interface that includes a plurality of drawing objects. When a drawing tool associated with a predefined operation is selected, the method includes detecting, via the one or more input devices, user input that moves to define a path within the drawing user interface. In response to detecting the user input that moves to define the path within the drawing user interface, the method includes performing the predefined operation with respect to two or more of the plurality of drawing objects that intersect the path.

Description

[0001] This application is a divisional application of invention patent application No. 201880033271.3, filed on March 21, 2018, entitled "Apparatus, Method and Graphical User Interface for Annotating Content". Technical Field

[0002] This disclosure relates in its entirety to electronic devices having touch-sensitive surfaces, including but not limited to electronic devices having touch-sensitive surfaces that allow users to annotate content. Background Technology

[0003] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary touch-sensitive surfaces include touchpads and touchscreen displays. Such surfaces are widely used to manipulate user interface objects on the display. Such manipulation includes adjusting the position and / or size of one or more user interface objects, activating buttons or opening files / applications represented by user interface objects, as well as associating metadata with one or more user interface objects or otherwise manipulating the user interface. Exemplary user interface objects include digital images, videos, text, icons, control elements (such as buttons), and other graphics. In some cases, users will need to perform such manipulations on user interface objects from the following categories: file management applications (e.g., Finder from Apple Inc., Cupertino, California); image management applications (e.g., Aperture, iPhoto, Photos from Apple Inc., California); digital content (e.g., video and music) management applications (e.g., iTunes from Apple Inc., California); drawing applications; presentation applications (e.g., Keynote from Apple Inc., California); word processing applications (e.g., Pages from Apple Inc., California); website creation applications (e.g., iWeb from Apple Inc., California); disc editing applications (e.g., iDVD from Apple Inc., California); or spreadsheet applications (e.g., Numbers from Apple Inc., California).

[0004] Some user interfaces display content and allow users to annotate it, for example, by adding drawings or handwritten text to various parts of the content. However, the methods used to add annotations to the content are cumbersome and inefficient. For example, using a sequence of mouse-based input to select one or more user interface objects and perform one or more actions on the selected objects is monotonous and places a significant cognitive burden on the user. Furthermore, these methods take longer than necessary, thus wasting energy. This latter consideration is particularly important in battery-powered devices. Summary of the Invention

[0005] Therefore, electronic devices require faster and more efficient methods and interfaces for annotating content. Such methods and interfaces optionally supplement or replace conventional methods for annotating content. They reduce the cognitive burden on users and create more efficient human-machine interfaces. For battery-powered devices, such methods and interfaces can save power and increase the time between battery charges.

[0006] The disclosed device can reduce or eliminate the aforementioned defects and other problems associated with the user interface of electronic devices with touch-sensitive surfaces. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, tablet, or handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also referred to as a "touchscreen" or "touchscreen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, and a program or set of instructions stored in memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, these functions optionally include image editing, drawing, rendering, word processing, web page creation, disk editing, spreadsheet creation, playing games, making and receiving phone calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, and / or digital video playback. Executable instructions for performing these functions are optionally included in a nontransitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0007] According to some embodiments, the present invention discloses performing a method on a device having non-transitory memory, a display, an input device, and one or more processors. The method includes displaying a drawing user interface comprising a plurality of drawing objects on the display. When a drawing tool associated with a predefined operation is selected, the method includes detecting user input via one or more input devices to define a path within the drawing user interface. In response to detecting user input to define a path within the drawing user interface, the method includes performing a predefined operation relative to two or more of the plurality of drawing objects intersecting the path.

[0008] According to some embodiments, the present invention discloses performing a method on a device having non-transitory memory, a display, an input device, and one or more processors. The method includes displaying on the display the first content and a representation of the second content associated with the first content. The method includes detecting user input at a location of the representation of the second content via one or more input devices. In response to detecting user input at the location of the representation of the second content, and based on determining that the user input includes stylus contact, the method includes displaying on the display a drawing user interface including the second content, and a toolbar area having multiple drawing tool indications. In response to detecting user input at the location of the representation of the second content, and based on determining that the user input does not include stylus contact, the method includes relinquishing the display of the drawing user interface.

[0009] According to some embodiments, a method is performed in a device having one or more processors, non-transitory memory, and a touch-sensitive display. The method includes displaying a content creation area on the touch-sensitive display, the content creation area including typed text and one or more blank areas without content. The method includes detecting user input on the touch-sensitive display, the user input including movement of a touch within the content creation area. In response to detecting user input and determining that the user input includes movement of a stylus touch within a corresponding blank area, the method includes drawing a mark based on the movement of the stylus touch during user input. In response to detecting user input and determining that the user input includes movement of a finger touch within a corresponding blank area, the method includes scrolling the content creation area based on the movement of the finger touch during user input.

[0010] According to some embodiments, a method is performed in a device having one or more processors, non-transitory memory, and a touch-sensitive display. The method includes displaying a content creation area on the touch-sensitive display. While displaying the content creation area, the method includes detecting one or more keystroke inputs. In response to detecting one or more keystroke inputs, the method includes editing content in the content creation area to create a blank area that does not contain visible content and is a text insertion area. While displaying the content creation area with the text insertion area, the method includes detecting a touch on the touch-sensitive display in the text insertion area. In response to detecting a touch on the touch-sensitive display in the text insertion area, and based on determining that the touch is a stylus touch, the method includes converting at least a portion of the text insertion area into a drawing insertion area including a mark based on the stylus touch. In response to detecting a touch on the touch-sensitive display in the text insertion area, and based on determining that the touch is a finger touch, the method includes converting at least a portion of the text insertion area into a blank drawing insertion area.

[0011] According to some embodiments, an electronic device includes a display, one or more input devices, and one or more processors. The one or more processors are configured to display a drawing user interface comprising a plurality of drawing objects on the display. When a drawing tool associated with a predefined operation is selected, the one or more processors are configured to detect user input via the one or more input devices to define a path within the drawing user interface. In response to detecting user input to define a path within the drawing user interface, the one or more processors are configured to perform a predefined operation relative to two or more of the plurality of drawing objects intersecting the path.

[0012] According to some embodiments, an electronic device includes a display, one or more input devices, and one or more processors. The one or more processors are configured to display on the display first content and a representation of second content associated with the first content. The one or more processors are configured to detect user input at a location of the representation of the second content via the one or more input devices. In response to detecting user input at the location of the representation of the second content and determining that the user input includes stylus contact, the one or more processors are configured to display on the display a drawing user interface including the second content and a toolbar area having multiple drawing tool indications. In response to detecting user input at the location of the representation of the second content and determining that the user input does not include stylus contact, the one or more processors are configured to abandon the display of the drawing user interface.

[0013] According to some embodiments, an electronic device includes a touch-sensitive display and one or more processors. The one or more processors are configured to display on the touch-sensitive display a content creation area including typed text and one or more blank areas without content. The one or more processors are configured to detect user input on the touch-sensitive display, the user input including movement of a touch in the content creation area. In response to detecting user input and determining that the user input includes movement of a stylus touch in a corresponding blank area, the one or more processors are configured to draw markers based on the movement of the stylus touch during the user input. In response to detecting user input and determining that the user input includes movement of a finger touch in a corresponding blank area, the one or more processors are configured to scroll the content creation area based on the movement of a finger touch during the user input.

[0014] According to some embodiments, an electronic device includes a touch-sensitive display and one or more processors. The one or more processors are configured to display a content creation area on the touch-sensitive display. While displaying the content creation area, the one or more processors are configured to detect one or more keystrokes. In response to detecting one or more keystrokes, the one or more processors are configured to edit content in the content creation area to create a blank area that does not contain visible content and is a text insertion area. While displaying the content creation area as a text insertion area, the one or more processors are configured to detect touches on the touch-sensitive display in the text insertion area. In response to detecting a touch on the touch-sensitive display in the text insertion area, and based on determining that the touch is a stylus touch, the one or more processors are configured to convert at least a portion of the text insertion area into a drawing insertion area including marks based on the stylus touch. In response to detecting a touch on the touch-sensitive display in the text insertion area, and based on determining that the touch is a finger touch, the one or more processors are configured to convert at least a portion of the text insertion area into a blank drawing insertion area.

[0015] According to some embodiments, an electronic device includes a display, one or more input devices, one or more processors, non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing to perform any of the methods described herein. According to some embodiments, a non-transitory computer-readable storage medium stores instructions therein that, when executed by one or more processors of an electronic device having a display and input devices, cause the device to perform or cause to perform any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device includes one or more elements displayed in any of the methods described above, wherein the electronic device has a display, input devices, memory, and one or more processors for performing one or more programs stored in non-transitory memory, the one or more elements being updated in response to input, as described in any of the methods described herein. According to some embodiments, an electronic device includes: a display, an input device; and means for performing or causing to perform any of the methods described herein. According to some embodiments, an information processing device for an electronic device having a display and input devices includes means for performing or causing to perform any of the methods described herein.

[0016] Therefore, electronic devices with displays, touch-sensitive surfaces, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface offer faster and more efficient methods and interfaces for annotating content, thereby improving effectiveness, efficiency, and user satisfaction. Such methods and interfaces can complement or replace conventional methods for interpreting content. Attached Figure Description

[0017] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0018] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.

[0019] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0020] Figure 2 A portable multi-functional device with a touchscreen is shown according to some embodiments.

[0021] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0022] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some implementations is shown.

[0023] Figure 4B An example user interface for a multi-functional device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0024] Figure 5A-5DR An example user interface for annotating content is shown according to some implementation schemes.

[0025] Figures 6A-6C This is a flowchart illustrating a method for manipulating drawing objects according to some implementation schemes.

[0026] Figures 7A-7B This is a flowchart illustrating a method for marking content according to some implementation schemes.

[0027] Figures 8A-8C This is a flowchart illustrating a method for creating an area based on manipulated content according to some implementation schemes.

[0028] Figures 9A-9C This is a flowchart illustrating a method for inserting a drawing insertion area according to some implementation schemes. Detailed Implementation

[0029] Many electronic devices have graphical user interfaces (GUIs) that display content such as images or text. Some applications allow users to add or otherwise manipulate annotations associated with the content. In the embodiments described below, the application allows users to manipulate hand-drawn annotations associated with the content. For example, in some embodiments, when a tool is selected, a predefined operation associated with that tool is performed on the annotation that intersects the drawing path. As another example, a drawing user interface (as opposed to a viewing user interface) for annotating the content is provided when user input on the displayed content is provided with a stylus (as opposed to a finger). As another example, annotations are added to the content when user input is provided with a stylus (as opposed to a finger) within the blank space of the content (as opposed to scrolling the content). As another example, annotations are added to the content when user input is provided with a stylus (as opposed to a finger) within a blank text insertion area of ​​the content (as opposed to a graphical insertion area for adding annotations provided by subsequent user input).

[0030] The following provides Figure 1A-1B , Figure 2 and Figure 3 Description of the example device. Figures 4A-4B and Figure 5A-5DR A sample user interface for annotating content is shown. Figures 6A-6C A flowchart illustrating the method of manipulating drawing objects is shown. Figures 7A-7B A flowchart illustrating the method for tagging content is shown. Figures 8A-8C A flowchart illustrating the method for manipulating the content creation area is shown. Figures 9A-9C A flowchart illustrating the method for inserting a drawing insertion area is shown. Figure 5A-5DR The user interface in the document is used to explain Figures 6A-6C , Figures 7A-7B , Figures 8A-8C and Figures 9A-9C The process in.

[0031] Exemplary device

[0032] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to those skilled in the art that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the various aspects of the embodiments.

[0033] It will also be understood that, although in some cases the terms “first,” “second,” etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact, without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0034] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the items listed in connection with the description. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0035] As used herein, depending on the context, the term "if" is optionally interpreted as meaning "when" followed by "at" or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is optionally interpreted as meaning "when it is determined..." or "in response to determination..." or "when [the stated condition or event] is detected" or "in response to detection of [the stated condition or event]."

[0036] This document describes implementations of electronic devices, user interfaces for such devices, and related processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, those from Apple Inc. (Cupertino, California). iPod and Device. Optionally, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0037] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0038] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0039] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0040] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touchscreen" for convenience, and is sometimes simply referred to as a touchscreen display. Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 120, one or more processing units (CPUs) 122, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 163 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0041] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the tactile surface remains unchanged, movement of the tactile surface can optionally be interpreted or sensed by the user as “roughness” of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of the user (e.g., “press click,” “release click,” “roughness”), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user.

[0042] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, firmware, or any combination thereof (including one or more signal processing circuits and / or application-specific integrated circuits).

[0043] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100 (such as one or more CPUs 122 and peripheral interface 118) is optionally controlled by memory controller 120.

[0044] Peripheral interface 118 can be used to couple the device's input peripherals and output peripherals to memory 102 and one or more CPUs 122. One or more processors 122 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0045] In some implementations, the peripheral interface 118, one or more CPUs 122, and memory controller 120 are optionally implemented on a single chip, such as chip 104. In some other implementations, they are optionally implemented on separate chips.

[0046] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals and vice versa, and communicates with communication networks and other communication devices via these electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks and other devices, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). The wireless communication may optionally use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolved Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including those not yet developed as of the date of this document submission.

[0047] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset with both output (e.g., a mono or binaural headphone) and input (e.g., a microphone).

[0048] I / O subsystem 106 couples input / output peripherals on device 100, such as touch-sensitive display system 112 and other input or control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from or send electrical signals to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, stylus, and / or pointing device such as mouse. One or more buttons (e.g., Figure 2 Optionally, 208) includes an up / down button for volume control of speaker 111 and / or microphone 113. One or more buttons optionally include a push-down button (e.g., Figure 2 (206 in the middle).

[0049] The touch-sensitive display system 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

[0050] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile / haptic contact. The touch-sensitive display system 112 and the display controller 156 (along with any associated modules and / or instruction sets in memory 102) detect contact on the touch-sensitive display system 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In one exemplary embodiment, the contact point between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0051] The touch-sensitive display system 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed thereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In one exemplary embodiment, projected mutual capacitance sensing technology, such as that from Apple Inc. (Cupertino, California), is used. iPod and The technology discovered in [the text].

[0052] The touch-sensitive display system 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi, or greater). Users optionally use any suitable object or accessory such as a stylus, finger, etc., to interact with the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.

[0053] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0054] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0055] The device 100 may optionally also include one or more optical sensors 164. Figure 1A An optical sensor coupled to an optical sensor controller 158 in I / O subsystem 106 is shown. One or more optical sensors 164 optionally include charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) phototransistors. The one or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), the one or more optical sensors 164 optionally capture still images and / or video. In some embodiments, the optical sensor is located on the rear of device 100, opposite to the touch-sensitive display system 112 on the front of the device, enabling the touchscreen to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to acquire images of the user (e.g., for selfies, for video conferencing while the user views other video conference participants on the touchscreen, etc.).

[0056] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor coupled to a strength sensor controller 159 in I / O subsystem 106 is shown. One or more contact strength sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact strength sensors 165 receive contact strength information (e.g., pressure information or substitutes for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100 opposite to the touch-sensitive display system 112 located on the front of device 100.

[0057] The device 100 optionally also includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 is coupled to an input controller 160 in an I / O subsystem 106. In some embodiments, the proximity sensor turns off and disables the touch-sensitive display system 112 when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).

[0058] The device 100 may optionally also include one or more tactile output generators 163. Figure 1A A haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 163 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymerizers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device). The haptic output generator 163 receives haptic feedback generation instructions from haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the rear of the device 100 opposite to the touch-sensitive display system 112 located on the front of the device 100.

[0059] The device 100 may optionally also include one or more accelerometers 167, gyroscopes 168 and / or magnetometers 169 (e.g., as part of an inertial measurement unit (IMU)) for obtaining information about the device’s position (e.g., attitude). Figure 1A Sensors 167, 168, and 169, coupled to peripheral interface 118, are shown. Alternatively, sensors 167, 168, and 169 may be coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from the one or more accelerometers. Device 100 may optionally include a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the location of device 100.

[0060] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application program (or instruction set) 136. Furthermore, in some embodiments, memory 102 stores device / global internal state 157, as shown in Figures 1A and 1B. Figure 3 As shown in the diagram. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, which indicates what applications, views or other information occupy the various areas of the touch-sensitive display system 112; sensor state, including information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information regarding the device's position and / or orientation.

[0061] Operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0062] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external ports are associated with Apple Inc. (Cupertino, California). iPod A multi-pin (e.g., 30-pin) connector that is the same as, similar to, and / or compatible with the 30-pin connector used in iPod devices. In some implementations, the external port is compatible with some Apple Inc. (Cupertino, California) connectors. iPod A Lightning connector that is the same as or similar to and / or compatible with the Lightning connector used in iPod devices.

[0063] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes software components for performing various operations related to contact detection (e.g., a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the intensity of the contact (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations may optionally be applied to single-point contact (e.g., single-finger contact or stylus contact) or simultaneous multi-point contact (e.g., "multi-touch" / multi-finger contact and / or stylus contact). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0064] The contact / motion module 130 optionally detects user gesture input. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the icon location). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event. Similarly, stylus taps, swipes, drags, and other gestures are optionally detected by detecting specific contact patterns of the stylus.

[0065] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, and includes, without limitation, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

[0066] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like to specify the graphic to be displayed, and, if necessary, also receives coordinate data and other graphic attribute data, and then generates screen image data for output to the display controller 156.

[0067] The haptic feedback module 133 includes various software components for generating instructions that are used by the haptic output generator 163 to produce haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0068] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email client 140, IM 141, browser 147, and any other application that requires text input).

[0069] GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to a phone 138 for location-based dialing; to a camera 143 as image / video metadata; and to applications that provide location-based services such as weather desktop apps, local yellow pages desktop apps, and map / navigation desktop apps).

[0070] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0071] • Contacts module 137 (sometimes called address book or contact list);

[0072] • Telephone module 138;

[0073] • Video conferencing module 139;

[0074] • Email client module 140;

[0075] • Instant Messaging (IM) module 141;

[0076] Fitness support module 142;

[0077] • Camera module 143 for still images and / or video images;

[0078] • Image management module 144;

[0079] • Browser module 147;

[0080] • Calendar module 148;

[0081] • Desktop mini-program module 149, which optionally includes one or more of the following: weather desktop mini-program 149-1, stock market desktop mini-program 149-2, calculator desktop mini-program 149-3, alarm clock desktop mini-program 149-4, dictionary desktop mini-program 149-5 and other desktop mini-programs obtained by the user, and desktop mini-programs created by the user 149-6;

[0082] • Desktop app creator module 150 for creating user-created desktop apps 149-6;

[0083] • Search module 151;

[0084] • Video and music player module 152, optionally composed of a video player module and a music player module;

[0085] • Memo module 153;

[0086] • Map module 154; and / or

[0087] • Online video module 155.

[0088] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0089] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 includes executable instructions for managing an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers and / or email addresses to initiate and / or facilitate communication via telephone 138, video conferencing 139, email client 140, or instant messaging 141; and so on.

[0090] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, telephone module 138 includes executable instructions for performing the following operations: inputting a character sequence corresponding to a telephone number, accessing one or more telephone numbers in the address book 137, modifying an input telephone number, dialing a corresponding telephone number, initiating a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communication optionally employs any of a variety of communication standards, protocols, and technologies.

[0091] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 12, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0092] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0093] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for performing the following operations: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Services (APNs), or IMPS for internet-based instant messaging), receiving an instant message, and viewing a received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" means both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0094] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, fitness support module 142 includes executable instructions for creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (in sports equipment and smartwatches); receiving fitness sensor data; calibrating sensors used for monitoring fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0095] In conjunction with the touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0096] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, displaying (e.g., in a digital slideshow or photo album), and storing still images and / or video images.

[0097] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) according to user instructions.

[0098] Incorporating RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0099] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet module 149 is optionally a micro-application downloaded and used by a user (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) or a user-created micro-application (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop applet includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop applet).

[0100] Combining RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet creator module 150 includes executable instructions for creating desktop applets (e.g., transferring user-specified portions of web pages to desktop applets).

[0101] In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0102] Incorporating the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, the video and music player module 152 includes executable instructions allowing users to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on the touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0103] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0104] Combining RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and map-related data (e.g., driving routes; data on shops and other points of interest at or near specific locations; and other location-based data) according to user instructions.

[0105] In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes executable instructions that allow users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen 112 or on an external display connected wirelessly or via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos.

[0106] Each module and application identified above corresponds to a set of executable instructions for performing one or more of the functions described above and the methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures described above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0107] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on device 100 can be optionally reduced.

[0108] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, home menu, or root menu. In such embodiments, a touchpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0109] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A (middle) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0110] Event classifier 170 receives event information and determines the application 136-1 to which the event information should be delivered and the application view 191 of application 136-1. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display system 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0111] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0112] Event monitor 171 receives event information from peripheral device interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display system 112 as part of a multi-touch gesture). Peripheral device interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 167, one or more gyroscopes 168, one or more magnetometers 169, and / or microphone 113 (via audio circuitry 110). The information received by peripheral device interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or touch-sensitive surfaces.

[0113] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other implementations, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0114] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0115] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0116] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0117] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that forms an event or potential event) occurs. Once the hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0118] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0119] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver module 182.

[0120] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0121] In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit (not shown) or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0122] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies events from the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0123] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device pose).

[0124] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on a displayed object for a predetermined duration, movement of the touch on the touch-sensitive display system 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0125] In some implementations, event definition 187 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on a touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0126] In some implementations, the definition of the corresponding event 187 also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0127] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0128] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0129] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some implementations, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with that tag retrieves the tag and executes a predefined procedure.

[0130] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined procedure.

[0131] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module 145. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 176 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0132] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0133] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0134] Figure 2 This illustrates a device with a touchscreen (e.g., according to some embodiments). Figure 1AA portable multi-functional device 100 (a touch-sensitive display system 112). The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the diagram) or one or more styluses 203 (not drawn to scale in the diagram). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0135] Device 100 optionally also includes one or more physical buttons, such as a "home" button or a menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display.

[0136] In some embodiments, device 100 includes a touchscreen display, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more haptic output generators 163 for generating haptic outputs for a user of device 100.

[0137] Figure 3This is a block diagram of an exemplary multi-functional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 163 and sensors 359 (e.g., similar to those mentioned above) Figure 1A The sensors 112, 164, 165, 166, 167, 168, and 169 are touch sensors, optical sensors, contact strength sensors, proximity sensors, acceleration sensors, attitude and / or magnetic sensors. Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores data with portable multifunction device 100 (…). Figure 1A The memory 370 stores programs, modules, and data structures similar to those stored in the memory 102 of the portable multifunction device 100, or subgroups thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0138] Figure 3Each of the elements identified above is optionally stored in one or more of the previously mentioned memory devices. Each of the modules identified above corresponds to a set of instructions for performing the functions described above. The modules or programs identified above (i.e., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subgroup of the modules and data structures described above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0139] Now turn our attention to the implementation of the user interface (“UI”) optionally implemented on the portable multifunction device 100.

[0140] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is shown. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes elements or a subset or superset thereof:

[0141] • Signal strength indicator 402 for wireless communications (such as cellular signals and Wi-Fi signals);

[0142] • Time 404;

[0143] Bluetooth indicator 405;

[0144] • Battery status indicator 406;

[0145] • Tray icon 408 with commonly used application icons, such as:

[0146] The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemails.

[0147] The email client module 140 has an icon 418 labeled "Mail" which optionally includes an indicator 410 for the number of unread emails.

[0148] The browser module 147 has an icon 420 labeled "Browser"; and

[0149] o Video and music player module 152 (also known as iPod (a trademark of Apple Inc.))

[0150] Module 152) has an icon 422 labeled "iPod"; and

[0151] Icons of other applications, such as:

[0152] o IM module 141's icon 424 marked as "text";

[0153] The icon 426 labeled "Calendar" in the calendar module 148.

[0154] The image management module 144 has an icon 428 labeled "Photo".

[0155] The icon 430 of camera module 143 labeled "camera"

[0156] The icon 432 in module 155 of the online video module is labeled "Online Video".

[0157] The icon labeled "Stock Market" in the oStock Market Desktop Mini Program 149-2 is 434.

[0158] The icon 436 labeled "Map" in map module 154.

[0159] The icon labeled "Weather" in the oWeather desktop mini-program 149-1 is 438.

[0160] The icon labeled "Clock" in the o alarm clock desktop mini-program 169-6 is 440;

[0161] The icon 442 labeled "Fitness Support" in the fitness support module 142.

[0162] The notepad module 153 has an icon 444 labeled "Notepad"; and

[0163] o An icon 446 is used to set up applications or modules, which provides access to settings for device 100 and its various applications 136.

[0164] It should be noted that Figure 4A The icon labels shown are merely exemplary. For example, in some embodiments, the icon 422 of the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0165] Figure 4B A touch-sensitive surface 451 separate from the display 450 is shown (e.g., Figure 3 Devices (e.g., tablets or touchpads 355) Figure 3An exemplary user interface on device 300. Device 300 may also optionally include one or more contact intensity sensors (e.g., one or more sensors in sensor 359) for detecting the intensity of contact on tactile surface 451 and / or one or more tactile output generators 359 for generating tactile output for the user of device 300.

[0166] Figure 4B A touch-sensitive surface 451 separate from the display 450 is shown (e.g., Figure 3 Devices (e.g., tablets or touchpads 355) Figure 3 An exemplary user interface on device 300). Although numerous subsequent examples are given with reference to input on touchscreen display 112 (which combines a touch-sensitive surface and a display), in some embodiments the device detects input on a touch-sensitive surface separate from the display, such as Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., on the display (e.g., 450) that is aligned with the main axis on the display (e.g., 451). Figure 4B The principal axis corresponding to 453 in the middle (e.g., Figure 4B (452 in the middle). According to these embodiments, the device detects contact with the touch-sensitive surface 451 at a position corresponding to the corresponding position on the display (e.g., Figure 4B (460 and 462 in the example) Figure 4B In the diagram, 460 corresponds to 468 and 462 corresponds to 470. Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated, user input detected by the device on the touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0167] User interface and related processes

[0168] Now attention is turned to implementations of user interfaces (“UIs”) and associated processes that can be implemented on electronic devices such as portable multifunction devices (PMDs) 100 or 300, which have displays, touch-sensitive surfaces, and one or more sensors for detecting the intensity of contact with the touch-sensitive surfaces.

[0169] Figure 5A-5DR Example user interfaces for annotation content are shown according to some implementation schemes. The user interfaces in these figures are used to illustrate the processes described below, including… Figures 6A-6C , Figures 7A-7B , Figures 8A-8C and Figures 9A-9C The process in question. Although some examples in the following examples will refer to input on a touchscreen display (which combines a touch-sensitive surface and a display), in some implementations, the device detects input on a touch-sensitive surface 451 separate from the display 450, such as... Figure 4B As shown in the image.

[0170] Figure 5A A drawing user interface 501 is shown as part of a user interface 500 displayed as part of a portable multi-functional device 100 (hereinafter referred to as "device 100"). Figure 5A In this drawing user interface 501, the drawing user interface 501 spans the entire user interface 500. The drawing user interface 501 includes an options bar 511, a content area 512, and a toolbar area 513. The options bar 511 includes an exit enable display 515 for replacing the drawing user interface 501 with, for example, the user interface of another application within the user interface 500. The options bar 511 includes an edit mode for making the content in the content area 512 editable (e.g., marked). Figure 5A The options bar 511 includes a switching indicator 516 that toggles between a viewing mode (shown in the image) and a viewing mode in which the content in the content area 512 can be viewed but not edited. The options bar 511 includes a sharing indicator 517 for sharing the content in the content area 512 with other users, for example, by sending a copy of the content via email or by sending a text message containing a copy of the content.

[0171] Content area 512 contains content that can be edited via drawing user interface 501. For example, in Figure 5A In this context, the content area includes a photograph of the fruit platter. In various specific implementations, the content area is blank, includes drawing objects (as further described below), includes images, or includes PDF (Portable Document Format) files.

[0172] Toolbar area 513 includes multiple drawing tool displays associated with various drawing tools. Figure 5A In the toolbar area 513, there are pen tool indications 532A associated with a pen tool that can be used to create opaque (or substantially opaque) strokes in the content area 512, highlighter tool selection indications 532B associated with a highlighter tool that can be used to create semi-transparent strokes in the content area 512, deletion tool selection indications 532C associated with a deletion tool that can be used to delete drawing objects in the content area 512, and selection tool selection indications 532D associated with a selection tool that can be used to select drawing objects in the content area 512.

[0173] Toolbar area 513 includes an undo indicator 531A for reversing the last action taken by the user (e.g., undoing the insertion of a drawing object) and a redo indicator 531B for resuming an undo action (e.g., redoing the insertion of a drawing object). In various cases, if no undo or redo action exists, the undo indicator 531a and / or the redo indicator 531B are not displayed or are grayed out.

[0174] The toolbar area 513 includes a color selection indicator 533 for selecting the color of a drawing object inserted via the drawing user interface 501. The toolbar area 513 also includes a toolbar menu indicator 534 for displaying the toolbar menu.

[0175] Figure 5A The finger contact 550A detected at the location indicated by the toolbar menu power indicator 534 is shown. Figure 5B This demonstrates a response to detecting finger contact with 550A at the location of the toolbar menu indicator 534. Figure 5A The user interface 500. In response to detecting finger contact 550A, the drawing user interface 501 includes a toolbar menu 535Z with multiple drawing object insertion indications 535A-535D for inserting drawing objects into the content area 512. In various specific embodiments, after selecting one of the drawing object insertion indications 535A-535D, the drawing object is inserted into the content area 512 with default characteristics. Through further interaction, the user can modify the inserted drawing object to have a desired size, position, text characters, or other characteristics. Figure 5B In the drawing object representations 535A-535D, there are text insertion representations 535A for inserting text drawing objects into the content area 512, signature insertion representations 535B for inserting user-defined shape drawing objects (e.g., hand-drawn signatures), magnifying glass insertion representations 535C for inserting magnified versions of magnifying glass drawing objects that display content behind magnifying glass drawing objects, and one or more shape insertion representations 535D for inserting application-defined shape drawing objects, such as squares, circles, lines, or bubbles.

[0176] Figure 5C It shows Figure 5BThe user interface 500 has multiple drawing objects in the content area 512 of the drawing user interface 501. Besides the photo below the fruit platter, the content area 512 also includes a square drawing object 521A (approximately surrounding the grapes in the fruit platter), a magnifying glass drawing object 521B (magnifying a portion of the leaves of an apple in the fruit platter), a text drawing object 521C (labeling "LEMON" on a lemon in the fruit platter), a set of three-stroke drawing objects 521D (which together form a smiley face on the apple), and a set of six-stroke drawing objects 521E (which together form a flower on the platter). In various specific implementations, the drawing tool associated with the stroke operation is selected (such as a pen tool associated with a pen tool selection indication 532A, a highlighter tool associated with a highlighter tool indication 532B, or a pencil tool, for example, with...). Figure 5BG When the pencil tool (which can represent the pencil tool associated with 532H) or other tools of the brush tool, the stroke drawing object is defined by continuous user input within the drawing user interface 501.

[0177] Figure 5C The image shows finger contact 550B detected at the location indicated by the deletion tool selection indicator 532C. Figure 5D This demonstrates a response to the detection of finger contact 550B at the location of the deletion tool indicator 532C. Figure 5C User interface 500. Figure 5D In the deletion tool section, the selection indicator 532C is changed (e.g., raised) to indicate that a deletion tool has been selected. Figure 5D The movement of a finger contact 550C is shown, which draws a path within the content area 512 of the drawing user interface 501. Although in Figure 5D The path shown is a straight line, but in various specific implementations, the path is not straight. The path extends from the first position to the second position and intersects with the top three stroke drawing objects in a set of six-stroke drawing objects 521E.

[0178] According to some specific implementations Figure 5E This illustrates the path drawn in response to detected movement of a finger contact 550C, intersecting with the top three stroke drawing objects in a set of six-stroke drawing objects 521E. Figure 5D User interface 500. Figure 5E In the middle, the three-stroke drawing object at the top stops displaying. In various specific implementations, when the delete tool is selected, the user input that drew the path deletes the drawing object that intersects the path.

[0179] According to some specific implementations Figure 5FThis illustrates the path drawn in response to detected movement of a finger contact 550C, intersecting with the top three stroke drawing objects in a set of six-stroke drawing objects 521E. Figure 5D User interface 500. Figure 5F In response to the movement of a finger touching 550C while drawing a path, all six of the six strokes in a group of six drawing objects 521E are stopped from being displayed. In various specific implementations, when the delete tool is selected, the user inputting the path deletes the drawing objects that intersect with the path, and also deletes the drawing objects that intersect with the deleted objects.

[0180] Figure 5G The user interface 500 showing the drawing user interface 501B is illustrated. The drawing user interface 501B includes a toolbar area 513B with an eraser tool selection indicator 532E, which is connected to... Figure 5A The deletion tool selection display of the drawing user interface 501 is shown in the opposite way to 532C. Figure 5G The movement of a finger contact 550D is shown, which draws a path within the content area 512 of the drawing user interface 501. Although in Figure 5D The path shown is a straight line, but in various specific implementations, the path is not straight. The path extends from the first position to the second position and intersects with the top three stroke drawing objects in a set of six-stroke drawing objects 521E.

[0181] According to some specific implementations Figure 5H This illustrates the path drawn in response to detected movement of a finger contact 550D, intersecting with the top three stroke drawing objects in a set of six-stroke drawing objects 521E. Figure 5G User interface 500. Figure 5H In the eraser tool, the portion of a drawing object that intersects a path (the top three strokes) stops displaying, while the portion that does not intersect the path continues to be displayed. Unlike the delete tool, which deletes drawing objects that intersect a path, the eraser tool deletes the portion of a drawing object that intersects a path, but not the portion that does not intersect the path.

[0182] Figure 5I It shows Figure 5F The user interface 500 has detected movement of a finger touching 550E, drawing a path within the content area 512 of the drawing user interface 501. Although in Figure 5I The path shown is a straight line, but in various specific implementations, the path is not straight. The path extends from the first position to the second position and intersects the square drawing object 521A. However, the square drawing object 521A continues to be displayed (e.g., because the path does not meet the object deletion criteria).

[0183] In various specific implementations, if the contact speed of a path drawn within the drawing user interface 501 exceeds a speed threshold, the device 100 stops displaying drawing objects intersecting the path; and if the contact speed does not exceed the speed threshold, the device 100 continues displaying drawing objects intersecting the path. Therefore, quickly swiping over a drawing object (even if the delete tool is selected) will not delete the object. In some implementations, the contact speed is determined separately for each drawing object intersecting the path. Therefore, if the contact speed while moving over a first object is higher than the speed threshold (e.g., the path meets the object deletion criteria), the first object is deleted; and if the contact speed while moving over a second object is lower than the contact speed threshold (e.g., the path does not meet the object deletion criteria), the second object is not deleted.

[0184] Figure 5J It shows Figure 5I The user interface 500 detects finger contact 550F at the location of the selection tool indicator 532D. Figure 5K This demonstrates a response to detecting finger contact with 550F at the location of the selection tool indicator 532D. Figure 5J User interface 500. Figure 5K In the middle, change (e.g., raise) the selection tool indicator 523D to indicate that the selection tool has been selected. Figure 5K The movement of a finger contact 550G is shown, which draws a path within the content area 512 of the drawing user interface 501. The path extends from a first position to a second position and intersects with two of the stroke drawing objects in a set of three-stroke drawing objects 521D.

[0185] Figure 5L This demonstrates a movement of 550G in response to the detection of finger contact as part of a drawn path. Figure 5K User interface 500. When a finger moves along a path by touching 550G, a path selection preview graphic 522A is displayed at the location where the finger touches 550G. In various specific embodiments, the path selection preview graphic has similar visual characteristics to the path selection graphic 522B described below.

[0186] Figure 5M This demonstrates how, in response to the detection of finger contact 550G, a path was completed intersecting two of the stroke drawing objects in a set of three-stroke drawing objects 521D. Figure 5LThe user interface 500. In response to detecting a finger contact 550G movement, a selection path is defined based on that path. In various embodiments, the selection path is the same as the drawing path. In various embodiments, the selection path is an extension of the drawing path, for example, as a closed loop. Therefore, in various embodiments, the selection path is a closed shape based on the shape of the drawing path. Figure 5K In the drawing user interface 501, the content area 512 includes a selection path graphic 522B corresponding to the selected path. In various embodiments, the selection path graphic 522B includes alternating shadows (e.g., black) and white dashed lines that are partially transparent (e.g., 50% opacity). In this way, the selection path graphic 522B can be seen on any color of the underlying content. In various embodiments, the selection path graphic is animated, for example, making the alternating shadows and white dashed lines appear to rotate around the selection path.

[0187] The selection path graphic 522B provides the user with visual indications of which drawing objects the selection tool has selected. In various embodiments, it selects drawing objects that intersect with the selection path. In various embodiments, it also selects drawing objects that intersect with objects that intersect with the selection path. In various embodiments, it also selects drawing objects enclosed by the selection path. Therefore, in Figure 5M In the middle, all three-stroke drawing objects 521D are selected, two are selected by intersecting with the selection path, and the third is selected by being surrounded by the selection path.

[0188] Figure 5N It shows Figure 5M The user interface, wherein finger contact 550H is detected at the location of the selection path graphic 522B and moves to another location. Figure 50 illustrates the response to the detected movement of finger contact 550H. Figure 5N The user interface 500. In Figure 50, the selection path graphic 522B stops displaying and moves the selected drawing object (e.g., stroke drawing object 521D) according to the movement of the contact 550H. Although the selection path graphic 522B stops displaying, in various embodiments, the selected drawing object is displayed with a pull-down shadow or other visual indicators not displayed before the selection of the selected drawing object to provide a visual indication of which drawing object was selected. In various embodiments, the selection path graphic 522B is replaced by an alternative visual indicator of selection.

[0189] Figure 5P A user interface 500 is shown in response to the detection of a finger touch 550H being lifted. Figure 5P In the middle, the selected path graphic 522B is displayed again (and the drop-down shadow or other visual indicator stops displaying to indicate that the graphic object is no longer selected).

[0190] Figure 5Q A user interface 500 displaying a drawing user interface 501C is shown. The drawing user interface 501C includes a toolbar area 513C with a color tool selection indicator 532F, which is connected to... Figure 5A The drawing user interface 501 selects the tool to display the relative display 532D. In Figure 5Q In the middle, the color selection indicator shows that the 533 indicator has selected gray (as in...). Figure 5A (Selected black relative).

[0191] Figure 5Q The movement of a finger touch 5501 is shown, with the finger touch 5501 drawing a path within the content area 512 of the drawing user interface 501. Although in Figure 5Q The path shown is a straight line, but in various specific implementations, the path is not straight. The path extends from the first position to the second position and intersects with the top two stroke drawing objects and the text drawing object 521C of a set of three-stroke drawing objects 521D.

[0192] Figure 5R A user interface 500 is shown in response to the detection of finger touch 5501 movement, whereby finger touch 5501 draws a path intersecting the top two stroke drawing objects and the text drawing object 521C in a set of three-stroke drawing objects 521D. Figure 5R In the middle, the top two stroke drawing objects and the text drawing object 521C in a set of three-stroke drawing objects 521D have changed color (e.g., from black to gray).

[0193] Figure 5S The user interface 500 showing the drawing user interface 501D is illustrated. The drawing user interface 501D includes a toolbar area 513D with a thinning tool selection indicator 532G, which is connected to... Figure 5A The drawing user interface 501 selects the selection tool to display the relative 532D. Figure 5S This illustrates the movement of a finger touch 550J drawing a path within the content area 512 of the drawing user interface 501. Although in Figure 5S The path shown is a straight line, but in various specific implementations, the path is not straight. The path extends from the first position to the second position and intersects with the left two stroke drawing objects of the three-stroke drawing object set 521D and the square drawing object 521A.

[0194] Figure 5T A user interface 500 is shown in response to the detection of movement of a finger touch 550J, whereby the finger touch 550J draws a path intersecting the left two stroke drawing objects and the square drawing object 521A in a set of three-stroke drawing objects 521D. Figure 5TIn the example, the left two stroke drawing objects and the square drawing object 521A in a set of three-stroke drawing objects 521D have had their line widths changed (e.g., the line widths have been reduced). In various embodiments, a drawing tool is provided that changes the size of drawing objects intersecting with a path. In various embodiments, a sizing drawing tool changes one or more dimensions of a drawing object (e.g., shrinks or enlarges the drawing object), changes the font size of the drawing object, or changes the line width of the drawing object.

[0195] Figure 5U It shows Figure 5P The user interface 500, in which the movement of a finger touches 550K draws a path that intersects with the text drawing object 521C. Figure 5V This illustrates a response to the detection of movement of a finger contact 550K drawing a path intersecting the text drawing object 521C. Figure 5U The user interface is 500. Figure 5V In the center, select (e.g., via the selection tool) the text drawing object 521C as indicated by the surrounding selection 522C. Since the text drawing object is selected, the toolbar area 513 includes a text option display 536A. The text option display 536A appears between the multiple drawing tool selection displays 521A-531D and the toolbar menu display 534. The text option display 536A appears next to the toolbar menu display 534. Figure 5V The image shows finger contact 550L detected at the location of the text option display 536A.

[0196] Figure 5W This demonstrates a response to detecting finger contact with 550L at the location of the text option display 536A. Figure 5V The user interface is 500. Figure 5V The image shows a text options menu 537A, which includes indications for changing the properties of the text drawing object 521C. In various specific embodiments, the text options menu 537A includes indications for changing the font, font size, and alignment of the text drawing object 521C.

[0197] Figure 5X It shows Figure 5W The user interface 500, in which the movement of a finger touches 550M draws a path that intersects with the square drawing object 521A. Figure 5Y This illustrates a response to the detection of movement of a finger contact 550M in drawing a path intersecting the square drawing object 521A. Figure 5X The user interface is 500. Figure 5YIn the selection, the square drawing object 521C is selected (e.g., via the selection tool), as indicated by selection marker 522D. Because the shape drawing object is selected, the toolbar area 513 includes a shape options display 536B. The shape options display 536B appears between the multiple drawing tool selection displays 521A-531D and the toolbar menu display 534. The shape options display 536B appears next to the toolbar menu display 534. Figure 5Y The image shows finger contact 550N detected at the location of the shape text option display 536A.

[0198] Figure 5Z This demonstrates a response to detecting finger contact 550N at the location of the shape option indicator 536B. Figure 5Y User interface 500. Figure 5Z The image shows a shape options menu 537B, which includes an indication of how to change the characteristics of a square drawing object 521A. In various specific embodiments, the shape options menu 537B includes an indication of how to change the fill characteristics (e.g., fill to no fill) of the square drawing object 521A, the line continuity characteristics (e.g., from dashed to solid lines) of the shape drawing object 521A, and the line width of the shape drawing object 521A.

[0199] Figure 5Z The movement of finger contact 550O is shown to draw a path that intersects with the magnifying glass drawing object 521B. Figure 5V This illustrates a movement of finger contact 550O in response to the detection of a path drawn intersecting with the magnifying glass drawing object. Figure 5Z User interface 500. Figure 5AA In the selection tool, the magnifying glass drawing object 521B is chosen. Because the magnifying glass drawing object is selected, the toolbar area 513 does not include the text option indicator 536A or the shape option indicator 536B. Instead, blank space 536C is provided in their place. Blank space 536C is displayed between the multiple drawing tool indicators 521A-531D and the toolbar menu indicator 534. Blank space 536C is displayed next to the toolbar menu indicator 534.

[0200] Figure 5ABUser interface 500 is shown, which includes an email user interface 502 below a device bar 503. The device bar 503 at the top of the display includes an identifier for device 100 (e.g., "iPad"), a wireless connectivity indicator, the current time, and a battery indicator indicating the charging level of device 100. User interface 500 includes an email user interface 502 below the device bar 501, which spans the rest of the display. Email user interface 502 includes message display indicators 541A-541D, each associated with a plurality of email messages. Figure 5AB A finger contact 551A is shown detected at the location of a first message enablement 541A associated with a first email message.

[0201] Figure 5AC This demonstrates a response to detecting finger contact 551A at the location of the first message indicator 541A. Figure 5AB User interface 500. In response to detecting finger contact 551A at the location of the first message display 541A, email user interface 502 displays the first email message. Figure 5AC In this email user interface 502, an information area 542A and a content area 542B are included. The information area 542A includes information about a first email message (e.g., sender, recipient, subject, and sending time), and the content area 542B includes the content of the first email message. The content area 542B includes a representation of the first content 543A in text form and a representation of the second content 543B in image form. In various embodiments, the second content is an attached image, and the representation of the second content is a scaled-down (or thumbnail) image of the attached image. In various embodiments, the second content is an attached PDF file, and the representation of the second content is an icon (as further described below).

[0202] Figure 5AD It shows Figure 5AC The user interface 500 detects finger contact 551B at the location indicated by the second content 543B.

[0203] Figure 5AE This illustrates a response to detecting finger contact 551B at the location indicated by the second content 543B. Figure 5ADUser interface 500. In response to detecting finger contact 551B at the indicated location of the second contact, user interface 500 includes photo viewer user interface 504A, which includes second content 544. The photo viewer user interface 504A displayed below device bar 503 includes an options bar 545A and a content area 545B. Options bar 545A includes photo processing capabilities 518 for manipulating photos (e.g., rotating the photo, resizing the photo, cropping the photo, or adjusting brightness or contrast). Options bar 545A includes a marking capability 516 for switching between an editing mode in which the content in content area 545B can be edited (e.g., marked) and a viewing mode in which the content in content area 545B can be viewed but not edited. Figure 5AE In the middle, a marker indication 516 is displayed in a certain way (e.g., mostly black) to indicate that viewing mode (not editing mode) is active. The options bar 545A includes a sharing indication 517 for sharing content in the content area 545B with other users, for example, by sending a copy of the content via email or sending a text message including a copy of the content.

[0204] Figure 5AE A finger contact 551C detected at the location indicated by the marker 516 is shown. Figure 5AF This demonstrates a response to detecting finger contact 551C at the location of the indicator 516. Figure 5AE The user interface 500 is displayed in a certain way (e.g., mostly white) to indicate that edit mode (not view mode) is active. Figure 5AF In this process, the photo viewer user interface 504A is replaced by a drawing user interface 501. The drawing user interface 501 includes second content 544 in the content area 512 and includes a toolbar area 513 with multiple drawing tool display indicators 532A-532D.

[0205] Figure 5AG It shows Figure 5AD The user interface 500 detects stylus contact 561A instead of... Figure 5AD Finger contact 551C is detected. Therefore, user interface 500 includes photo viewer user interface 504A, which includes second content 544. Figure 5AG The stylus contact 561A detected at the location of the second content 544 is shown. Figure 5AH This illustrates the response to detecting stylus contact 561A at the location of the second content 544. Figure 5AG User interface 500. Figure 5AHIn this process, the photo viewer user interface 504A is replaced by a drawing user interface 501. The drawing user interface 501 includes second content 544 in the content area 512 and includes a toolbar area 513 with multiple drawing tool display indicators 532A-532D.

[0206] Figure 5AI It shows Figure 5AD The user interface 500 detects movement of the stylus contact 561B, rather than... Figure 5AD The finger touches 551C. Therefore, the user interface 500 includes a photo viewer user interface 504A, which includes second content 544. Figure 5AI The movement of stylus contact 561B detected at the location of the second content 544 is shown. Figure 5AJ This illustrates the response to the detection of movement of stylus contact 561B at the position of the second content 544. Figure 5AI User interface 500. Figure 5AJ In this drawing interface, the photo viewer user interface 504A is replaced by a drawing user interface 501. The drawing user interface 501 includes second content 544 in a content area 512 and a toolbar area 513 with multiple drawing tool indications 532A-532D. Although the movement of the stylus contact 561B defines a path from a first position within the second content 544 to a second position within the second content 544, the drawing user interface 501 also includes a stroke drawing object 521F corresponding to that path.

[0207] Figure 5AJ The contact 551Z detected at the location of the shared power indicator 517 is shown. Figure 5 AK The following is shown in response to the detection of contact 551Z at the location of shared power indicator 517. Figure 5AJ User interface 500. User interface 500 includes a sharing menu 519A. Sharing menu 519A includes a sharing capability 519B via text message for sharing content in content area 512 with another user via text message, which includes second content 544 marked by a stroke drawing object 521F. Sharing menu 519A includes a sharing capability 519C via reply for replying to a first email message (sent to the sender of the first email message) with content attached to the reply content area 512. Sharing menu 519A includes a sharing capability 519D via new email for attaching content in content area 512 to a new email message to be sent to the user's selected receipt. Sharing menu 519A includes a new photo save capability 519E for saving content in content area 512 as a new photo on device 100.

[0208] Figure 5AL It shows Figure 5AE The user interface 500 detects finger contact 551D at the location of the second content 544, rather than at... Figure 5AE The marker in the image indicates that finger contact 551C was detected at the location 516. Therefore, the user interface 500 includes a photo viewer user interface 504A, which includes second content 544. Figure 5AL A finger contact 551D detected at the location of the second content 544 is shown.

[0209] Figure 5AM This illustrates a response to detecting finger contact 551D at position 544 of the second content. Figure 5AL User interface 500. Figure 5AM A photo viewer user interface 504A in full-screen mode is shown, in which second content 544 is displayed and the device bar 503 and options bar 545A are not displayed. In various specific embodiments, in full-screen mode, the user interface 500 includes only the second content 544.

[0210] Figure 5AN It shows Figure 5AE The user interface 500 detects movement of a finger touch 551E at the location of the second content 544, rather than at... Figure 5AE The marker in the image indicates that finger contact 551C was detected at the location 516. Therefore, the user interface 500 includes a photo viewer user interface 504A, which includes second content 544. Figure 5AN The movement of finger contact 551E detected at the location of the second content 544 is shown.

[0211] Figure 5AO This illustrates a response to the detection of movement of a finger contact 551E at the location of the second content 544. Figure 5AN User interface 500. User interface 500 includes photo viewer user interface 504A, wherein second content 544 moves in response to movement of finger contact 551E. Therefore, in response to detecting movement of finger contact within the second content 544, content area 545B is scrolled, thereby moving the second content 544.

[0212] Figure 5AP It shows Figure 5AC The user interface 500, wherein stylus contact 561C is detected at the location indicated by the second content 543B. Figure 5AQ This illustrates the response to detecting stylus contact 561 at the location indicated by the second content 543B. Figure 5AP User interface 500. Figure 5AQIn this embodiment, the email user interface 502 is replaced by a drawing user interface 501. The drawing user interface 501 includes second content 544 in the content area 512 and includes a toolbar area 513 with multiple drawing tool display indicators 532A-532D.

[0213] Figure 5AR It shows Figure 5AC The user interface 500 detects movement of stylus contact 561D at the location represented by the second content 543B. According to one embodiment, Figure 5AS This illustrates the response to detecting movement of stylus contact 561D at the location indicated by the second content 543B. Figure 5AR User interface 500. Figure 5AS In this embodiment, the email user interface 502 is replaced by a drawing user interface 501. The drawing user interface 501 includes second content 544 in a content area 512 and a toolbar area 513 with multiple drawing tool indications 532A-532D. Although the movement of the stylus contact 561D defines a path from a first position within the representation of the second content 543B to a second position within the representation of the second content 543B, the drawing user interface 501 also includes a stroke drawing object 521G corresponding to that path. In various specific embodiments, when the representation of the second content 543B is an image and the second content 544 is a magnified version of the image of the second content, the stroke drawing object 521G corresponding to the path is a magnified version of the path.

[0214] According to another implementation scheme, Figure 5AT This illustrates the response to detecting movement of stylus contact 561D at the location indicated by the second content 543B. Figure 5AR User interface 500. Figure 5AT In this embodiment, the email user interface 502 is replaced by a drawing user interface 501. The drawing user interface 501 includes second content 544 in a content area 512 and a toolbar area 513 with multiple drawing tool indications 532A-532D. Although the movement of the stylus contact 561D defines a path from a first position within the representation of the second content 543B to a second position within the representation of the second content 543B, the drawing user interface 501 also includes a stroke drawing object 521H corresponding to that path. In various specific embodiments, when the representation of the second content 543B is an image and the second content 544 is a magnified version of the image of the second content, the stroke drawing object 521H corresponding to that path is not a magnified version of that path, but rather a version of the path with the same size as the path defined by the movement of the stylus contact 561D.

[0215] Figure 5AU It shows Figure 5ACThe user interface 500 detects movement of finger contact 551F at the location indicated by the second content 543B. Figure 5AV This illustrates a response to the detection of finger contact 551F at the location indicated by the second content 543B. Figure 5AU The user interface 500 includes an email user interface 502, wherein the representation of second content 543B (and first content 543B) moves according to the movement of a finger touch 551F. Therefore, in response to detecting movement of a finger touch at the location of the representation of the second content 543B, the content area 542B of the email user interface 502 is scrolled, thereby moving the representations of the first content 543A and the second content 543B.

[0216] Figure 5AW It shows Figure 5AV The user interface 500 detects movement of finger contact 551G at a location in the content area 542B rather than at a representation location in the second content 543B. Figure 5AX This illustrates a response to the detection of movement of finger contact 551G at a location within content area 542B. Figure 5AW The user interface 500 includes an email user interface 502, wherein the representation of second content 543B (and first content 543A) moves according to the movement of a finger touch 551G. Therefore, in response to detecting movement of a finger touch within the content area 542B (at or not at the location of the representation of second content 543B), the content area 542B of the email user interface 502 is scrolled, thereby moving the representation of the first content 543A and the second content 543B.

[0217] Figure 5AY It shows Figure 5AB User interface 500, wherein finger contact 551H is detected at the location of the second message display 541B associated with the second email message, rather than at... Figure 5AB The first message in the message indicates that finger contact with 551A was detected at position 541A. Figure 5AZ This demonstrates a response to detecting finger contact with 551H at the location of the second message prompt 541B. Figure 5AY User interface 500. In response to detecting finger contact 551H at the location of the second message display 541B, email user interface 502 displays a second email message. Figure 5AZIn the email user interface 502, there are an information area 542A and a content area 542B. The information area 542A includes information about the second email message (e.g., sender, recipient, subject, and sending time), and the content area 542B includes the content of the second email message. The content area 542B includes a representation of the first content 546A in text form and the second content 546B in icon form.

[0218] Figure 5BA It shows Figure 5AZ The user interface 500, wherein finger contact 5511 is detected at the location represented by the second content 546B. Figure 5BB This illustrates a response to detecting finger contact 551I at the location indicated by the second content 546B. Figure 5BA User interface 500. In response to detecting finger contact 551I at the indicated location of the second contact 546B, user interface 500 includes a PDF viewer user interface 504B having second content 547. The PDF viewer user interface 504A displayed below device bar 503 includes an options bar 545A and a content area 545B. The options bar 545A includes a marker indication 516 for switching between an editing mode in which the content in the content area 545B can be edited (e.g., marked) and a viewing mode in which the content in the content area 545B can be viewed but not edited.

[0219] Figure 5BC It shows Figure 5AZ The user interface 500, wherein stylus contact 561E is detected at the location represented by the second content 546B. Figure 5BD This illustrates the response to detecting stylus contact 561E at the indicated location of the second contact 546B. Figure 5BC User interface 500. In response to detecting stylus contact 561E at the indicated location of second contact 546B, user interface 500 includes drawing user interface 501. Drawing user interface 501 includes second content 547 in content area 512 and includes toolbar area 513 having multiple drawing tool indications 532A-532D.

[0220] Figure 5BE User interface 500 is shown, which includes a notepad user interface 505 below device bar 503. Device bar 503 at the top of the display includes an identifier for device 100 (e.g., "iPad"), a wireless connectivity indicator, the current time, and a battery indicator indicating the charging level of device 100. User interface 500 includes notepad user interface 505 below device bar 501 and spans the rest of the display.

[0221] The Notepad user interface 505 includes an options bar 555A, a content creation area 555B, and a toolbar area 555C. The options bar 555A includes a sharing function 556A for sharing content in the content creation area 555B with other users, for example, by sending a copy of the content via email or sending a text message containing a copy of the content. The options bar 555A includes a deletion function 556B for deleting content in the content creation area 555B. The options bar 555A includes a new note function 556C for saving (and clearing) content in the content creation area 555B.

[0222] The content creation area 555B includes the first content in the form of typed text 557A. The content creation area 555B also includes a blank area 557B below the typed text 557A. The toolbar area 555C includes an undo indicator 531A (e.g., undo inserting content into the content creation area 555B) that can be used to reverse the user's last action (e.g., undo inserting content into the content creation area 555B) and a redo indicator 531B (e.g., redo inserting a drawing object) that can be used to restore an undo action. In various cases, if no undo or redo action exists, the undo indicator 531a and / or the redo indicator 531A are not displayed or are grayed out.

[0223] The toolbar area 555C includes multiple content insertion capabilities 558A-558C for inserting content into the content creation area 555B. The multiple content insertion capabilities include a checkbox insertion capability 558A for inserting checkboxes into the content creation area 555B. The multiple insertion capabilities include an image insertion capability 558B for inserting images into the content creation area 555B. In various embodiments, upon detecting a selection of the image insertion capability 558B, the device 100 displays an image selection user interface to assist the user in selecting an image to insert into the content creation area 555B. The multiple content insertion capabilities include a drawing insertion capability 558C for inserting drawings into the content creation area 555B. In various embodiments, after detecting a selection of the drawing insertion capability 558C, the notepad user interface 505 is replaced by a drawing user interface to assist the user in creating drawings. When drawing is complete, the drawing user interface is replaced with a notepad user interface 505 that includes the drawing in the content creation area 555B. The toolbar area 555C includes a display indicator 558D for showing the drawing tool selection indicator in the toolbar area and allowing online drawings to be inserted into the content creation area 555B, as further described below.

[0224] Figure 5BF It shows Figure 5BEThe user interface 500 detects finger contact 552A at the location where a drawing tool display 558D is shown. Figure 5BG This illustrates a response to detecting finger contact with 558A at the location of the display drawing tool indicator 558D. Figure 5BF User interface 500. Figure 5BG In this context, at least a portion of the blank area 557B is converted into a drawing insertion area 557C. Furthermore, a visual separator 557D is displayed between the typed text 557A and the drawing insertion area 557C. Figure 5BG In this mode, toolbar area 555C changes from general content insertion mode to online drawing mode and includes multiple drawing tool indicators 532A, 532B, 532E, 532H, and 532I. Toolbar area 555C also includes a color selection indicator 533 for selecting the color of drawing objects inserted into drawing insertion area 557C. Toolbar area 555C includes a hidden drawing tool indicator 558E, which changes toolbar area 555C back from online drawing mode to general content insertion mode.

[0225] Figure 5BG The movement of finger contact 552B detected at a location within the drawing insertion area 557C is shown. Figure 5BH This illustrates the response to the detection of movement of finger contact 552B within the drawing insertion area 557C. Figure 5BG User interface 500. Figure 5BH A stroke drawing object 559A is shown in the drawing insertion area 557C. The stroke drawing object 559A is a mark based on the movement of a finger contact 552B. Specifically, displaying the stroke drawing object 559A includes displaying a mark at the location where finger contact 552B is detected.

[0226] Figure 5BH The image shows finger contact 552C detected at the location of the hidden drawing tool indicator 558E. Figure 5 BI This demonstrates a response to detecting finger contact with 552C at the location of the hidden drawing tool display 558E. Figure 5BH User interface 500. Figure 5 BI In the drawing, the insertion area 557C is replaced by the drawing area 557E, which includes the stroke drawing object 559A. Figure 5 BI In the middle, the visual separator 557D stops displaying, and the toolbar area 555C changes to the general content insertion mode.

[0227] Figure 5BJ It shows Figure 5BE The user interface 500, wherein stylus contact 562A is detected at a location in the blank area 557B. Figure 5BKThis illustrates the response to detecting stylus contact 562A at a location within blank area 557B. Figure 5BJ The user interface 500. The blank area 557B is at least partially replaced by the drawing insertion area 557C, and a visual separator is displayed between the typed text 557A and the drawing insertion area 557C. The toolbar area 555C is changed to include drawing tool indications 532A, 532B, 532E, 532H, and 532I.

[0228] Figure 5BK The movement of the stylus contact 562B detected at a location within the drawing insertion area 557C is shown. Figure 5BL This illustrates the response to detecting movement of stylus contact 562B in the drawing insertion area 557C. Figure 5BK User interface 500. Figure 5BL A stroke drawing object 559A is shown in the drawing insertion area 557C. The stroke drawing object 559A is a mark based on the movement of the stylus contact 562B. Specifically, displaying the stroke drawing object 559A includes displaying a mark at the location where the stylus contact 562B is detected.

[0229] Figure 5BL The stylus contact 562C is shown at the location where the hidden drawing tool display 558E is located. Figure 5BM This illustrates the response to detecting stylus contact 562C at the location of the hidden drawing tool indicator 558E. Figure 5BL User interface 500. Figure 5BM In the drawing, the insertion area 557C is replaced by the drawing area 557E, which includes the stroke drawing object 559A. Figure 5BM In the middle, the visual separator 557D stops displaying, and the toolbar area 555C changes to the general content insertion mode.

[0230] Figure 5BN It shows Figure 5BE The user interface 500, wherein movement of stylus contact 562D is detected at a location in a blank area 557B. Figure 5B0 illustrates the response to the detection of movement of stylus contact 562D in the blank area 557B. Figure 5BN User interface 500. Figure 5BOIn this process, at least a portion of the blank area 557B is converted into a drawing insertion area 557C, and a visual separator 557D is displayed between the typed text 557A and the drawing insertion area 557C. Furthermore, the toolbar area 555C is changed to online drawing mode and includes multiple drawing tool indications 532A, 532B, 532E, 532H, and 532I. In response to the detection of pen contact 562D movement, the drawing insertion area 557C includes a stroke drawing object 559A. The stroke drawing object 559A is a mark based on the movement of the pen contact 562D. Specifically, displaying the stroke drawing object 559A includes displaying a mark at the location where pen contact 562D is detected.

[0231] Figure 5BO The stylus contact 562E is shown at the location where the hidden drawing tool indicator 558E is located. Figure 5BP A user interface 500 of Figure 5B0 is shown in response to the detection of stylus contact 562E at a location where the drawing tool indicator 558E is hidden. Figure 5BP In this process, the drawing insertion area 557C is replaced with a drawing area 557E that includes the stroke drawing object 559A. Figure 5BN In the middle, the first visual separator 557D stops displaying, and the toolbar area 555C is changed to the general content insertion mode.

[0232] therefore, Figure 5BE-5BP This shows how to insert drawing objects. Figure 5BE The content creation area 555B has three methods. Figure 5BF-5BI A method for inserting a drawing object using finger contact is shown, which includes making a finger contact at the display drawing tool indicator 558D and then moving the finger contact. Figure 5BJ-5BM A method for inserting a drawing object using a stylus contact is shown, wherein the finger contact includes a stylus contact at a location within a blank area 557B, and then the stylus contact is moved. Figure 5BN-5BP A method for inserting drawing objects using stylus contact is shown, starting with the movement of the stylus contact in the blank area 557B.

[0233] Figure 5BQ-5BW Two additional ways to insert a drawing object into the content creation area 555B when a drawing object already exists are shown. Figure 5BQ-5BT This demonstrates a method for inserting drawing objects using finger touch. Figure 5BU-5BW This demonstrates a method for inserting drawing objects using a stylus.

[0234] Figure 5BQ It shows Figure 5BP The user interface 500 detects finger contact 552D at a location within the drawing area 557E. Figure 5BR This illustrates a response to the detection of finger contact 552D at a location within the drawing area 557E. Figure 5BQ The user interface 500. Between the text input area 557A and the drawing insertion area 557C, a drawing area 557E, which is replaced by the drawing insertion area 557C (still including the stroke drawing object 559A) and a visual separator 557D, is displayed. The toolbar area 555C is changed to include drawing tool indications 532A, 532B, 532E, 532H, and 532I.

[0235] Figure 5BR The movement of finger contact 552E detected at a location within the drawing insertion area 557C is shown. Figure 5BS This illustrates a response to the detection of movement of finger contact 552E within the drawing insertion area 557C. Figure 5BR User interface 500. Figure 5BS The second stroke drawing object 559B (in addition to the stroke drawing object 559A) is shown in the drawing insertion area 557C.

[0236] Figure 5BS The image shows finger contact 552F detected at the location of the hidden drawing tool indicator 558E. Figure 5BT This demonstrates a response to detecting finger contact with 552F at the location of the hidden drawing tool indicator 558E. Figure 5BS User interface 500. Figure 5BT In the middle, the drawing insertion area 557C is replaced by the drawing area 557E, which includes the stroke drawing object 559A and the second stroke drawing object 559B. Figure 5BT In the middle, the visual separator 557D stops displaying, and the toolbar area 555C changes to the general content insertion mode.

[0237] Figure 5BU It shows Figure 5BP The user interface 500 detects movement of stylus contact 562F at a location within the drawing area 557E. Figure 5BV This illustrates a response to the detection of movement of stylus contact 562F at a location within the drawing area 557E. Figure 5BU The user interface 500. Between the typed text 557A and the drawing insertion area 557C, a drawing area 557E is displayed, which is replaced by the drawing insertion area 557C (still including the stroke drawing object 559A) and a visual separator 557D. The toolbar area 555C is changed to include drawing tool indications 532A, 532B, 532E, 532H, and 532I. In response to the detection of stylus contact 562F, the drawing insertion area 557C includes a second stroke drawing object 559B.

[0238] Figure 5BV The stylus contact 562G is shown at the location where the hidden drawing tool power indicator 558E is located. Figure 5BW This demonstrates the response to detecting stylus contact 562G at the location of the hidden drawing tool display 558E. Figure 5BV User interface 500. Figure 5BW In the middle, the drawing insertion area 557C is replaced by the drawing area 557E, which includes the stroke drawing object 559A and the second stroke drawing object 559B. Figure 5BW In the middle, the first visual separator 557D stops displaying, and the toolbar area 555C is changed to the general content insertion mode.

[0239] Figure 5BU-5BW The illustration shows how movement of the stylus contact within the drawing area results in the display of markers in the drawing area. Figure 5BX-5BY This demonstrates how moving a finger within the drawing area causes the content creation area to scroll.

[0240] Figure 5BX It shows Figure 5BW The user interface 500 detects movement of finger contact 552G within the drawing area 557E. Figure 5BY This illustrates the response to the detection of movement of finger contact 552G within the drawing area 557E. Figure 5BX The user interface is 500. Figure 5BY In this context, the content creation area 555B is scrolled upwards (based on the movement of the finger touch 552G). Specifically, the display of typed text 557A (and the display of drawing area 557E) moves upwards (based on the movement of the finger touch 552G).

[0241] Figure 5BN-5BP The text demonstrates how moving the stylus contact within a blank area results in a mark being displayed in that area. Figure 5BZ-5CA This demonstrates how finger movement within a blank area causes the content creation area to scroll.

[0242] Figure 5BZ It shows Figure 5BY The user interface 500 detects movement of finger contact 552H within the blank area 557F of the content creation area 555B below the drawing area 557E. Figure 5CA This illustrates a response to the detection of movement of a finger contact 552H within a blank area 557F. Figure 5BZ User interface 500. Figure 5CA In the middle, the content creation area 555B scrolls down (based on the movement of the finger touch 552H). Specifically, the display of typed text 557A (and the display of drawing area 557E) moves down (based on the movement of the finger touch 552H).

[0243] Figure 5CB It shows Figure 5CA The user interface 500 detects finger contact 552I at a location within the text being typed 557A. Figure 5CC This illustrates a response to detecting finger contact 552I at a location within the typed text 557A. Figure 5CB User interface 500. In response to detecting finger contact 552I at a location within typed text 557A, device 100 displays cursor 557G at the location within typed text 557A, and toolbar area 555C is changed to keyboard mode. In keyboard mode, toolbar area 555C has an undo indicator 531A for reversing the last action taken by the user (e.g., undoing inserted text) and a redo indicator 531B for resuming the undo action (e.g., redoing inserted text). In various cases, if no undo or redo action exists, undo indicator 531a and / or redo indicator 531A are not displayed or are grayed out. Toolbar area 555C includes a checkbox insertion indicator 558A for inserting checkboxes into content creation area 555B, an image insertion indicator 558B for inserting images into content creation area 555B, and a drawing insertion indicator 558C for inserting drawings into content creation area 555B. Toolbar area 555C includes a display drawing tool indicator 558D for displaying drawing tool selection indicators within the toolbar area 555C. Toolbar area 555C includes a keyboard with a multi-character indicator 558G for inserting text into the content creation area 555B. Toolbar area 555C includes a font options indicator 558F for changing the size or font of text inserted into the content creation area 555B. Toolbar area 555C includes a hidden keyboard indicator 558H for changing the toolbar area 555C to a general content insertion mode.

[0244] Figure 5CD It shows Figure 5CC The user interface 500, wherein finger contact 552J is detected at a location in typed text 557A within at least a predetermined threshold time. Figure 5CE This illustrates a response to the detection of contact 552J at a location in typed text 557A for at least a predetermined threshold time. Figure 5CD User interface 500. Figure 5CEA menu is shown at a location in the typed text, including a selection indicator 557H for selecting a portion of the typed text 557A, a text manipulation indicator 5571 for changing the characteristics (e.g., retain, italic, or underline) of at least a portion of the typed text 557A, and a drawing indicator 557J for inserting a drawing insertion area at the location of the typed text 557A. Figure 5CE The contact 552K detected at the location of the online plotted energy representation 557J is shown.

[0245] Figure 5CF This demonstrates the response to the detection of contact 552K at the location of the online plotting display 557J. Figure 5CE The user interface 500. In response to the selection of the online drawing display 557J, the drawing insertion area 557M is inserted into the content creation area 555B at the contact position 552K, and the typed text 557A is divided into a first part of typed text 557K and a second part of typed text 557L. A first visual separator 557N is displayed between the drawing insertion area 557M and the first part of typed text 557K, and a second visual separator 5570 is displayed between the drawing insertion area 557M and the second part of typed text 557M. The toolbar area 555C is changed to include drawing tool display indicators 532A, 532B, 532E, 532H, and 532I.

[0246] Figure 5CF The movement of finger contact 552L detected at a location within the insertion area 557M is shown. Figure 5CG This illustrates a response to the detection of movement of finger contact 552L within the drawing insertion area 557M. Figure 5CF User interface 500. Figure 5CG A stroke drawing object 559C is shown in the drawing insertion area 557M. The stroke drawing object 559C is a mark based on the movement of a finger contact 552L. Specifically, displaying the stroke drawing object 559M includes displaying a mark at the location where finger contact 552L is detected.

[0247] Figure 5CG The movement of contact 552Z detected at the location of the second visual separator 557O is shown. Figure 5CH This illustrates the response to the detection of movement of contact 552Z at the location of the second visual separator 557O. Figure 5CG User interface 500. Figure 5CH In the middle, the second visual separator 5570 moves downward to increase the size of the drawing insertion area 557M. Although the second visual separator 557O moves downward (as do the second part of the typed text 557L and the drawing area 557E), the stroke drawing object 559C does not move.

[0248] Figure 5CH The image shows a finger contact 552M detected at the location where the drawing tool display 558E is hidden. Figure 5C1 illustrates the response to the detection of finger contact 552M at the location where the drawing tool display 558E is hidden. Figure 5CH The user interface 500. In Figure 5C1, the drawing insertion area 557M is replaced by the drawing area 557P, which includes the stroke drawing object 559C. In Figure 5C1, the first visual separator 557N and the second visual separator 557O are no longer displayed, and the toolbar area 555C is changed to the general content insertion mode.

[0249] Figure 5CJ It shows Figure 5CI The user interface 500 detects movement of finger contact 552N at the location of the first part of the typed text 557K. Figure 5CK This illustrates a response to the detection of movement of finger contact 552N within the first portion of typed text 557K. Figure 5CJ User interface 500. Figure 5CK In the text, the text covered by the movement of a finger touching 552N is covered by a selection mark 557Q (e.g., the text is highlighted), and the selection mark 557Q indicates that the text covered by the movement of a finger touching 552N has been selected.

[0250] Figure 5CL User interface 500 is shown, which includes a Notepad user interface 505 below the device bar 503. The Notepad user interface 505 includes an options bar 555A, a content creation area 555B, and a toolbar area 555C. The content creation area 555B includes typed text 557A with a carriage return character displayed in gray. In various specific embodiments, the carriage return character is not displayed to the user; however, for ease of explanation and discussion, it is shown as a paragraph mark in the following figures. The content creation area 555B also includes a cursor 557G at the end of the typed text 557A.

[0251] Toolbar area 555C is displayed in keyboard mode. In keyboard mode, toolbar area 555C includes an undo indicator 531A for reversing the last action taken by the user (e.g., undoing inserted text) and a redo indicator 531B for resuming an undo action (e.g., redoing inserted text). In various cases, if no undo or redo action exists, the undo indicator 531a and / or redo indicator 531A are not displayed or are grayed out. Toolbar area 555C includes a checkbox insertion indicator 558A for inserting checkboxes into content creation area 555B, an image insertion indicator 558B for inserting images into content creation area 555B, and a drawing insertion indicator 558C for inserting drawings into content creation area 555B. Toolbar area 555C includes a display drawing tool indicator 558D for displaying drawing tool selection indicators in toolbar area 555C. Toolbar area 555C includes a keyboard with multiple character display 558G for inserting text into content creation area 555B. Toolbar area 555C includes a font options display 558F for changing the size or font of text inserted into content creation area 555B. Toolbar area 555C includes a hidden keyboard display 558H for changing toolbar area 555C to general content insertion mode.

[0252] The character display representation 558G used for inserting text into the content creation area 555B is the input display representation 558I used for inserting a carriage return character into the content creation area 555B. Figure 5CL The finger contact 553A is shown at the position of the input power indicator 558I. Figure 5CM This illustrates a response to the detection of finger contact 553A (and multiple subsequent contacts at the location of the input display 558I) at the position of the input display 558I. Figure 5CL User interface 500. Figure 5CL In the toolbar area 555B, the content creation area includes typed text 557A and multiple carriage returns below the typed text 557A. Its creation does not include visible content (because carriage returns are optionally not displayed to the user) and is a blank area of ​​the blank text insertion area 567A. The content creation area 555B includes a cursor 557C below the blank text insertion area 567A, and the toolbar area 555C is in keyboard mode, including multiple character display representations 558G.

[0253] Figure 5CN This illustrates the response to the detection of multiple contacts at the location of the character display 558G. Figure 5CMThe user interface 500. The content creation area 555B includes typed text 557A, a blank text insertion area 567A, appended typed text 567B, and a cursor 557G at the end of the appended typed text 567B. Figure 5CN The finger contact 553B is shown at the location of the hidden keyboard power indicator 558H. Figure 5CO This demonstrates a response to detecting finger contact with 553B at the location of the hidden keyboard power indicator 558H. Figure 5CN The user interface 500. In Figure 5C0, the toolbar area 555C is in general content insertion mode. Therefore, the toolbar area 555C includes an undo indicator 531A that can be used to reverse the last action taken by the user (e.g., undo inserting content into the content creation area 555B) and a redo indicator 531B that can be used to restore the undo action (e.g., redo inserting a drawing object). In various cases, if there is no undo or redo action, the undo indicator 531a and / or the redo indicator 531A are not displayed or are grayed out. The toolbar area 555C includes a checkbox insertion indicator 558A, an image insertion indicator 558B, a drawing insertion indicator 558C, and a display drawing tools indicator 558D.

[0254] Figure 5CP It shows Figure 5C The user interface 500 detects movement of finger contact 553C at a location within the blank text insertion area 567A. Figure 5CQ This illustrates a response to the detection of finger contact 553C at a location within the blank text insertion area 567A. Figure 5CP User interface 500. Figure 5CQ In this context, the content creation area 555B is scrolled upwards (e.g., based on the movement of a finger touch 553C). Specifically, the display of typed text 557A, blank text insertion area 567A, and appended typed text 567B is moved. Figure 5CQ The movement of stylus contact 563A is shown at the location of blank text insertion area 567A.

[0255] Figure 5CR This illustrates a response to the detection of movement of stylus contact 563A at a location within the blank text insertion area 567A. Figure 5CQ User interface 500. Figure 5CR In this context, the content creation area 555B scrolls down (e.g., based on the movement of the stylus touch 563A). Specifically, the display of typed text 557A, the blank text insertion area 567A, and the appended typed text 567B is moved. Figure 5CR The finger contact 553D is shown at the location in the blank text insertion area 567A.

[0256] Figure 5CS This illustrates a response to the detection of finger contact 553D at location 567A in the blank text insertion area. Figure 5CR User interface 500. Figure 5CS In the middle, the cursor 557G is displayed at the position where the finger touches 553D, and the toolbar area 555C is displayed in keyboard mode. Figure 5CS The image shows finger contact with 553E at the location indicated by the display of drawing tools, specifically at position 558D.

[0257] Figure 5 CT This illustrates a response to detecting finger contact with 553E at the location of the display drawing tool indicator 558D. Figure 5CS User interface 500. Figure 5 CT In the middle, the toolbar area 555C is changed to online drawing mode and includes multiple drawing tool displays 532A, 532B, 532E, 532H and 532I.

[0258] Figure 5CU It shows Figure 5 CT The user interface 500 detects finger contact 553F at a location within the blank text insertion area 567A (while displaying multiple drawing tool indications 532A, 532B, 532E, 532H and 532I). Figure 5CV This illustrates a response to detecting finger contact 553F at a location within the blank text insertion area 567A. Figure 5CU User interface 500. Figure 5CV In this context, a portion of the blank text insertion area 567A is converted into a blank drawing insertion area 567D. A first blank text insertion area 567C is left above the drawing insertion area 567D (and below the typed text 567A), and a second blank text insertion area 567E is left below the drawing insertion area 567D (and above the additional typed text 567B). The first blank text insertion area 567C and the drawing insertion area 567D are separated by a first visual separator 567F. The second blank text insertion area 567E and the drawing insertion area 567D are separated by a second visual separator 567G.

[0259] Figure 5CV The movement of finger contact 553G detected within the drawing insertion area 567D is shown. Figure 5CW This illustrates the response to the detection of finger contact 553G moving within the drawing insertion area 567D. Figure 5CV User interface 500. Figure 5CWA stroke drawing object 569A is shown in the drawing insertion area 567D. The stroke drawing object 569A is a mark based on the movement of a finger contact 553G. Specifically, displaying the stroke drawing object 569A includes displaying a mark at the location where finger contact 553G is detected.

[0260] Figure 5CX It shows Figure 5 CT The user interface 500 detects finger movement 553H at a location within the blank text insertion area 567A (while simultaneously displaying multiple drawing tool indications 532A, 532B, 532E, 532H, and 532I). As shown above (e.g., in... Figure 5BZ-5CA In various specific implementations, when the drawing tools are not displayed, movement of the finger touch will cause the scroll content creation area 555B to be scrolled. Figure 5CY This illustrates a response to the detection of finger contact 553H at a location within the blank text insertion area 567A. Figure 5CX User interface 500. Figure 5CY In this design, a portion of the blank text insertion area 567A is converted into a blank drawing insertion area 567D. A first blank text insertion area 567C is left above the drawing insertion area 567D (and below the typed text 567A), and a second blank text insertion area 567E is left below the drawing insertion area 567D (and above the additional typed text 567B). The first blank text insertion area 567C and the drawing insertion area 567D are separated by a first visual separator 567F. The second blank text insertion area 567E and the drawing insertion area 567D are separated by a second visual separator 567G. Therefore, in various specific embodiments, finger movement within the blank text insertion area (when the drawing tool indication is displayed) results in the same user interface as non-moving finger contact.

[0261] Figure 5CZ It shows Figure 5 CT The user interface 500 detects finger contact 5531 at the location of a blank area 567H in the content creation area 555B below the attached typed text 567B (while displaying multiple drawing tool indications 532A, 532B, 532E, 532H and 532I). Figure 5DA This illustrates the response to detecting finger contact 553I at location 567H in the blank area. Figure 5CZ User interface 500. Figure 5DA (similar to) Figure 5BG In the response shown, at least a portion of the blank area 567H is converted into a drawing insertion area 567I. Furthermore, a visual separator 557J is displayed between the additional typed text 567B and the drawing insertion area 567I.

[0262] Figure 5DA The movement of finger contact 553J detected at a location within the drawing insertion area 567I is shown. Figure 5DB This illustrates the response to the detection of movement of finger contact 553J in the drawing insertion area 567I. Figure 5DA User interface 500. Figure 5DB A stroke drawing object 559B is shown in the drawing insertion area 567I. The stroke drawing object 569B is a mark based on the movement of a finger contact 553J. Specifically, displaying the stroke drawing object 569B includes displaying a mark at the location where finger contact 553J is detected.

[0263] Figure 5DC It shows Figure 5 CT The user interface 500 detects finger contact 553K at a location within the additional typed text 567B (while displaying multiple drawing tool indications 532A, 532B, 532E, 532H and 532I). Figure 5DD This demonstrates a response to the detection of contact 553K at a location within the additional typed text 567B. Figure 5DC User interface 500. In response to detecting finger contact 553K at a position within the additional typed text 567B, device 100 displays cursor 557G at a position within the typed text 557A, and toolbar area 555C is changed to keyboard mode including multiple character display representations 558G for inserting additional text.

[0264] Figure 5DE It shows Figure 5CR The user interface 500, wherein stylus contact 563B is detected at a location in the blank text insertion area 567A (rather than) Figure 5CR (Finger touch 553D). Figure 5DF This illustrates the response to detecting stylus contact 563B at a location where blank text insertion area 567A is detected. Figure 5CR User interface 500. Figure 5DF In the middle, the cursor 557G is displayed at the position where the stylus touches 563B, and the toolbar area 555C is displayed in keyboard mode. Figure 5DF The stylus contact 563C is shown at the location of the display drawing tool indicator 558D.

[0265] Figure 5DG This illustrates a response to detecting stylus contact 563C at the location of the display drawing tool indicator 558D. Figure 5DF User interface 500. Figure 5DGIn the middle, the toolbar area 555C is changed to online drawing mode and includes multiple drawing tool displays 532A, 532B, 532E, 532H and 532I. Figure 5DG The stylus contact 563D detected at a location within the blank text insertion area 567A is shown (e.g., when multiple drawing tool indications 532A, 532B, 532E, 532H, and 532I are displayed).

[0266] Figure 5DH This illustrates the response to detecting stylus contact 563D at a location within the blank text insertion area 567A. Figure 5DG User interface 500. Figure 5DH In this process, a portion of the blank text insertion area 567A is converted into a drawing insertion area 567J, which includes a mark 569C at the location of the stylus contact 563D. This conversion leaves a first blank text insertion area 567C above the drawing insertion area 567J (and below the typed text 567A), and a second blank text insertion area 567E below the drawing insertion area 567J (and above the additional typed text 567B). Figure 5DH In the diagram, the first blank text insertion area 567C and the drawing insertion area 567J are separated by a first visual separator 567F. The second blank text insertion area 567E and the drawing insertion area 567J are separated by a second visual separator 567G. In various specific embodiments, the marker 569C is a point or stroke drawing object 569C with a length of zero or minimum length.

[0267] Figure 5DI It shows Figure 5DG The user interface 500 detects movement of stylus contact 563E at a location within the blank text insertion area 567A (rather than...). Figure 5DG (Non-moving stylus contact 563D). Figure 5 DJ This illustrates a response to the detection of movement of stylus contact 563E at a location within the blank text insertion area 567A. Figure 5DI The user interface 500 (simultaneously displaying multiple drawing tools) is represented by 532A, 532B, 532E, 532H, and 532I. Figure 5 DJIn this process, a portion of the blank text insertion area 567A is converted into a drawing insertion area 567K that includes the stroke drawing object 569A. This conversion leaves a first blank text insertion area 567C above the drawing insertion area 567K (and below the typed text 567A), and a second blank text insertion area 567E below the drawing insertion area 567K (and above the additional typed text 567B). The first blank text insertion area 567C and the drawing insertion area 567K are separated by a first visual separator 567F. The second blank text insertion area 567E and the drawing insertion area 567K are separated by a second visual separator 567G.

[0268] Figure 5DK It shows Figure 5 DJ The user interface 500, wherein the movement of the stylus contact 563F originates from the drawing insertion area 567K and passes through the second visual separator 567G into the second blank text insertion area 567E. Figure 5DL This demonstrates the response to detecting movement of the stylus contact 563F. Figure 5DK User interface 500. Figure 5DL In the middle, the second visual separator 567G moves downward, indicating an increase in the size of the drawing insertion area 567K. As the second blank text insertion area 567E has been partially converted into a part of the drawing insertion area 567K (leaving the third blank text insertion area 567L), the size of the drawing insertion area 567K increases. Furthermore, the drawing insertion area 567K (in addition to the stroke drawing object 569A) includes a second stroke drawing object 569D. The second stroke drawing object 569D is a marker based on the movement of the stylus contact 563F. Specifically, displaying the second stroke drawing object 569D includes displaying a marker at the location where the stylus contact 563F is detected.

[0269] Figure 5DM It shows Figure 5DL The user interface 500 detects movement of stylus contact 563G at the location of the second visual separator 567G. Figure 5DN This illustrates a response to the detection of movement of stylus contact 563G at the location of the second visual separator 567G. Figure 5DM User interface 500. Figure 5DN In the middle, the second visual separator 567G moves downward to increase the size of the drawing insertion area 567K, thereby converting the third blank text insertion area 567L into a part of the drawing insertion area 567K. Although the second visual separator 567G moves downward (as does the additional typed text 567B), the stroke drawing object 569A and the second stroke drawing object 569D do not move.

[0270] Figure 5DOThe movement of the stylus contact 563H detected at a position within the drawing insertion area 567K is shown. Figure 5DP This illustrates the response to detecting movement of stylus contact 563H within the drawing insertion area 567K. Figure 5DO The user interface 500 includes a drawing insertion area 567K comprising a stroke drawing object 569A and a second stroke drawing object 569D. Furthermore, the drawing insertion area 567K includes a third stroke drawing object 569E. The third stroke drawing object 569E is a marker based on the movement of the stylus contact 563H. Specifically, displaying the third stroke drawing object 569E includes displaying a marker at the location where the stylus contact 563H is detected.

[0271] Figure 5DQ It shows Figure 5DG The user interface 500 detects movement of stylus contact 563I at the location of the blank area 567H below the attached typed text 567B (rather than at...). Figure 5DG The stylus touches 563D in the blank text insertion area 567A. Figure 5DR This illustrates the response to detecting movement of stylus contact 563I in blank area 567H. Figure 5DQ The user interface 500 (simultaneously displays multiple drawing tools as indicated by 532A, 532B, 532E, 532H and 532I). Figure 5DR A stroke drawing object 559B is shown in the drawing insertion area 567I. Therefore, the blank area 567H has been at least partially converted into the drawing insertion area 567I. The stroke drawing object 569B is a mark based on the movement of the stylus contact 563I. Specifically, displaying the stroke drawing object 569B includes displaying a mark at the location where the stylus contact 563I is detected.

[0272] Figures 6A-6C A flowchart illustrating a method 600 for manipulating drawing objects according to some embodiments is shown. In an electronic device having a display and one or more input devices (e.g., Figure 1A Portable multi-functional device 100 or Figure 3 Method 600 is performed at device 300. In some embodiments, the display is a touchscreen display and one or more input devices are on or integrated with the display. In some embodiments, the display is separate from one or more input devices. Some operations in method 600 may optionally be combined, and / or the order of some operations may optionally be changed.

[0273] As described below, method 600 provides an intuitive way to manipulate drawing objects. This method reduces the cognitive burden on the user when manipulating drawing objects, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to manipulate drawing objects faster and more efficiently saves power and increases the time between battery charging cycles.

[0274] The device displays (602) a drawing user interface on the monitor, which includes multiple drawing objects. For example, in Figure 5C In the process, device 100 displays a drawing user interface 501, which includes multiple drawing objects, including a square drawing object 521A (approximately surrounding the grapes in the fruit plate), a magnifying glass drawing object 521B (magnifying a portion of the apple leaves in the fruit plate), a text drawing object 521C (labeling "LEMON" on the lemon in the fruit plate), a set of three-stroke drawing objects 521D (affecting the smiley face on the apple), and a set of six-stroke drawing objects 521E (affecting the flowers on the plate).

[0275] In some implementations, the drawing user interface includes (604) a toolbar area that includes multiple drawing tool displays associated with multiple drawing tools, respectively. For example, in Figure 5A In this drawing user interface 501, a toolbar area 503 includes multiple drawing tool indications, including a pen tool selection indication 532A associated with a pen tool for creating opaque strokes, a highlighter tool selection indication 532B associated with a highlighter tool for creating semi-transparent strokes, an delete tool selection indication 532C associated with a delete tool for deleting drawing objects, and a selection tool selection indication 532D associated with a selection tool for selecting drawing objects. By displaying multiple drawing tool indications, the user interface provides the user with an efficient mechanism for selecting drawing tools, thereby reducing the amount of user interaction required to perform various predefined operations on drawing objects. Reduced user interaction reduces device wear and tear. Reduced user interaction also leads to faster initiation of predefined operations, and thus reduces power consumption during the execution of predefined operations, thereby extending device battery life.

[0276] In some implementations, the device detects (606) the selection of user input at a location in one of a plurality of drawing tools selection indications associated with a predefined operation, via one or more input devices. For example, in Figure 5C In the process, device 100 detects contact 550B at the location of deletion tool selection indicator 532C associated with the deletion tool.

[0277] In some implementations, the drawing user interface includes (608) content that labels multiple drawing objects. For example, in Figure 5C In the drawing user interface 501, there are photos marked by multiple drawing objects 521A-521E.

[0278] In some implementations, multiple drawing objects include (610) stroke drawing objects defined by sequential user input within the drawing user interface when a drawing tool associated with a stroke operation is selected. For example, in Figure 5C In the drawing, multiple drawing objects include a set of three-stroke drawing objects 521D (affecting the smiley face on the apple) and a set of six-stroke drawing objects 521E (affecting the flower on the plate). Furthermore, in Figure 5C In the drawing user interface 501, there are pen tool indications 532A associated with pen tools that can be used to create opaque strokes and highlighter tool indications 532B associated with highlighter tools that can be used to create semi-transparent strokes.

[0279] In some implementations, the plurality of drawing objects includes (612) at least one of a shape drawing object, a stroke drawing object, a magnifying glass drawing object, or a text drawing object. For example, in Figure 5C The drawing objects include shape drawing objects in the form of square drawing objects 521A, stroke drawing objects in the form of either stroke drawing objects 521D or 521E, magnifying glass drawing objects 521B, and text drawing objects 521C. Presenting various drawing objects and the ability to insert such objects provides users with an efficient mechanism for annotating content in various ways, thereby reducing user interaction with the annotated content. Reduced user interaction reduces device wear and tear and power consumption for annotation, thus extending device battery life.

[0280] When a drawing tool associated with a predefined operation is selected, the device detects (614) user input via one or more input devices to define a path within the drawing user interface. For example, in Figure 5D In the process, when the delete tool is selected, device 100 detects contact 550C moving to define a path within the drawing user interface 501. As another example, in... Figure 5I In the process, when the delete tool is selected, device 100 detects contact 550E moving to define a path within the drawing user interface. As another example, in... Figure 5K In the process, when the selection tool is selected, device 100 detects a movement of contact 550G (by...) to define a path within the drawing user interface. Figure 5M(See diagram 522B for the selection path). Allowing users to manipulate drawing objects using mobile user input provides an efficient mechanism, enabling users to manipulate multiple drawing objects with a single gesture, thereby reducing the amount of user interaction required to manipulate multiple drawing objects. Reduced user interaction reduces device wear and tear. Reduced user interaction also leads to faster manipulation of drawing objects, thus reducing the power consumption required to perform the manipulation and extending the device's battery life.

[0281] In response to detected user input defining a path within the drawing user interface, the device performs (616) a predefined operation relative to two or more of a plurality of drawing objects intersecting the path. For example, in Figure 5E In response to the detection of contact 550C that moves to define a path, device 100 stops displaying three of a set of six-stroke drawing objects 521E. As another example, in Figure 5M In response to the detection of a movement to define a path contact 550G, device 100 selects a set of three-stroke drawing objects 521D.

[0282] In some implementations, when a predefined operation is performed, the device stops (618) to display two or more of a plurality of drawing objects that intersect the path. For example, in Figure 5E In response to the detection of contact 550C, device 100 stops displaying three of a set of six-stroke drawing objects 521E. In some embodiments, when stopping the display of two or more of a plurality of drawing objects intersecting the path, the device stops (620) to display another of a plurality of objects intersecting one or more of a plurality of drawing objects intersecting the path. For example, in Figure 5F In response to the detection of contact 550C, device 100 stops displaying all of a set of six-stroke drawing objects 521E. Allowing users to delete multiple drawing objects with a single gesture reduces the amount of user interaction required to perform such operations. Reduced user interaction reduces device wear and leads to faster startup, thereby reducing power consumption and extending device battery life.

[0283] In some implementations, during the execution of a predefined operation, if the speed of user input used to move and define a path within the graphical user interface exceeds a speed threshold, the device stops (622) to display two or more of the multiple drawing objects intersecting the path, and continues to display two or more of the multiple drawing objects intersecting the path if the speed does not exceed the speed threshold. For example, in Figure 5E In the middle, device 100 stops displaying three of the six-stroke drawing objects 521E. However, in Figure 5IIn the middle, device 100 continues to display the square drawing object 521A. Selectively deleting drawing objects based on the speed of user input prevents unintentional deletion due to accidental rapid scanning of the drawing object, thus avoiding the need for further user input to cancel the deletion operation and wasting time and device battery power.

[0284] In some implementations, when performing a predefined operation, the device selects (624) two or more of a plurality of drawing objects that intersect the path. For example, in Figure 5M In this process, device 100 selects at least the left two strokes of a set of three-stroke drawing objects 521D. Allowing users to select multiple drawing objects with a single gesture reduces the amount of user interaction required to perform such operations. Reduced user interaction reduces device wear and tear and results in faster startup, thereby reducing power consumption and extending device battery life.

[0285] In some implementations, when selecting two or more drawing objects intersecting with a path, the device selects (626) another object intersecting with one or more of the drawing objects intersecting with the path. In some implementations, when selecting two or more drawing objects intersecting with a path, the device selects (628) another object enclosed by the path. For example, in Figure 5M In this setup, the device selects all of a set of three-stroke drawing objects 521D. Selecting drawing objects other than those intersecting the path also provides the user with an efficient mechanism to select (and subsequently manipulate) multiple drawing objects with a single gesture. Reduced user interaction decreases device wear and leads to faster initiation of drawing object manipulation, thereby reducing power consumption and extending device battery life.

[0286] In some implementations, after selecting two or more of a plurality of drawing objects intersecting the path, the device detects (630) repositioning user input as the drawing user interface moves. For example, in Figure 5N In this context, device 100 detects a touch 550H moving within the drawing user interface 501. In response to the detected repositioning user input, the device moves (632) two or more of a plurality of drawing objects based on the repositioning user input. For example, in Figure 50, device 100 moves a set of three-stroke drawing objects 521D downwards based on the movement of touch 550H. Allowing the user to move multiple drawing objects with a single gesture reduces the amount of user interaction required to perform such an operation. Furthermore, moving multiple drawing objects as a whole provides an efficient mechanism for moving drawing objects while maintaining their relative positions. Reduced user interaction reduces device wear and leads to faster startup, thereby reducing power consumption and extending device battery life.

[0287] In some implementations, when two or more of a plurality of drawing objects intersecting the path are selected, the device displays (634) a selection path graphic corresponding to that path. For example, in Figure 5M In this embodiment, device 100 displays a selection path graphic 522B corresponding to the path defined by the movement of contact 550G. In some embodiments, the selection path graphic includes (636) partially transparent alternating shadows and white dashed lines. In some embodiments, displaying the selection path graphic includes animate (638) the selection path graphic. In some embodiments, when the selection path graphic is displayed, the device detects repositioning user input that moves with the drawing user interface. For example, in... Figure 5N In this process, device 100 detects (640) a movement of contact 550H within the drawing user interface 501. In response to the detected repositioning user input, device 100 stops (642) the display of the selected path graphic and moves two or more of the multiple drawing objects according to the repositioning user input. For example, in Figure 5O In this process, device 100 stops displaying the path selection graphic 522B and moves a set of three-stroke drawing objects 521D downwards based on the movement of contact 550H. Displaying the path selection graphic provides visual feedback to the user, indicating that the drawing object has been selected, thereby preventing accidental manipulation of the drawing object (leading to further user interaction to undo the manipulation) and preventing manipulation of some but not all of the drawing objects that the user wishes to manipulate (leading to further user interaction to manipulate the remaining drawing objects). Preventing unnecessary user interaction reduces wear and tear on the device and, for battery-powered devices, extends battery life.

[0288] In some implementations, when performing a predefined operation relative to two or more of a plurality of drawing objects intersecting the path, the device erases (644) the portions intersecting the path from two or more of the plurality of drawing objects intersecting the path, without deleting one or more portions not intersecting the path. For example, in Figure 5H In the process, the device erases a portion of a group of six-stroke drawing objects 521E that intersect with the path, while not erasing the portions that do not intersect with the path.

[0289] In some implementations, when performing a predefined operation relative to two or more of the multiple drawing objects intersecting the path, the device changes (646) the color of two or more of the multiple drawing objects intersecting the path, changes the size of two or more of the multiple drawing objects intersecting the path, or changes the opacity of two or more of the multiple drawing objects intersecting the path. For example, in Figure 5RIn this example, device 100 changes the color of two of the three-stroke drawing objects 521D and changes the color of the text drawing object 521C. As another example, in... Figure 5T In this process, device 100 changes the size (e.g., line width) of two of a set of three-stroke drawing objects 521D, and changes the size (e.g., line width) of a square drawing object 521A.

[0290] It should be understood that, Figures 6A-6C The specific order in which the operations described herein are presented is merely exemplary and not intended to indicate that the order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that the details of other processes described herein in conjunction with other methods (e.g., methods 700, 800, and 900) are similarly applicable to the above-described combinations. Figures 6A-6C The method 600. For example, the user interface, user input (including finger contact and stylus contact), drawing object, and drawing tool indication described above with reference to method 600 may optionally have one or more of the features of the user interface, user input (including finger contact and stylus contact), drawing object, and drawing tool indication described herein with reference to other methods (e.g., methods 700, 800, and 900). For the sake of brevity, these details will not be repeated here.

[0291] Figures 7A-7B A flowchart of a method 700 for marking content according to some embodiments is shown. In an electronic device having a display and one or more input devices (e.g., Figure 1A Portable multi-functional device 100 or Figure 3 Method 700 is performed at device 300. In some embodiments, the display is a touchscreen display and one or more input devices are on or integrated with the display. In some embodiments, the display is separate from one or more input devices. Some operations in method 700 are optionally combined, and / or the order of some operations is optionally changed.

[0292] As described below, method 700 provides an intuitive way to annotate content. This method reduces the cognitive burden on users when annotating content, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to annotate content faster and more efficiently saves power and increases the time between battery charging sessions.

[0293] The device displays (702) a representation of first content and a second content associated with the first content on a display. For example, in Figure 5ACIn the device 100, an email user interface 502 is displayed. The email user interface 502 includes a content area 542B representing a first content 543A in the form of text for an email message and a second content 543B in the form of an image, representing an image attached to the email message. As another example, in Figure 5AZ In the device 100, an email user interface 502 is displayed. The email user interface 502 includes a content area 542B that represents a first content 546A in the form of text for an email message and a second content 546B in the form of icons, representing a PDF file attached to the email message.

[0294] In some implementations, the representation of the second content includes (704) an image. For example, in Figure 5AC In this context, the representation of the second content 543B includes an image. In some embodiments, the second content includes (706) a magnified version of the image. For example, in Figure 5AE The second content, 544, includes... Figure 5AC The second content 543B represents a magnified version.

[0295] In some implementations, the representation of the second content includes icons. For example, in Figure 5BA In this context, the representation of the second content 546B includes icons. In some embodiments, the second content includes attachments. For example, in... Figure 5BB The second content, 547, includes a PDF file.

[0296] The device detects (708) user input at the location of the representation of the second content via one or more input devices. For example, in Figure 5AD In this context, device 100 detects finger contact 551B at the location indicated by the second content 543B. As another example, in... Figure 5AP In this process, the device detects stylus contact 561C at the location represented by the second content 543B. As described above, in some embodiments, the representation of the second content includes an image. Furthermore, in some embodiments, the user input defines (710) a path from a first location within the image to a second location within the image. For example, in Figure 5AR In the process, device 100 detects the movement of stylus contact 561D from a first position within the representation of second contact 543B to a second position within the representation of second content 543B.

[0297] In response to detecting user input at the location of the second content and determining that the user input includes stylus contact, the device displays (710) a drawing user interface including the second content and a toolbar area with multiple drawing tool indications on the display. For example, in Figure 5AQIn response to detecting stylus contact 561C at the location indicated by the second content 543B, device 100 displays a drawing user interface 501, which includes the second content 544 and a toolbar area 513 with multiple drawing tool selection indications 532A-532D. Accelerating the display of the drawing user interface in response to detecting a stylus contact more likely to be used for marking content reduces the amount of user interaction with the marked content. Reduced user interaction reduces wear and tear on the device. Reduced user interaction also leads to faster initiation of marking operations, and thus reduces power consumption during marking operations, thereby extending the device's battery life.

[0298] In some implementations, when a user inputs a path defining a path from a first location within an image to a second location within an image, the device displays (714) a drawing object corresponding to that path in the drawing user interface. For example, in Figure 5AS In the middle, device 100 displays and Figure 5AP The stroke drawing object 521G corresponds to the path of the stylus contact 561D. As another example, in Figure 5AT In the middle, device 100 displays and Figure 5AR The stroke drawing object 521H corresponds to the path of the stylus contact 561D. In some embodiments, the drawing object corresponding to the path includes (716) a scaled-up version of the path. For example, in Figure 5AS In the middle, device 100 displays a 521G stroke drawing object, which is Figure 5AR The stylus touches a magnified version of the path of the 561D. In some implementations, the drawing object corresponding to the path includes (718) a version of the path with the same size as the path defined by the user input. For example, in Figure 5AT In the middle, device 100 displays and Figure 5AR The stylus touches the stroke drawing object 521H with the same path size as the pen contact 561D. This provides an efficient mechanism for users to mark content directly from the embedded representation, thereby reducing the amount of user interaction with the device. Reduced user interaction reduces device wear. Reduced user interaction also leads to faster initiation of marking operations, and thus reduces the power consumption of performing marking operations, thereby extending the device's battery life.

[0299] In response to detecting a user at the location of the second content representation and determining that the user input does not include stylus contact, the device abandons (720) the display of the drawing user interface. For example, in Figure 5AE In response to detecting finger contact 551B at the location represented by the second content 543B, device 100 displays a photo viewer user interface 504B. As another example, in Figure 5AVIn response to detecting a finger touch 551F moving at the position indicated by the second content 543B, the device 100 scrolls the content area 542B of the email user interface 502.

[0300] In some implementations, when the drawing user interface is not displayed, the device displays (722) a viewer user interface including second content, without multiple drawing tool indications. For example, in Figure 5AE In this model, the photo viewer user interface 504B includes second content 544 without multiple drawing tool indications. Displaying the viewer user interface in response to, for example, detecting a finger touch more likely to be used for viewing content (e.g., scrolling and / or zooming) reduces user interaction when viewing content. This reduction in user interaction reduces device wear. It also results in faster initiation of viewing operations, and thus reduces power consumption during viewing operations, thereby extending device battery life.

[0301] In some implementations, when the viewer user interface is displayed, the device detects (724) user input via one or more input devices to select a graphical representation of the viewer user interface. For example, in Figure 5AE In this embodiment, device 100 detects finger contact 551C at the location of drawing indicator 516 in the photo viewer user interface 504A. In some implementations, in response to detecting user input selecting a drawing indicator, the device displays (726) a drawing user interface on the display, which includes second content and a toolbar area with multiple drawing tool selection indicators. For example, in... Figure 5AF In this device 100, a drawing user interface 501 is displayed, which includes second content 544 and a toolbar area 513 with multiple drawing tool display indicators 532A-532D. Allowing the user to switch from the viewer user interface to the drawing user interface via the drawing display provides an efficient mechanism for marking content. Reduced user interaction reduces device wear. Reduced user interaction also leads to faster initiation of marking operations, and thus reduces power consumption during marking operations, thereby extending the device's battery life.

[0302] In some implementations, when the viewer user interface is displayed, the device detects (724) user input at the location of the second content via one or more input devices. For example, in Figure 5AG In this context, device 100 detects stylus contact 561A at the location of the second content 544. As another example, in... Figure 5AL In the middle, device 100 detects finger contact 551D at the position of the second content 544.

[0303] In some implementations, based on user input determining the location of the second content, including stylus contact, the device displays (730) a drawing user interface on the display, which includes the second content and a toolbar area displaying multiple drawing tools. For example, in Figure 5AH In response to the detection of stylus contact 561A, device 100 displays a drawing user interface 501 including second content 544 with multiple drawing tool indications 532A-532D. Accelerating the display of the drawing user interface in response to the detection of a stylus contact more likely to be used for marking content reduces the amount of user interaction with the marked content. Reduced user interaction reduces wear and tear on the device. Reduced user interaction also leads to faster initiation of marking operations, and thus reduces power consumption during marking operations, thereby extending the device's battery life.

[0304] In some implementations, if user input at the location of the second content does not include stylus contact, the device abandons (732) the display of the drawing user interface. For example, in Figure 5AM In response to detecting finger contact 551D, device 100 displays second content 544 in full-screen mode. In some embodiments, when abandoning the display of the drawing user interface, the device moves (734) the display of the second content according to the movement of the second content's position based on user input. For example, in Figure 5AO In this process, device 100 moves the display of second content 544 according to the movement of finger contact 551E at the position of the second content.

[0305] In some implementations, while displaying representations of first and second content, and when abandoning the display of the drawing user interface, the device moves the display of the first content and the representation of the second content according to user input and movement of the position of the second content representation. For example, in Figure 5AV In this process, device 100 moves the display of first content 543A and the representation of second content 543B according to the movement of finger contact 551F at the representation position of the second content.

[0306] It should be understood that, Figures 7A to 7B The specific order of operations described herein is merely exemplary and not intended to indicate that the order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that the details of other processes described herein in conjunction with other methods (e.g., methods 600, 800, and 900) are similarly applicable to the above-described combinations. Figures 7A-7BThe method 700. For example, the user interface, user input (including finger contact and stylus contact), drawing object, and drawing tool display shown above with reference to method 700 may optionally have one or more of the features of the user interface, user input (including finger contact and stylus contact), drawing object, and drawing tool selection display shown herein with reference to other methods (e.g., methods 600, 800, and 900). For the sake of brevity, these details will not be repeated here.

[0307] Figures 8A-8C A flowchart of a method 800 for creating an area by manipulating content according to some embodiments is shown. Method 800 is used in electronic devices having a display and a touch-sensitive surface (e.g., Figure 1A Portable multi-functional device 100, or Figure 3 The operation is performed at device 300. In some embodiments, the display is a touch-sensitive display, and the touch-sensitive surface is on or integrated with the display. In some embodiments, the display and the touch-sensitive surface are separate. Some operations in method 800 are optionally combined, and / or the order of some operations is optionally changed.

[0308] As described below, method 800 provides an intuitive way to manipulate a content creation area. This method reduces the cognitive burden on the user when manipulating the content creation area, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to manipulate the content creation area faster and more efficiently saves power and increases the time between battery charging sessions.

[0309] The device displays a content creation area (802) on a touch-sensitive display, which includes typed text and one or more blank areas without content. For example, in Figure 5BE In the device 100, a notepad user interface 505 is displayed. The notepad user interface 505 includes a content creation area 555B for typing text 557A and a blank area 557B without content. As another example, in Figure 5BZ In the middle, device 100 displays a content creation area 555B, which has typed text 557A and a blank area 557F that does not contain content.

[0310] The device detects (804) user input on the touch-sensitive display, which includes movement of the touch within the content creation area. For example, in Figure 5BN In this process, device 100 detects the movement of stylus contact 562D within the content creation area 555B. As another example, in... Figure 5BZ In the process, device 100 detects the movement of finger contact 552H within the content creation area 555B.

[0311] In response to detecting user input and determining that the user input includes movement of stylus contact in a corresponding blank area, the device draws (806) a mark based on the movement of the stylus contact during user input. For example, in FIG5B0, device 100 draws a mark based on stylus contact 562D during user input (in... Figure 5BN The marker (in the form of a stroke drawing object 559A) is drawn by moving within a blank area 557B during the period shown in Figure 5B0. In some embodiments, while drawing the marker, the device displays (808) the marker on a touch-sensitive display at the location where stylus contact is detected during user input. For example, in Figure 5B0, the device displays the stroke drawing object 559A (in the form of a stroke drawing object 559A) at the location where stylus contact 562D is detected. Figure 5BN (In Chinese). Allowing users to directly annotate content with a stylus provides an efficient mechanism for user-defined content annotation, thereby reducing user interaction with the device. Reduced user interaction reduces device wear and tear. Reduced user interaction also leads to faster initiation of annotation operations, and thus reduces power consumption during annotation, extending device battery life.

[0312] In some embodiments, when marking, the device displays (810) a toolbar area on a touch-sensitive display, which includes multiple drawing tool indications for selecting the appropriate drawing tool. For example, in FIG5B0, device 100 displays a toolbar area 555C, which includes multiple drawing tool indications 532A, 532B, 532E, 532H, and 532I for selecting the appropriate drawing tool. In some embodiments, when marking, the device displays (812) an undo indication for removing the mark on a touch-sensitive display. For example, in FIG5B0, device 100 displays an undo indication 531A. Undo indications provide an efficient mechanism for users to undo operations, thereby reducing the amount of user interaction required to undo unexpected or unsatisfactory operations. Reduced user interaction reduces wear and tear on the device. Reduced user interaction also leads to faster undo operations, thereby reducing power consumption for performing storage management operations and extending the device's battery life.

[0313] In some implementations, when marking is being done, the device displays (814) a visual separator on a touch-sensitive display between the corresponding blank area and the typed text. For example, in FIG5B0, device 100 displays a visual separator 557D between the black area 557C and the typed text 557A. Displaying the visual separator provides the user with visual feedback indicating the location of the drawing insertion area, thereby preventing accidental manipulation of text or other content outside the drawing insertion area (leading to further user interaction to undo the manipulation). Preventing unnecessary user interaction reduces wear and tear on the device and, for battery-powered devices, extends battery life.

[0314] Based on the determined user input, including finger movement within the corresponding blank area, the device scrolls (816) the content creation area based on the finger movement during user input. For example, in Figure 5CA In China, based on Figure 5BZ The movement of a finger touching 552H scrolls the content creation area 555B. In some implementations, while scrolling the content creation area, the device moves (818) the display of typed text. For example, as... Figure 5BZ and Figure 5CA In between, the typed text 557A moves in response to the movement of the finger contact 552H. This provides an efficient mechanism for performing two different operations in response to the same gesture (movement of contact), based on whether the gesture is performed with a stylus or a finger, thereby reducing the amount of user interaction with the device to perform at least one operation. Reducing user interaction with the device reduces wear and tear on the device, and for battery-powered devices, extends battery life.

[0315] In some implementations, the device detects (820) contact at the location in the typed text. For example, in Figure 5CB In this embodiment, the device detects finger contact 5521 at the location in the typed text 557A. In some implementations, in response to the detection of contact at the location in the typed text, the device displays (822) a cursor at the location in the typed text and a keyboard for inserting additional typed text on a touch-sensitive display. For example, in Figure 5CC In the device 100, a cursor 557G is displayed at the position where a finger touches 5521, and a keyboard 558G is used to insert additional typed text.

[0316] In some implementations, the device detects (824) movement of the contact at a location within the typed text. For example, in Figure 5CJ In this embodiment, device 100 detects movement of contact 552N within a first portion of the typed text 557K. In some implementations, in response to detecting movement of the contact within the typed text, the device selects (826) the typed text covered by that movement. For example, in... Figure 5CK In this configuration, the typed text covered by the movement of a finger touching 552N is covered by a selection marker 557Q (e.g., the text is highlighted), and the selection marker 557Q indicates that the typed text covered by the movement of a finger touching 552N has been selected. Selecting text in response to movement of a finger touching the text provides a more efficient mechanism for users to select text, thereby reducing the amount of user interaction required to select text (and perform actions on the selected text). Reducing user interaction with the device reduces wear and tear on the device, and for battery-powered devices, extends battery life.

[0317] In some implementations, the device detects (828) contact at a location in the typed text for at least a predefined threshold amount of time. For example, in Figure 5CD In this embodiment, device 100 detects contact 552J at a location in the typed text 557A for at least a predefined threshold time. In some implementations, in response to detecting contact at a location in the typed text for at least the predefined threshold time, the device displays (830) a menu on a touch-sensitive display at the location in the typed text, the menu including an indication for inserting a drawing insertion area at the location in the typed text. For example, in... Figure 5CE In this context, device 100 displays a menu at the location indicated in the typed text 557A. The menu includes an online drawing indicator 557J for inserting a drawing insertion area at the location indicated in the typed text. The online drawing indicator provides the user with an efficient mechanism to insert the drawing insertion area (and marking content), thereby reducing the amount of user interaction required to perform marking operations. Reduced user interaction with the device reduces wear and tear on the device and, for battery-powered devices, extends battery life.

[0318] In some implementations, the device detects contact (832) at the location of the indicator used for inserting the drawing insertion area. For example, in Figure 5CE In this embodiment, the device detects contact 552K at a location on the online drawing indicator 557J. In some implementations, in response to detecting contact at a location on the indicator for inserting a drawing insertion area, the device moves at least one of a first portion or a second portion of the typed text to introduce a blank drawing insertion area between the first and second portions. For example, in... Figure 5CF In this embodiment, device 100 moves the second portion of typed text 557M to introduce a blank drawing insertion area 557M between the first portion of typed text 557K and the second portion of typed text 557L. In some embodiments, in response to detecting contact at a location indicated for inserting the drawing insertion area, the device displays (836) a first visual separator between the drawing insertion area and the first portion and a second visual separator between the drawing insertion area and the second portion on a touch-sensitive display. For example, in Figure 5CF In the device 100, a first visual separator 557N is displayed between a first portion of the drawing insertion area 557M and the typed text 55K, and a second visual separator 557O is displayed between a second portion of the drawing insertion area 557O and the typed text 557L. In some embodiments, in response to a detected touch at the location of an indication for inserting the drawing insertion area, the device displays (838) a toolbar area on a touch-sensitive display, the toolbar area including multiple drawing tool indications for selecting the corresponding drawing tool. For example, in Figure 5CFIn the device 100, the display toolbar area 555C includes multiple drawing tool displays 532A, 532B, 532E, 532H and 532I.

[0319] In some implementations, the device detects (840) movement of the contact within the drawing insertion area. For example, in Figure 5CF In this embodiment, device 100 detects movement of contact 552L within drawing insertion area 557M. In some implementations, in response to detecting movement of the contact within the drawing insertion area, the device draws a mark (842) in the drawing insertion area based on that movement. For example, in... Figure 5CG In the middle, the equipment is based on Figure 5CF The movement of contact 552L in the drawing insertion area 557M displays the stroke drawing object 559C.

[0320] In some implementations, the device detects (844) movement of the contact from the location of the second visual separator. For example, in Figure 5CG In this embodiment, device 100 detects movement of contact 552Z from the position of the second visual separator 557O. In some implementations, in response to detecting movement of the contact from the position of the second visual separator, the device moves (846) the second visual separator to change the size of the drawing insertion area. For example, in... Figure 5CH In this process, device 100 moves the second visual separator 557O to change the size of the drawing insertion area 557M. Therefore, the visual separator provides the user with an efficient mechanism to adjust the size of the drawing insertion area, thereby reducing the amount of user interaction required to resize the drawing insertion area. Reduced user interaction with the device reduces wear and tear, and for battery-powered devices, extends battery life.

[0321] It should be understood that, Figures 8A-8C The specific order in which the operations described herein are presented is merely exemplary and not intended to indicate that the order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that the details of other processes described herein in conjunction with other methods (e.g., methods 600, 700, and 900) are similarly applicable to the above-described combinations. Figures 8A-8CThe method 800. For example, the user interface described above with reference to method 800 optionally has one or more features, including user input (including finger contact and stylus contact), markers, content creation areas, drawing insertion areas, and drawing tool selection indications. The user interface described herein with reference to other methods (e.g., methods 600, 700, and 900) includes user input (including finger contact and stylus contact), markers, content creation areas, drawing insertion areas, and drawing tool selection indications. For the sake of brevity, these details are not repeated here.

[0322] Figures 9A-9C A flowchart of a method 900 for inserting a drawing insertion area according to some embodiments is shown. Method 900 is used in electronic devices having a display and a touch-sensitive surface (e.g., Figure 1A Portable multi-functional device 100, or Figure 3 The operation is performed at device 300. In some embodiments, the display is a touch-sensitive display, and the touch-sensitive surface is on or integrated with the display. In some embodiments, the display and the touch-sensitive surface are separate. Some operations in method 900 are optionally combined, and / or the order of some operations is optionally changed.

[0323] As described below, method 900 provides an intuitive way to insert a drawing insertion area. This method reduces the cognitive burden on the user when inserting a drawing insertion area, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to insert drawing insertion areas faster and more efficiently saves power and increases the time between battery charging sessions.

[0324] The device displays a content creation area (902) on a touch-sensitive display. For example, in Figure CL, device 100 displays a content creation area 555B as part of a notepad user interface 505.

[0325] The device detected (904) one or more typed inputs. For example, in Figure 5CL In this context, device 100 detects one or more key inputs in the form of a contact at the location of the input indicator 558I. As another example, in... Figure 5CM and Figure 5CN Between, device 100 detects one or more key inputs in the form of contact at the location of character display 558F.

[0326] In response to detecting one or more keystrokes, the device edits (906) the content in the content creation area to create a blank area that does not contain visible content and is a text insertion area. For example, in Figure 5CN In the content creation area 555B, the content includes blank text insertion area 567A.

[0327] In some implementations, the device detects (908) contact at the location of the text insertion area. For example, in Figure 5CR In the middle, device 100 detects finger contact 553D at the location of blank text insertion area 567A. As another example, in Figure 5DE In the process, device 100 detects stylus contact 563B at the location of blank text insertion area 567A.

[0328] In some implementations, in response to detecting a touch at the location of the text insertion area, the device displays (910) a keyboard on a touch-sensitive display, the keyboard including multiple character indications for inserting text into the text insertion area and including indications for displaying drawing tools. For example, in Figure 5CS In this configuration, device 100 displays the toolbar area 555C in keyboard mode, which includes a character display 558G and a drawing tool display 558D. As another example, in... Figure 5DF In the middle, device 100 displays toolbar area 555C in keyboard mode, which includes character display 558G and display drawing tool display 558D.

[0329] In some implementations, the device detects (912) user input requesting insertion into the drawing insertion area. In some implementations, the device detects (914) contact at the location where the drawing tool indicator is displayed. For example, in Figure 5CS In this example, device 100 detects finger contact 553E at the location where the drawing tool display indicator 558D is displayed. As another example, in... Figure 5DF In this process, device 100 detects stylus contact 563C at the location where the drawing tool display indicator 558D is displayed. The function of displaying the drawing tool provides the user with an efficient mechanism to enter online drawing mode, thereby reducing the amount of user interaction required for marking content. Reduced user interaction reduces wear and tear on the device. Reduced user interaction also leads to faster initiation of marking operations, and thus reduces the power consumption of performing marking operations, thereby extending the device's battery life.

[0330] In some implementations, in response to a detected touch at a location where drawing tool indications are displayed, the device displays (916) a toolbar area that includes multiple drawing tool indications for selecting the appropriate drawing tool. For example, in Figure 5 CT In this context, device 100 displays the toolbar area 555C in an online drawing mode that includes multiple drawing tools, represented by icons 532A, 532B, 532E, 532H, and 532I. As another example, in... Figure 5DGIn the process, device 100 displays toolbar area 555C in an online drawing mode that includes multiple drawing tools, displaying indicators 532A, 532B, 532E, 532H and 532I.

[0331] When creating a region to display content using a text insertion area, the device detects (918) touches on a touch-sensitive display within the text insertion area. For example, in Figure 5CU In the process, device 100 detects finger contact 553F in the blank text insertion area 567A. As another example, in... Figure 5DG In the process, device 100 detects stylus contact 563D in blank text insertion area 567A.

[0332] In response to detecting contact in the text insertion area and determining that the contact is a stylus contact, the device converts (920) at least a portion of the text insertion area into a drawing insertion area including markers based on the stylus contact. For example, in Figure 5DH In the middle, device 100 is based on Figure 5DG The stylus contact 563D transforms a portion of the blank text insertion area 567A into a drawing insertion area 567J including markers 569C. In some embodiments, the device transforms (922) at least a portion of the text insertion area into a drawing insertion area including markers based on the movement of the stylus contact. For example, in Figure 5 DJ In the middle, device 100 is based on Figure 5DI The movement of the stylus contact 563E transforms a portion of the blank text insertion area 567A into a drawing insertion area 567K that includes a stroke drawing object 569A. Accelerating the display of markers in response to detecting stylus contacts more likely to be used for marking content reduces the amount of user interaction required for marking. Reduced user interaction reduces device wear. Reduced user interaction also leads to faster initiation of marking operations, and thus reduces power consumption during marking operations, thereby extending device battery life.

[0333] In response to detecting contact in the text insertion area and determining that the contact is a finger contact, the device converts (924) at least a portion of the text insertion area into a blank drawing insertion area. For example, in Figure 5CV In this embodiment, device 100 converts a portion of the blank text insertion area 567A into a blank drawing insertion area 567D. In some implementations, the device converts (926) at least a portion of the text insertion area into a blank drawing insertion area, thereby ignoring movement caused by finger contact. For example, in... Figure 5CY In this process, device 100 converts a portion of the blank text insertion area 567A into a blank drawing insertion area 567D, thereby ignoring... Figure 5CXThe finger contact portion 553H moves. In response to detected finger contact, a blank drawing area is provided, allowing the user to provide a more accurate indication of the desired mark's location, thus preventing unnecessary user interaction to replace the desired mark in case of misplacement. Preventing unnecessary user interaction with the device reduces wear and tear, and for battery-powered devices, extends battery life.

[0334] In some implementations, the device detects (928) movement of a touch on a touch-sensitive display within the content creation area. For example, in Figure 5CJ In this example, device 100 detects the movement of a finger contact 552N within the content creation area 555B. As another example, in... Figure 5CP In this example, device 100 detects the movement of a finger touching 553C within the content creation area 553C. As another example, in... Figure 5CQ In the process, device 100 detects the movement of stylus contact 563A within content creation area 553C.

[0335] In some implementations, in response to detecting movement of a touch on a touch-sensitive display in the content creation area and based on determining the movement of the touch in the typed text, the device selects (930) the text covered by the movement. For example, in Figure 5CK In this context, the typed text covered by the movement of a finger touching 552N is covered by a selection marker 557Q (e.g., the text is highlighted), and the selection marker 557Q indicates that the typed text covered by the movement of a finger touching 552N has been selected. In some embodiments, in response to detecting movement of a touch on a touch-sensitive display in the content creation area and determining that the movement of the touch is in a blank area that serves as a text insertion area, the device scrolls (932) the content creation area. For example, in Figure 5CQ In response to Figure 5CP The movement of a finger touching 553C in the blank text insertion area 567A of the device 100 scrolls the content to create area 555B. As another example, in Figure 5CR In response to Figure 5CQ The movement of the stylus contact 563A in the blank text insertion area 567A of the device 100 scrolls the content creation area 555B. Based on whether the gesture is performed within the typed text or the blank text insertion area, responding to the same gesture (movement of contact) to perform two different operations provides an efficient mechanism to perform either operation, thereby reducing the amount of user interaction with the device to perform at least one operation. Reducing user interaction with the device reduces device wear and tear, and for battery-powered devices, extends battery life.

[0336] In some implementations, the device displays (934) a first visual separator on a touch-sensitive display between the typed text in the drawing insertion area and the content creation area. For example, in Figure 5 DJ In this configuration, device 100 displays a first visual separator 567F between the drawing insertion area 567K and the typed text 557A. Displaying this visual separator provides the user with visual feedback indicating the location of the drawing insertion area, thereby preventing accidental manipulation of text or other content outside the drawing insertion area (leading to further user interaction to undo the manipulation). Preventing unnecessary user interaction reduces wear and tear on the device and, for battery-powered devices, extends battery life.

[0337] In some implementations, based on determining that the drawing insertion area lies between first typed text in the content creation area and second typed text in the content creation area, the device displays (936) a first visual separator between the drawing insertion area and the first typed text and a second visual separator between the drawing insertion area and the second typed text. For example, in Figure 5 DJ In this embodiment, device 100 displays a first visual separator 567F between the drawing insertion area 567K and the typed text 557A, and a second visual separator 567G between the drawing insertion area 567K and the additional typed text 567B. In some embodiments, based on determining that the drawing insertion area is below all typed text in the content creation area, the device displays (938) the first visual separator between the drawing insertion area and the typed text and omits the display of the second visual separator. For example, in Figure 5DR In this device 100, a first visual separator 567J is displayed between all typed text, typed text 557A, and additional typed text 567B, in the drawing insertion area 567I and the content creation area 555B. Displaying only a single visual separator makes more efficient use of screen space, resulting in a more efficient human-machine interface. For battery-powered electronic devices, a more efficient human-machine interface saves power and increases the time between battery charging. Additionally, a more efficient human-machine interface reduces the amount of user interaction with the device and reduces wear and tear. A smaller (and less expensive) screen can provide the same usability by using less space on the screen.

[0338] In some implementations, the device detects movement of the contact (940) from the position of the first visual separator. For example, in Figure 5DM In the process, device 100 detects the movement of stylus contact 563G from the position of the second visual separator 567G.

[0339] In some implementations, in response to detecting movement of contact from the position of the first visual separator, the device converts (942) at least another portion of the text insertion area into another portion of the drawing insertion area. For example, in Figure 5DN In this process, device 100 converts the third blank text insertion area 567L into a part of the drawing insertion area 567K. Therefore, the visual separator provides the user with an efficient mechanism to adjust the size of the drawing insertion area, thereby reducing the amount of user interaction required to resize the drawing insertion area. Reduced user interaction with the device reduces wear and tear, and for battery-powered devices, extends battery life.

[0340] In some implementations, the device detects (944) movement of the contact from a first position within the drawing insertion area to a second position within an additional portion of the text insertion area. For example, in Figure 5DK In the process, device 100 detects the movement of stylus contact 563F from a first position within the drawing insertion area 567K to a second position within the second blank text insertion area 567E.

[0341] In some implementations, in response to detecting movement of contact from a first position within the drawing insertion area to a second position within another portion of the text insertion area, the device at least additionally converts (946) the text insertion area into an additional portion of the drawing insertion area. For example, in Figure 5DL In this process, device 100 converts a portion of the second blank text insertion area 567E into an additional portion of the drawing insertion area 567K. Therefore, in some cases, the size of the drawing insertion area will be automatically adjusted to accommodate the required markings, thereby reducing user interaction with the device to manually adjust the size of the drawing insertion area (e.g., using visual separators). Reducing user interaction with the device reduces wear and tear on the device and, for battery-powered devices, extends battery life.

[0342] In some implementations, in response to detecting movement of the contact from a first position within the drawing insertion area to a second position within an additional portion of the text insertion area, the device draws (948) a mark in the additional portion of the drawing insertion area based on the movement of the contact from the first position to the second position. For example, in Figure 5DL In the drawing insertion area 567K, device 100 displays the second stroke drawing object 569D.

[0343] It should be understood that, Figures 9A to 9CThe specific order of operations described herein is merely exemplary and not intended to indicate that the order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that the details of other processes described herein in conjunction with other methods (e.g., methods 600, 700, and 800) are similarly applicable to the above-described combinations. Figures 9A-9C The method 900. For example, the user interface described above with reference to method 900 optionally includes one or more features such as user input (including finger and stylus contact), markers, content creation areas, drawing insertion areas, and drawing tool selection indications. The user interface described herein with reference to other methods (e.g., methods 600, 700, and 800) includes user input (including finger and stylus contact), markers, content creation areas, drawing insertion areas, and drawing tool selection indications. For the sake of brevity, these details are not repeated here.

[0344] For purposes of explanation, the foregoing description has been given by reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible based on the teachings above. The embodiments were chosen and described to best elucidate the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention with various modifications suitable for the contemplated particular purpose, as well as the various described embodiments.

Claims

1. A method comprising: In a device having one or more processors, non-transitory memory, a display, and one or more input devices: A drawing user interface comprising multiple drawing objects is displayed on the display, the multiple drawing objects including a first drawing object and a second drawing object; When a drawing tool associated with a predefined operation is selected, user input is detected via the one or more input devices, the user input moving to define a path within the drawing user interface, wherein the path intersects a portion of a first drawing object smaller than the whole of a first drawing object and a portion of a second drawing object smaller than the whole of a second drawing object; as well as In response to the user input that detects movement to define the path within the drawing user interface, the predefined operation is performed on the entirety of the first drawing object and the entirety of the second drawing object, wherein performing the predefined operation includes stopping the display of the entirety of the first drawing object and the entirety of the second drawing object but not stopping the display of other drawing objects among the plurality of drawing objects.

2. The method of claim 1, wherein the drawing user interface includes a toolbar area, the toolbar area including a plurality of drawing tool selection indicators respectively associated with a plurality of drawing tools.

3. The method of claim 2, further comprising, prior to detecting user input that moves to define the path within the drawing user interface, detecting a selection user input at a location of one of the plurality of drawing tool selection enablements associated with one of the plurality of drawing tools associated with the predefined operation.

4. The method of claim 1, wherein the drawing user interface includes content marked by the plurality of drawing objects.

5. The method of claim 1, wherein the plurality of drawing objects includes stroke drawing objects defined by continuous user input within the drawing user interface when a drawing tool associated with a stroke operation is selected.

6. The method according to claim 1, wherein the plurality of drawing objects includes at least one of a shape drawing object, a stroke drawing object, a magnifying glass drawing object, or a text drawing object.

7. The method of claim 1, wherein performing the predefined operation further includes stopping the display of another drawing object among the plurality of drawing objects that intersect with one or more of the first drawing object and the second drawing object.

8. The method of claim 1, wherein performing the predefined operation comprises: Based on the determination that the speed of the user input moving to define the path within the drawing user interface exceeds a speed threshold, the display of the entire first drawing object and the entire second drawing object is stopped; and Based on the determination that the speed does not exceed the speed threshold, the display of the entire first drawing object and the entire second drawing object continues.

9. The method of claim 1, wherein the plurality of drawing objects includes a third drawing object, and wherein performing the predefined operation includes maintaining the display of the third drawing object.

10. The method of claim 1, wherein the device includes a touch-sensitive surface, wherein the user input includes a touch on the touch-sensitive surface, wherein movement of the touch on the touch-sensitive surface defines the path within the drawing user interface, and wherein stopping the display of the entire first drawing object and the entire second drawing object occurs while the touch on the touch-sensitive surface continues to be detected.

11. An electronic device, comprising: monitor; One or more input devices; and One or more processors, said one or more processors being configured to: A drawing user interface comprising multiple drawing objects is displayed on the display, the multiple drawing objects including a first drawing object and a second drawing object; When a drawing tool associated with a predefined operation is selected, user input is detected via the one or more input devices, the user input moving to define a path within the drawing user interface, wherein the path intersects a portion of a first drawing object smaller than the whole of a first drawing object and a portion of a second drawing object smaller than the whole of a second drawing object; as well as In response to the user input that detects movement to define the path within the drawing user interface, the predefined operation is performed on the entirety of the first drawing object and the entirety of the second drawing object, wherein performing the predefined operation includes stopping the display of the entirety of the first drawing object and the entirety of the second drawing object but not stopping the display of other drawing objects among the plurality of drawing objects.

12. The electronic device of claim 11, wherein the drawing user interface includes content marked by the plurality of drawing objects.

13. The electronic device of claim 11, wherein the plurality of drawing objects includes stroke drawing objects defined by continuous user input within the drawing user interface when a drawing tool associated with a stroke operation is selected.

14. The electronic device of claim 11, wherein the plurality of drawing objects includes at least one of a shape drawing object, a stroke drawing object, a magnifying glass drawing object, or a text drawing object.

15. The electronic device of claim 11, wherein performing the predefined operation further includes stopping the display of another drawing object among the plurality of drawing objects that intersect with one or more of the first drawing object and the second drawing object.

16. A non-transitory computer-readable storage medium storing one or more programs, said one or more programs comprising instructions that, when executed by an electronic device having a display and one or more input devices, cause the electronic device to: A drawing user interface comprising multiple drawing objects is displayed on the display, the multiple drawing objects including a first drawing object and a second drawing object; When a drawing tool associated with a predefined operation is selected, user input is detected via the one or more input devices, the user input moving to define a path within the drawing user interface, wherein the path intersects a portion of a first drawing object smaller than the whole of a first drawing object and a portion of a second drawing object smaller than the whole of a second drawing object; and In response to the user input that detects movement to define the path within the drawing user interface, the predefined operation is performed on the entirety of the first drawing object and the entirety of the second drawing object, wherein performing the predefined operation includes stopping the display of the entirety of the first drawing object and the entirety of the second drawing object but not stopping the display of other drawing objects among the plurality of drawing objects.

Images