Devices, methods, and systems for manipulating a user interface

Abstract

Various implementations include systems, methods, and devices for manipulating user interfaces. For example, in some implementations, a method includes moving a drawing palette based on a corresponding request. The movable drawing palette enables a larger portion of a display to be used for content modification operations. As another example, in some implementations, a method includes displaying a screenshot editing interface for editing a captured screenshot based on an input type of a detected input. By displaying the screenshot editing interface in response to performing a screenshot capture, the method provides an enhanced user experience with fewer system resources. As yet another example, in some implementations, a method includes selectively erasing portions of an object using a multi-mode eraser tool. A pixel erasure input deletes a portion of a displayed object, while an object erasure input deletes the entire displayed object.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 2020800341211, filed on April 10, 2020, entitled "Apparatus, Method and System for Manipulating User Interface". Technical Field

[0002] This disclosure relates in general to electronic devices having user interfaces, and more particularly to electronic devices having one or more input devices for detecting inputs used to manipulate those user interfaces. Background Technology

[0003] Using input to manipulate the user interface of electronic devices has become a common practice. For example, in various embodiments, electronic devices use peripheral input devices (e.g., touchscreen input, mouse, keyboard) to influence the display of one or more displayed user interfaces.

[0004] However, most of these input devices offer limited and inefficient control over manipulating the user interface. Therefore, manipulating the user interface to enable an electronic device to perform specific operations may require repetitive, complex, and / or cumbersome inputs or input types. Summary of the Invention

[0005] Therefore, a robust and efficient mechanism is needed for manipulating the user interface of a display in an electronic device. Specifically, electronic devices require faster and more efficient methods and interfaces for manipulating the user interface. Such methods and interfaces optionally supplement or replace conventional methods for manipulating the user interface. These methods and interfaces reduce the amount, extent, and / or nature of input from the user, resulting in a more efficient human-machine interface. Therefore, for battery-powered devices, such methods and interfaces save power and increase the time between battery charges.

[0006] The disclosed devices, systems, and methods can reduce or eliminate the aforementioned defects and other problems associated with the user interface of electronic devices having touch-sensitive surfaces. In some embodiments, the electronic device is a desktop computer. In some embodiments, the electronic device is portable (e.g., a laptop, tablet, or handheld device). In some embodiments, the electronic device is a personal electronic device, such as a mobile phone or wearable device (e.g., a smartwatch). In some embodiments, the electronic device has a touchpad. In some embodiments, the electronic device has a touch-sensitive display (also referred to as a “touchscreen” or “touchscreen display”). In some embodiments, the electronic device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, and a program or set of instructions stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, when the stylus is not in physical contact with the touch-sensitive surface, the user interacts with the GUI primarily through user interaction with the stylus. In some embodiments, when the user is holding the stylus, the user interacts with the GUI primarily through finger and / or hand contact and gestures on the stylus. In some embodiments, these functions optionally include image editing, drawing, presentation, word processing, spreadsheet creation, playing games, making and receiving phone calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, note-taking, and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0007] According to some embodiments, a method is performed in an electronic device having one or more processors, non-transitory memory, input devices, and display devices. The method includes displaying a first drawing palette at a first location within a first application interface via the display device. The first drawing palette at the first location has a first appearance, in which a representation of a currently selected drawing tool is simultaneously displayed with representations of one or more other available drawing tools. The method further includes detecting a first input via the input device corresponding to a request to move the first drawing palette within the first application interface. The method further includes: in response to detecting the first input: displaying the first drawing palette having the first appearance at a second location based on determining that the first input corresponds to a request to move the first drawing palette to a second location within the first application interface; and displaying the first drawing palette having a second appearance different from the first appearance at a third location based on determining that the first input corresponds to a request to move the first drawing palette to a third location within the first application interface different from the second location, wherein when the first drawing palette has the second appearance, a representation of the currently selected drawing tool is displayed in the first drawing palette, but representations of other drawing tools are not displayed in the first drawing palette.

[0008] According to some embodiments, a method is performed in an electronic device having one or more processors, non-transitory memory, input devices, and display devices. The method includes detecting a screenshot capture input while content is displayed via the display device. The method further includes: in response to detecting the screenshot capture input: capturing a screenshot image of the content displayed via the display device; based on determining that the screenshot capture input is a first input type, displaying a screenshot editing interface via the display device for editing the screenshot image, wherein the screenshot editing interface includes the screenshot image; and based on determining that the screenshot capture input corresponds to a second input type different from the first input type, displaying a thumbnail representation of the screenshot image overlaid on the content captured within the screenshot image via the display device.

[0009] According to some embodiments, a method is performed in an electronic device having one or more processors, non-transitory memory, input devices, and display devices. The method includes displaying a drawing user interface via the display device. The method further includes detecting an object insertion input corresponding to a request to insert an object into the drawing user interface when the drawing user interface is displayed. The method further includes inserting a corresponding object into the drawing user interface in response to detecting the object insertion input. The method further includes detecting a pixel erase input when the corresponding object is displayed in the drawing user interface. The method further includes stopping the display of a first portion of the corresponding object without stopping the display of a second portion of the corresponding object, and without stopping the display of a third portion of the corresponding object, in response to detecting the pixel erase input. The method further includes detecting an object erase input pointing to a portion of the corresponding object. The method further includes: in response to detecting the object erase input: stopping the display of the second part of the corresponding object without stopping the display of the third part of the corresponding object based on determining that the object erase input points to the second part of the corresponding object and that the second part of the corresponding object is not connected to the third part of the corresponding object; and stopping the display of the third part of the corresponding object without stopping the display of the second part of the corresponding object based on determining that the object erase input points to the third part of the corresponding object and that the third part of the corresponding object is not connected to the second part of the corresponding object.

[0010] According to some embodiments, an electronic device includes one or more processors, non-transitory memory, an input device, a display device, and one or more programs. The one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors. The one or more programs include instructions for performing or causing to perform any of the methods described herein. According to some embodiments, instructions are stored in a non-transitory computer-readable storage medium that, when executed by an electronic device having one or more processors, an input device, and a display device, cause the electronic device to perform or cause to perform any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device having non-transitory memory, an input device, a display device, and one or more processors for performing one or more programs stored in the non-transitory memory includes one or more elements displayed as described in any of the methods described herein, the one or more elements being updated in response to input, as described in any of the methods described herein. According to some embodiments, an electronic device includes: one or more processors, non-transitory memory, an input device, a display device, and means for performing or causing to perform any of the methods described herein. According to some embodiments, an information processing apparatus used in an electronic device having one or more processors, non-transitory memory, input devices, and display devices includes means for performing or causing to perform any of the methods described herein.

[0011] Therefore, electronic devices with input and display devices utilize various inputs detected via these input devices, such as touch input, mouse input, keyboard input, etc. Based on these inputs, the electronic device performs various operations, such as drawing palette manipulation operations (e.g., moving and resizing / reorienting the drawing palette), screenshot capture operations, and editing operations. In some embodiments, fewer inputs enable the electronic device to perform specific operations compared to previously available systems, thereby improving the functionality of the electronic device. Examples of improved functionality include longer battery life and less wear and tear, as well as more efficient and accurate user interaction with the electronic device. Attached Figure Description

[0012] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0013] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.

[0014] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some embodiments.

[0015] Figure 2 A portable multi-functional device with a touchscreen is shown according to some embodiments.

[0016] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some embodiments.

[0017] Figure 4 This is a block diagram of an exemplary electronic stylus according to some embodiments.

[0018] Figures 5A to 5B The positional state of the stylus relative to the touch-sensitive surface is shown according to some embodiments.

[0019] Figure 6A An exemplary user interface for an application menu on a portable multifunction device according to some embodiments is shown.

[0020] Figure 6B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0021] Figures 7A to 7CF This is an example of a user interface for repositioning a drawing palette, according to some embodiments.

[0022] Figures 8A to 8AL These are examples of user interfaces for invoking and utilizing screenshot editing interfaces, according to some embodiments.

[0023] Figures 9A to 9P This is an example of capturing a screenshot image based on detected stylus input, according to some embodiments.

[0024] Figures 10A to 10D This is a flowchart of a method for repositioning a drawing palette according to some embodiments.

[0025] Figures 11A to 11C This is a flowchart of a method for invoking and utilizing a screenshot editing interface according to some embodiments.

[0026] Figures 12A to 12AP This is an example of a user interface for selectively erasing portions of an object, according to some embodiments.

[0027] Figures 13A to 13D This is a flowchart of a method for selectively erasing portions of an object, according to some embodiments. Detailed Implementation

[0028] Many electronic devices manipulate the user interface based on detected input. However, existing methods for manipulating user interfaces can be slow, cumbersome, and inefficient.

[0029] For example, in various embodiments, the electronic device may display a drawing palette that enables the modification of the currently selected drawing tool and / or its attributes (e.g., color). However, the drawing palette may be fixed to a specific location (e.g., fixed to a specific side of the display). Therefore, drawing operations (e.g., drawing lines, entering text, pasting shapes) cannot be applied to the portion of the user interface where the drawing palette is located, thus limiting the usable display area of ​​the user interface. Furthermore, the electronic device does not change the appearance of the drawing palette (e.g., the size, orientation, or number of types of drawing tools displayed in the drawing palette) in response to the detection of input requesting movement of the drawing palette. However, the various embodiments disclosed herein specify that, in response to the detection of a request to move the drawing palette to a specific location within the user interface, the electronic device displays the drawing palette at that specific location and, in various cases, with a different appearance. By changing the position and orientation of the drawing palette, the electronic device provides a larger usable portion of the display for drawing operations and other content modification operations.

[0030] As another example, in various embodiments, electronic devices may provide screenshot capture functionality that offers a non-intuitive, unuser-friendly experience. For instance, the screenshot capture functionality provides limited mechanisms for manipulating the captured screenshot image. Furthermore, the screenshot image is often saved to a background clipboard unknown to unfamiliar users. However, the various embodiments disclosed herein specify that, based on the type of input detected, the electronic device displays a screenshot editing interface or a thumbnail representation of the screenshot image. Additionally, in some embodiments, the screenshot editing interface includes a rich set of manipulation options for applying to the screenshot image (e.g., annotation, changing opacity levels, or displaying additional related content). Therefore, after a screenshot capture is performed, the electronic device displays a screenshot editing interface, providing a seamless and intuitive user experience that requires less time and less user input to manipulate the screenshot image. This also reduces power consumption and extends the battery life of the electronic device by enabling users to use it more quickly and efficiently.

[0031] As yet another example, in various embodiments, the electronic device provides an erasing tool for erasing portions of content. However, this erasing tool is limited in its functionality. For example, the erasing tool cannot perform different erasing operations on a portion based on whether that portion of an object was previously separated from other portions of the object (e.g., split or separated). On the other hand, the various embodiments disclosed herein specify that after an object is divided into multiple separate portions based on a pixel-based erasing input, the electronic device stops displaying a specific separate portion and retains the other remaining portions in response to an object erasing input. Therefore, the electronic device provides more functionality and control relative to the erasing operation. Furthermore, the electronic device does not need to receive a drag-and-drop erasing input that extends spatially with the separate portions of the object to erase that separate portion. By utilizing an object erasing input instead of a drag-and-drop erasing input to erase separate portions, the electronic device reduces processing and battery usage and experiences less wear and tear.

[0032] under, Figures 1A to 1B , Figures 2 to 4 , Figures 5A to 5B as well as Figures 6A to 6B A description of an exemplary device is provided. Figures 7A to 7CF This is an example of a user interface for repositioning a drawing palette, according to some embodiments. Figures 7A to 7CF The user interface in the document is used to show Figures 10A to 10D The process in. Figures 8A to 8AL These are examples of user interfaces for invoking and utilizing screenshot editing interfaces, according to some embodiments. Figures 9A to 9P This is an example of capturing a screenshot image based on detected stylus input, according to some embodiments. Figures 8A to 8AL and Figures 9A to 9P The user interface in the document is used to show Figures 11A to 11C The process in. Figures 12A to 12AP This is an example of a user interface for selectively erasing portions of an object, according to some embodiments. Figures 12A to 12AP The user interface in the document is used to show Figures 13A to 13D The process in.

[0033] Exemplary device Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to those skilled in the art that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure various aspects of the embodiments.

[0034] It will also be understood that, although in some cases the terms “first,” “second,” etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact, without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0035] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “includes”, “including”, “comprises”, and / or “comprising” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0036] As used herein, depending on the context, the term “if” is optionally interpreted as meaning “when…” followed by “at…” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrase “if it is determined…” or “if [the stated condition or event] is detected” is optionally interpreted as meaning “when it is determined…” or “in response to determination…” or “when [the stated condition or event] is detected” or “in response to the detection of [the stated condition or event].”

[0037] This document describes embodiments of electronic devices, user interfaces of such devices, and related processes for using such devices. In some embodiments, the electronic device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functionality. Exemplary embodiments of portable multi-functional devices include, but are not limited to, iPhone®, iPod Touch®, and iPad® devices from Apple Inc. (Cupertino, California). Optionally, other portable electronic devices, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads), may be used. It should also be understood that in some embodiments, the electronic device is not a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0038] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0039] The electronic device typically supports a variety of applications, such as one or more of the following: note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0040] Various applications running on electronic devices optionally utilize at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the electronic device (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0041] Now let’s turn our attention to examples of portable devices with touch-sensitive displays. Figure 1AThis is a block diagram illustrating a portable multi-functional device 100 having a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touchscreen" for convenience, and is sometimes simply referred to as a touchscreen display. The electronic device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. The electronic device 100 optionally includes one or more optical sensors 164. The electronic device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the electronic device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the electronic device 100). Electronic device 100 optionally includes one or more tactile output generators 163 for generating tactile output on electronic device 100 (e.g., generating tactile output on a touch-sensitive surface such as a touch-sensitive display system 112 of electronic device 100 or a touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0042] As used in this specification and claims, the term "haptic output" refers to the physical displacement of an electronic device relative to a previous position of the electronic device, the physical displacement of a component of the electronic device (e.g., a touch-sensitive surface) relative to another component of the electronic device (e.g., a housing), or the displacement of a component relative to the center of mass of the electronic device, which is detected by a user using their tactile sense. For example, when an electronic device or a component of the electronic device comes into contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the electronic device or a component of the electronic device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with the touch-sensitive surface, which has been physically pressed (e.g., displaced) by the user's movement, does not move. For example, even when the smoothness of the touch-sensitive surface remains unchanged, the movement of the touch-sensitive surface can optionally be interpreted or sensed by the user as the "roughness" of the touch-sensitive surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when tactile output is described as corresponding to a specific sensory perception of the user (e.g., "release click", "press click", "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the electronic device or its components, which will generate the sensory perception of a typical (or ordinary) user.

[0043] It should be understood that electronic device 100 is merely an example of a portable multifunctional device, and electronic device 100 may optionally have more or fewer components than shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.

[0044] Memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of electronic device 100, such as one or more CPUs 120 and peripheral interface 118, is optionally controlled by memory controller 122.

[0045] Peripheral interface 118 can be used to couple input and output peripheral devices of an electronic device to one or more CPUs 120 and memory 102. The one or more CPUs run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of electronic device 100 and process data.

[0046] In some embodiments, the peripheral interface 118, one or more CPUs 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0047] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via these electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks and other devices, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). This wireless communication may optionally use any of a variety of communication standards, protocols, and / or technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolved Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE...). 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including those not yet developed as of the date of this document submission.

[0048] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and electronic device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 The headset jack 212 in the middle provides an interface between the audio circuitry 110 and removable audio input / output peripherals, such as headphones that are output-only or headsets that have both outputs (e.g., mono or binaural headphones) and inputs (e.g., microphones).

[0049] I / O subsystem 106 couples input / output peripherals on electronic device 100, such as touch-sensitive display system 112 and other input or control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from / send electrical signals to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled (or not coupled) to any of the following: keyboard, infrared port, USB port, stylus and / or pointing device such as mouse. One or more buttons (e.g., Figure 2 Button 208 may optionally include volume up / down buttons for volume control of speaker 111 and / or microphone 113. The one or more buttons may optionally include push-down buttons (e.g., Figure 2 (Press button 206 in the middle).

[0050] The touch-sensitive display system 112 provides an input and output interface between the electronic device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. Visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output corresponds to a user interface object. As used herein, the term "enabled representation" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to input directed to it). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0051] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or tactile contact. The touch-sensitive display system 112 and the display controller 156 (along with any associated modules and / or instruction sets in the memory 102) detect touches on the touch-sensitive display system 112 (and any movement or interruption of those touches) and translate the detected touches into interactions with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In an exemplary embodiment, the contact point between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0052] The touch-sensitive display system 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112, to detect contact and any movement or interruption therein. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In one exemplary embodiment, projected mutual capacitance sensing technology, such as that found in Apple Inc.'s (Cupertino, California) iPhone®, iPod Touch®, and iPad®, is used.

[0053] The touch-sensitive display system 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi, or greater). Users optionally interact with the touch-sensitive display system 112 using any suitable object or accessory such as a stylus, finger, etc. In some embodiments, the user interface is designed to work with finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the electronic device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.

[0054] In some embodiments, in addition to a touchscreen, the electronic device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the electronic device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0055] Electronic device 100 also includes a power system 162 for supplying power to various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0056] The electronic device 100 optionally also includes one or more optical sensors 164. Figure 1A An optical sensor coupled to an optical sensor controller 158 in I / O subsystem 106 is shown. One or more optical sensors 164 optionally include charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) phototransistors. The one or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), the one or more optical sensors 164 optionally capture still images and / or video. In some embodiments, the optical sensor is located on the rear of the electronic device 100, opposite to a touch-sensitive display system 112 on the front of the electronic device 100, enabling the touchscreen to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the electronic device 100 to acquire images of the user (e.g., for selfies, for video conferencing when the user is watching other video conference participants on the touchscreen, etc.).

[0057] The electronic device 100 optionally also includes one or more contact strength sensors 165. Figure 1A A contact strength sensor coupled to a strength sensor controller 159 in I / O subsystem 106 is shown. One or more contact strength sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact strength sensors 165 receive contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of electronic device 100, opposite to the touchscreen display system 112 located on the front of the electronic device 100.

[0058] The electronic device 100 optionally also includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 is coupled to an input controller 160 in an I / O subsystem 106. In some embodiments, when the electronic device 100 is placed near a user's ear (e.g., when the user is making a phone call), the proximity sensor 166 turns off and disables the touch-sensitive display system 112.

[0059] The electronic device 100 optionally also includes one or more tactile output generators 163. Figure 1AA haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. One or more haptic output generators 163 optionally include one or more electroacoustic devices (such as speakers or other audio components), and / or electromechanical devices that convert energy into linear motion (such as motors, solenoids, electroactive polymerizers, piezoelectric actuators, electrostatic actuators), or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on electronic devices). One or more haptic output generators 163 receive haptic feedback generation instructions from haptic feedback module 133 and generate haptic outputs on electronic device 100 that can be felt by a user of electronic device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates haptic outputs by moving the haptic surface vertically (e.g., in / outward from the surface of electronic device 100) or laterally (e.g., backward and forward in the same plane as the surface of electronic device 100). In some embodiments, at least one tactile output generator sensor is located on the rear of the electronic device 100, opposite to the touch-sensitive display system 112 located on the front of the electronic device 100.

[0060] Electronic device 100 may optionally also include one or more accelerometers 167, gyroscopes 168 and / or magnetometers 169 (e.g., as part of an inertial measurement unit (IMU)) for acquiring information about the position (e.g., attitude) of the electronic device. Figure 1A Sensors 167, 168, and 169, coupled to peripheral interface 118, are shown. Alternatively, sensors 167, 168, and 169 may be coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from the one or more accelerometers. Electronic device 100 optionally includes a GPS (or GLONASS or other global navigation system) receiver (not shown) for acquiring information about the position and orientation (e.g., portrait or landscape) of electronic device 100.

[0061] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 stores device / global internal state 157, as shown in Figures 1A and 1B. Figure 3As shown. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, which indicates what applications, views or other information occupy various areas of the touch-sensitive display system 112; sensor state, which includes information acquired from various sensors of the electronic device and other input or control devices 116; and position and / or orientation information regarding the position and / or orientation of the electronic device.

[0062] Operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0063] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, similar to, and / or compatible with, the 30-pin connector used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. (Cupertino, California). In some embodiments, the external port is a Lightning connector that is the same as, similar to, and / or compatible with, the Lightning connector used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. (Cupertino, California).

[0064] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., via a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the intensity of the contact (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations can optionally be applied to single-point contact (e.g., single-finger contact or stylus contact) or multi-point simultaneous contact (e.g., "multi-touch" / multi-finger contact and / or stylus contact). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0065] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the icon location). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event. Similarly, stylus taps, swipes, drags, and other gestures are optionally detected by detecting specific contact patterns of the stylus.

[0066] In conjunction with accelerometer 167, gyroscope 168, and / or magnetometer 169, position module 131 optionally detects position information about the electronic device, such as the attitude of the electronic device in a particular reference frame (e.g., roll, pitch, and / or yaw). Position module 131 includes software components for performing various operations related to detecting the position of the electronic device and detecting changes in the position of the electronic device. In some embodiments, position module 131 uses information received from a stylus used with electronic device 100 to detect position information about the stylus, such as detecting the positional state of the stylus relative to electronic device 100 and detecting changes in the positional state of the stylus.

[0067] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, and non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0068] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphics to be displayed from applications, etc., and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data to output to the display controller 156.

[0069] The haptic feedback module 133 includes various software components for generating instructions that are used by one or more haptic output generators 163 to generate haptic output at one or more locations on the electronic device 100 in response to user interaction with the electronic device 100.

[0070] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications, such as contacts 137, email 140, IM 141, browser 147, and any other application that requires text input.

[0071] GPS module 135 determines the location of electronic device 100 and provides that information for use in various applications (e.g., to telephone 138 for location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services such as weather desktop apps, local yellow pages desktop apps, and map / navigation desktop apps).

[0072] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: • Contacts module 137 (sometimes called address book or contact list); • Telephone module 138; • Video conferencing module 139; • Email client module 140; • Instant Messaging (IM) module 141; • Fitness support module 142; • Camera module 143 for still images and / or video images; • Image management module 144; • Browser module 147; • Calendar module 148; • Desktop mini-program module 149, which optionally includes one or more of the following: weather desktop mini-program 149-1, stock market desktop mini-program 149-2, calculator desktop mini-program 149-3, alarm clock desktop mini-program 149-4, dictionary desktop mini-program 149-5, and other desktop mini-programs obtained by the user, and desktop mini-programs created by the user 149-6. • Desktop Mini Program Creator Module 150 for creating user-created desktop mini programs 149-6; • Search module 151; • A video and music player module 152, optionally composed of a video player module and a music player module; • Memo module 153; • Map module 154; • Online video module 155; and / or • Annotation application 195, which provides annotations to the user interface and optionally stores and / or accesses saved annotations 196 in memory 102.

[0073] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0074] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 includes executable instructions for managing (e.g., stored in the application internal state 192 of the contact module 137 in memory 102 or memory 370) an address book or contact list. This management includes: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via the telephone module 138, video conferencing module 139, email client module 140, or IM module 141; and so on.

[0075] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, telephone module 138 includes executable instructions for performing the following operations: inputting a character sequence corresponding to a telephone number, accessing one or more telephone numbers in the address book 137, modifying an input telephone number, dialing a corresponding telephone number, initiating a conversation, and disconnecting or hanging up the phone when the conversation is complete. As described above, wireless communication optionally employs any of a variety of communication standards, protocols, and technologies.

[0076] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating video conferences between the user and one or more other participants, based on user instructions.

[0077] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0078] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for performing the following operations: inputting a character sequence corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Services (APNs), or IMPS for internet-based instant messaging), receiving an instant message, and viewing a received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0079] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, fitness support module 142 includes executable instructions for performing the following operations: creating fitness (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (in sports equipment and smartwatches); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness; and displaying, storing, and transmitting fitness data.

[0080] In conjunction with the touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0081] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing) or otherwise manipulating, tagging, deleting, displaying (e.g., in a digital slideshow or photo album), and storing still images and / or video images.

[0082] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0083] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0084] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet module 149 is optionally a micro-application downloaded and used by a user (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) or a user-created micro-application (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop applet includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop applet).

[0085] Combining RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet creator module 150 includes executable instructions for creating desktop applets (e.g., transferring user-specified portions of web pages to desktop applets).

[0086] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0087] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, the video and music player module 152 includes executable instructions allowing users to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on the touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, the electronic device 100 optionally includes the functionality of an MP3 player such as the iPod (a trademark of Apple Inc. of Cupertino, California).

[0088] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the memo module 153 includes executable instructions for creating and managing memos, to-do items, etc., according to user instructions.

[0089] Combining RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and map-related data (e.g., driving routes; data on shops and other points of interest at or near specific locations; and other location-based data) according to user instructions.

[0090] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes executable instructions that allow users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen 112 or on an external display connected wirelessly or via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, the instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos.

[0091] Each module and application identified above corresponds to a set of executable instructions for performing one or more of the functions described above and the methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures described above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0092] In some embodiments, electronic device 100 is an electronic device whose operation of a predefined set of functions is uniquely performed via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating electronic device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on electronic device 100 is optionally reduced.

[0093] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates the electronic device 100 from any user interface displayed on the electronic device 100 to a main menu, home menu, or root menu. In such embodiments, a touchpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0094] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 (in Figure 1A (in) or memory 370 (in) Figure 3 The (in the middle) includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0095] Event classifier 170 receives event information and determines the application 136-1 to which the event information should be delivered and the application view 191 of application 136-1. Classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display system 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0096] In some embodiments, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and an undo / redo queue for the user's previous actions.

[0097] Event monitor 171 receives event information from peripheral device interface 118. The event information includes information about sub-events, such as user touches on touch-sensitive display system 112 as part of a multi-touch gesture. Peripheral device interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 167, one or more gyroscopes 168, one or more magnetometers 169, and / or microphone 113 (via audio circuitry 110). The information received by peripheral device interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or touch-sensitive surfaces.

[0098] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0099] In some embodiments, the event classifier 170 further includes a hit view determination module 172 and / or an activity event recognizer determination module 173. When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that a user can see on the display.

[0100] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0101] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that forms an event or potential event) occurs. Once the hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0102] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, the higher-level view in the hierarchy will still remain the actively participating view.

[0103] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver module 182.

[0104] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0105] In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each application view including instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more event recognizers among the event recognizers 180 are part of a separate module, such as a user interface toolkit (not shown) or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0106] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event recognizer 180 includes an event receiver module 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0107] Event receiver module 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the electronic device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the current orientation of the electronic device (also referred to as device orientation).

[0108] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on a displayed object for a predetermined duration, movement of the touch on the touch-sensitive display system 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0109] In some embodiments, event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on a touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each display object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0110] In some embodiments, the definition of the corresponding event 187 also includes delay actions that delay the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0111] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0112] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0113] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some embodiments, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with the tag retrieves the tag and executes a predefined procedure.

[0114] In some embodiments, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and executes a predetermined procedure.

[0115] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or video files used in storage video player module 145. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0116] In some embodiments, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0117] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; movement of the electronic device; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0118] Figure 2 It is shown that, according to some embodiments, there is a touchscreen (e.g., Figure 1A A portable multi-functional device 100 (a touch-sensitive display system 112). The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figures) or one or more styluses 203 (not drawn to scale in the figures). In some embodiments, selection of one or more graphics occurs when the user interrupts contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the electronic device 100. In some embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0119] The stylus 203 includes a first end 276 and a second end 277. In various embodiments, the first end 276 corresponds to the tip of the stylus 203 (e.g., the tip of a pencil), and the second end 277 corresponds to the opposite end or bottom of the stylus 203 (e.g., the eraser of a pencil).

[0120] The stylus 203 includes a touch-sensitive surface 275 to receive touch input from a user. In some embodiments, the touch-sensitive surface 275 corresponds to a capacitive touch element. The stylus 203 includes a sensor or group of sensors that detect input from the user based on tactile and / or sensory contact with the touch-sensitive surface 275. In some embodiments, the stylus 203 includes any of a variety of touch sensing technologies now known or to be developed thereafter, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive surface 275, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies. Because the stylus 203 includes a variety of sensors and various types of sensors, the stylus 203 can detect a variety of inputs from the user, including gestures disclosed herein with respect to the touchscreen of the portable multifunction device 100. In some embodiments, one or more sensors can detect a single touch input or continuous touch input in response to a user tapping once or multiple times on the touch-sensitive surface 275. In some embodiments, one or more sensors can detect a swipe input on the stylus 203 in response to a user swiping along the touch-sensitive surface 275 with one or more fingers. In some embodiments, if the speed at which a user swipes along the touch-sensitive surface 275 exceeds a threshold, one or more sensors detect a flick input instead of a swipe input.

[0121] The stylus 203 also includes one or more sensors, such as an accelerometer, magnetometer, gyroscope, etc., for detecting the orientation (e.g., angular position relative to the electronic device) and / or movement of the stylus 203. The one or more sensors can detect various rotational movements of the stylus 203 by the user, including the type and direction of rotation. For example, the one or more sensors can detect the user rolling and / or rotating the stylus 203, and can detect the direction of rolling / rotating (e.g., clockwise or counterclockwise). In some embodiments, the detected input depends on the angular position of the first end 276 and the second end 277 of the stylus 203 relative to the electronic device. For example, in some embodiments, if the stylus 203 is substantially perpendicular to the electronic device 100 and the second end 277 (e.g., an eraser) is closer to the electronic device, contacting the surface of the electronic device with the second end 277 results in an erasing operation. On the other hand, if the stylus 203 is substantially perpendicular to the electronic device and the first end 276 (e.g., a tip) is closer to the electronic device, contacting the surface of the electronic device with the first end 276 results in a marking operation.

[0122] Electronic device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on electronic device 100. Alternatively, in some embodiments, menu button 204 is implemented as a soft key in a GUI displayed on a touchscreen display.

[0123] In some embodiments, the electronic device 100 includes a touchscreen display, a menu button 204, a push-button 206 for powering on / off the electronic device and locking the electronic device 100, a volume control button 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to: power on / off the electronic device by pressing the push-button 206 and holding it in the pressed state for a predefined time interval; lock the electronic device 100 by pressing the push-button 206 and releasing it before the predefined time interval has elapsed; and / or unlock the electronic device 100 or initiate an unlocking process. In some embodiments, the electronic device 100 also accepts voice input via a microphone 113 for activating or deactivating certain functions. The electronic device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112 and / or one or more haptic output generators 163 for generating haptic outputs for the user of the electronic device 100.

[0124] Figure 3 This is a block diagram of an exemplary multi-functional device 300 having a display and a touch-sensitive surface according to some embodiments. The electronic device 300 need not be portable. In some embodiments, the electronic device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). The electronic device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The one or more communication buses 320 optionally include circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. The electronic device 300 includes an input / output (I / O) interface 330 having a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on the electronic device 300 (e.g., similar to the reference above). Figure 1AThe tactile output generator 163 and sensor 359 (e.g., similar to the one mentioned above) are described. Figure 1A The sensors 112, 164, 165, 166, 167, 168, and 169 are touch sensors, optical sensors, contact strength sensors, proximity sensors, acceleration sensors, attitude sensors, and / or magnetic sensors. Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices remotely located with one or more CPUs 310. In some embodiments, memory 370 stores data with portable multifunction device 100 (… Figure 1A The programs, modules, and data structures stored in the memory 102 of the portable multifunction device 100 are similar to, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 ( Figure 1A The memory 102 may optionally not store these modules.

[0125] Figure 3 Each of the elements identified above is optionally stored in one or more of the previously mentioned memory devices. Each of the modules identified above corresponds to a set of instructions for performing the functions described above. The modules or programs identified above (i.e., the instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subgroup of the modules and data structures described above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0126] Figure 4This is a block diagram of an exemplary electronic stylus 203 according to some embodiments. The electronic stylus 203 is sometimes simply referred to as a stylus. The stylus 203 includes a memory 402 (which optionally includes one or more computer-readable storage media), a memory controller 422, one or more processing units (CPUs) 420, a peripheral device interface 418, RF circuitry 408, an input / output (I / O) subsystem 406, and other input or control devices 416. The stylus 203 optionally includes an external port 424 and one or more optical sensors 464. The stylus 203 optionally includes one or more intensity sensors 465 for detecting the intensity of contact of the stylus 203 on device 100 (e.g., when the stylus 203 is used with a touch-sensitive surface such as a touch-sensitive display system 112 of electronic device 100) or on other surfaces (e.g., a table surface). The stylus 203 optionally includes one or more haptic output generators 463 for generating haptic output on the stylus 203. These components can optionally communicate via one or more communication buses or signal lines 403.

[0127] In some embodiments, the term "haptic output" discussed above refers to the physical displacement of an accessory (e.g., stylus 203) of an electronic device (e.g., electronic device 100) relative to a previous position of the accessory, the physical displacement of a component of the accessory relative to another component of the accessory, or the displacement of the component relative to the center of mass of the accessory, as detected by the user using their tactile senses. For example, when the accessory or a component of the accessory comes into contact with a touch-sensitive surface of the user (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the accessory or a component of the accessory. For example, movement of a component (e.g., the housing of stylus 203) may optionally be interpreted by the user as a "click" on a physically actuated button. In some cases, the user will feel a tactile sensation such as a "click" even when the physically actuated button associated with the stylus, which is physically pressed (e.g., displaced) by the user's movement, does not move. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when tactile output is described as corresponding to a specific sensory perception of a user (e.g., "click"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of an electronic device or its components that would generate the sensory perception of a typical (or ordinary) user.

[0128] It should be understood that stylus 203 is merely an example of an electronic stylus, and stylus 203 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 4The various components shown are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.

[0129] Memory 402 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more flash memory devices or other non-volatile solid-state memory devices. Access to memory 402 by other components of stylus 203, such as one or more CPUs 420 and peripheral interface 418, is optionally controlled by memory controller 422.

[0130] Peripheral interface 418 can be used to couple the stylus's input and output peripherals to one or more CPUs 420 and memory 402. One or more CPUs 420 run or execute various software programs and / or instruction sets stored in memory 402 to perform various functions of stylus 203 and process data.

[0131] In some embodiments, the peripheral interface 418, one or more CPUs 420, and memory controller 422 are optionally implemented on a single chip, such as chip 404. In some other embodiments, they are optionally implemented on separate chips.

[0132] RF (Radio Frequency) circuit 408 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 408 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with electronic device 100 or electronic device 300, communication networks, and / or other communication devices via electromagnetic signals. RF circuit 408 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 408 optionally communicates wirelessly with networks and other devices, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). The wireless communication may optionally use any of a variety of communication standards, protocols, and / or technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolved Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and / or IEEE... 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including those not yet developed as of the date of this document submission.

[0133] The I / O subsystem 406 couples input / output peripherals such as other input or control devices 416 on the stylus 203 to the peripheral interface 418. The I / O subsystem 406 optionally includes an optical sensor controller 458, an intensity sensor controller 459, a haptic feedback controller 461, and one or more input controllers 460 for the other input or control devices. One or more input controllers 460 receive electrical signals from / send electrical signals to the other input or control device 416. The other input or control devices 416 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, click wheels, etc. In some alternative embodiments, one or more input controllers 460 are optionally coupled to (or not coupled to) any of the following: an infrared port and / or a USB port.

[0134] The stylus 203 also includes a power system 462 for supplying power to various components. The power system 462 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in portable devices and / or portable accessories.

[0135] The stylus 203 may optionally also include one or more optical sensors 464. Figure 4 An optical sensor coupled to an optical sensor controller 458 in an I / O subsystem 406 is shown. One or more optical sensors 464 optionally include charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) phototransistors. The one or more optical sensors 464 receive light projected from the environment through one or more lenses and convert the light into data representing an image.

[0136] The stylus 203 may optionally also include one or more contact intensity sensors 465. Figure 4 A contact strength sensor coupled to a strength sensor controller 459 in I / O subsystem 406 is shown. One or more contact strength sensors 465 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a surface). One or more contact strength sensors 465 receive contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to the tip of stylus 203.

[0137] The stylus 203 may optionally also include one or more proximity sensors 466. Figure 4 A proximity sensor 466 coupled to a peripheral device interface 418 is shown. Alternatively, the proximity sensor 466 is coupled to an input controller 460 in an I / O subsystem 406. In some embodiments, the proximity sensor 466 determines the proximity of the stylus 203 to an electronic device (e.g., electronic device 100).

[0138] The stylus 203 may optionally also include one or more haptic output generators 463. Figure 4 A haptic output generator 463 coupled to a haptic feedback controller 461 in an I / O subsystem 406 is shown. One or more haptic output generators 463 optionally include one or more electroacoustic devices (such as speakers or other audio components), and / or electromechanical devices that convert energy into linear motion (such as motors, solenoids, electroactive polymerizers, piezoelectric actuators, electrostatic actuators), or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on electronic devices). One or more haptic output generators 463 receive haptic feedback generation instructions from a haptic feedback module 433 and generate a haptic output on a stylus 203 that can be felt by a user of the stylus 203. In some embodiments, at least one haptic output generator 463 is juxtaposed or adjacent to the length (e.g., body or housing) of the stylus 203, and optionally generates the haptic output by moving the stylus 203 vertically (e.g., in a direction parallel to the length of the stylus 203) or laterally (e.g., in a direction perpendicular to the length of the stylus 203).

[0139] The stylus 203 may optionally also include one or more accelerometers 467, gyroscopes 468 and / or magnetometers 469 (e.g., as part of an inertial measurement unit (IMU)) for acquiring information about the position and position state of the stylus 203. Figure 4 Sensors 467, 468, and 469, coupled to peripheral interface 418, are shown. Alternatively, sensors 467, 468, and 469 may be coupled to input controller 460 in I / O subsystem 406. Stylus 203 may optionally include a GPS (or GLONASS or other global navigation system) receiver (not shown) for acquiring information about the location of stylus 203.

[0140] The stylus 203 includes a touch-sensitive system 432. The touch-sensitive system 432 detects input received at the touch-sensitive surface 275. These inputs include those discussed herein with respect to the touch-sensitive surface 275 of the stylus 203. For example, the touch-sensitive system 432 can detect tap, rotate, scroll, flick, and swipe inputs. The touch-sensitive system 432 cooperates with a touch interpretation module 477 to interpret specific types of touch input received at the touch-sensitive surface 275 (e.g., rotate / scroll / flick / swipe, etc.).

[0141] In some embodiments, the software components stored in memory 402 include an operating system 426, a communication module (or instruction set) 428, a contact / motion module (or instruction set) 430, a location module (or instruction set) 431, and a Global Positioning System (GPS) module (or instruction set) 435. Furthermore, in some embodiments, memory 402 stores device / global internal state 457, such as... Figure 4 As shown. Furthermore, although not shown, memory 402 includes a touch interpretation module 477. Device / global internal state 457 includes one or more of the following: sensor state, which includes information obtained from various sensors of the stylus and other input or control devices 416; position state, which includes information about the stylus's position relative to an electronic device (e.g., electronic device 100) (e.g., position, orientation, tilt, roll, and / or distance, such as...). Figure 5A and Figure 5B Information (as shown); and location information about the stylus position (e.g., determined by GPS module 435).

[0142] Operating system 426 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, power management, etc.) and facilitates communication between various hardware and software components.

[0143] Communication module 428 is optionally adapted to communicate with other devices via one or more external ports 424, and also includes various software components for processing data received by RF circuitry 408 and / or external ports 424. External ports 424 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, Wireless LAN, etc.). In some embodiments, external port 424 is a Lightning connector that is the same as, similar to, and / or compatible with, the Lightning connector used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc., Cupertino, California.

[0144] The contact / motion module 430 optionally detects contact with the stylus 203 and other touch-sensitive devices of the stylus 203 (e.g., buttons or other touch-sensitive components of the stylus 203). The contact / motion module 430 includes software components for performing various operations related to the detection of contact (e.g., detection of contact between the tip of the stylus 203 and a touch-sensitive display such as the touchscreen 112 of the electronic device 100 or with another surface such as a table surface). These various operations include determining whether a contact has occurred (e.g., detecting a touch press event), determining the intensity of the contact (e.g., the force or pressure of the contact or a substitute for force or pressure), determining whether there is movement of the contact and tracking across that movement (e.g., across the touchscreen 112 of the electronic device 100), and determining whether the contact has stopped (e.g., detecting a lift-off event or a contact interruption). In some embodiments, the contact / motion module 430 receives contact data from the I / O subsystem 406. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. As described above, in some embodiments, one or more of these operations related to the detection of contact are performed by the electronic device 100 using the contact / motion module 130 (as a supplement to or replacement of the stylus 203, the contact / motion module 430 is used).

[0145] The contact / motion module 430 optionally detects gesture input via the stylus 203. Different gestures performed with the stylus 203 have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single tap gesture includes detecting a touch press event, followed by detecting a lift-off event at the same (or substantially the same) location as the touch press event (e.g., at the location of an icon). As another example, detecting a swipe gesture includes detecting a touch press event, followed by detecting one or more stylus drag events, and then detecting a lift-off event. As described above, in some embodiments, gesture detection is performed by the electronic device 100 using the contact / motion module 130 (using the contact / motion module 430 as a supplement to or alternative to the stylus 203).

[0146] In conjunction with accelerometer 467, gyroscope 468, and / or magnetometer 469, position module 431 optionally detects position information about stylus 203, such as the stylus's attitude (roll, pitch, and / or yaw) in a particular frame of reference. In conjunction with accelerometer 467, gyroscope 468, and / or magnetometer 469, position module 431 optionally detects stylus movement gestures, such as flicking, tapping, and rotating of stylus 203. Position module 431 includes software components for performing various operations related to detecting the stylus's position and detecting changes in the stylus's position in a particular frame of reference. In some embodiments, position module 431 detects the positional state of stylus 203 relative to electronic device 100 and detects changes in the positional state of stylus 203 relative to electronic device 100. As described above, in some embodiments, electronic device 100 or electronic device 300 uses position module 131 to determine the position state of stylus 203 relative to electronic device 100 and changes in the position state of stylus 203 (position module 431 is used as a supplement or alternative to stylus 203).

[0147] The haptic feedback module 433 includes various software components for generating instructions used by one or more haptic output generators 463 to produce haptic output at one or more locations on the stylus 203 in response to user interaction with the stylus 203.

[0148] GPS module 435 determines the location of stylus 203 and provides that information for use in various applications (e.g., applications that provide location-based services, such as applications for finding lost devices and / or accessories).

[0149] Touch interpretation module 477 cooperates with touch-sensitive system 432 to determine (e.g., interpret or recognize) the type of touch input received at touch-sensitive surface 275 of stylus 203. For example, if a user swipes a sufficient distance on touch-sensitive surface 275 for a sufficiently short period of time, touch interpretation module 477 determines that the touch input corresponds to a swipe input (rather than a tap). Similarly, if the user swipes on touch-sensitive surface 275 quickly enough to correspond to a voice input that corresponds to a swipe, touch interpretation module 477 determines that the touch input corresponds to a flick input (rather than a swipe). The threshold speed of the swipe can be preset and varied. In various embodiments, the pressure and / or force of the touch received at touch-sensitive surface determines the type of input. For example, a light touch may correspond to a first type of input, while a stronger touch may correspond to a second type of input.

[0150] Each module and application identified above corresponds to a set of executable instructions for performing one or more of the functions described above and the methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 402 optionally stores a subset of the modules and data structures described above. Furthermore, memory 402 optionally stores additional modules and data structures not described above.

[0151] Figures 5A to 5B The diagram illustrates the positional state of a stylus 203 relative to a touch-sensitive surface (e.g., the touchscreen 112 of the electronic device 100) according to some embodiments. In some embodiments, the positional state of the stylus 203 corresponds to (or indicates): the projected position of the tip (or other representative portion) of the stylus 203 on the touch-sensitive surface (e.g., Figure 5A The (x,y) position 504 in the stylus 203 relative to the touch-sensitive surface (e.g., Figure 5A Orientation 506), tilt of stylus 203 relative to touch-sensitive surface (e.g., Figure 5B The tilt 512) and / or the distance of the stylus 203 relative to the touch-sensitive surface (e.g., Figure 5B The distance 514 in the distance). In some embodiments, the position state of the stylus 203 corresponds to (or indicates) the pitch, yaw, and / or rotation of the stylus (e.g., the attitude of the stylus 203 relative to a particular reference frame such as a touch-sensitive surface (e.g., touchscreen 112) or the ground). In some embodiments, the position state includes a set of position parameters (e.g., one or more position parameters). In some embodiments, the position state is detected based on one or more measurements sent from the stylus 203 to an electronic device (e.g., electronic device 100). For example, the stylus 203 measures the tilt of the stylus (e.g., Figure 5B Inclination 512) and / or orientation (e.g., Figure 5A The orientation 506 in the stylus 203) and the measurement results are sent to the electronic device 100. In some embodiments, instead of or in combination with the position state detected based on one or more measurements from the stylus 203, the position state is detected based on the raw output from one or more electrodes in the stylus sensed by a touch-sensitive surface (e.g., the touchscreen 112 of the electronic device 100). For example, the touch-sensitive surface receives the raw output from one or more electrodes in the stylus 203 and calculates the tilt and / or orientation of the stylus 203 based on the raw output (optionally, in combination with position state information provided by the stylus 203 based on sensor measurements generated by the stylus 203).

[0152] Figure 5AThe image shows a stylus 203 relative to a touch-sensitive surface, viewed from a perspective directly above the touch-sensitive surface (e.g., the touchscreen 112 of electronic device 100), according to some embodiments. Figure 5A In this configuration, the z-axis 594 points out of the page (i.e., in a direction perpendicular to the plane of the touchscreen 112), the x-axis 590 is parallel to the first edge (e.g., length) of the touchscreen 112, the y-axis 592 is parallel to the second edge (e.g., width) of the touchscreen 112, and the y-axis 592 is perpendicular to the x-axis 590.

[0153] Figure 5A The tip of stylus 203 is shown at (x,y) position 504. In some embodiments, the tip of stylus 203 is the end of stylus 203 configured to determine the proximity of stylus 203 to a touch-sensitive surface (e.g., touchscreen 112). In some embodiments, the projection of the tip of stylus 203 onto the touch-sensitive surface is an orthogonal projection. In other words, the projection of the tip of stylus 203 onto the touch-sensitive surface is at the end of a line perpendicular to the surface of the touch-sensitive surface from the stylus tip (e.g., (x,y) position 504, where the tip of stylus 203 would touch the touch-sensitive surface if the stylus 203 were moved directly along a path perpendicular to the touch-sensitive surface). In some embodiments, (x,y) position 504 at the lower left corner of touchscreen 112 is position (0,0) (e.g., (0,0) position 502), and other (x,y) positions on touchscreen 112 are relative to the lower left corner of touchscreen 112. Alternatively, in some embodiments, the (0,0) position is located at another location on the touchscreen 112 (e.g., at the center of the touchscreen 112) and other (x,y) positions are relative to the (0,0) position on the touchscreen 112.

[0154] in addition, Figure 5A The stylus 203 is shown to have an orientation 506. In some embodiments, orientation 506 is the orientation of the projection of the stylus 203 onto the touchscreen 112 (e.g., an orthogonal projection of the length of the stylus 203 or a line corresponding to the line between the projections of two different points of the stylus 203 onto the touchscreen 112). In some embodiments, orientation 506 is relative to at least one axis in a plane parallel to the touchscreen 112. In some embodiments, orientation 506 is relative to a single axis in a plane parallel to the touchscreen 112 (e.g., axis 508, having a clockwise rotation angle varying from 0 degrees to 360 degrees starting from axis 508, such as...). Figure 5A (As shown). Alternatively, in some embodiments, orientation 506 is relative to a pair of axes in a plane parallel to touchscreen 112 (e.g., x-axis 590 and y-axis 592, as shown). Figure 5A As shown, or a pair of axes associated with an application displayed on touchscreen 112.

[0155] In some embodiments, an indication (e.g., indication 516) is displayed on a touch-sensitive display (e.g., the touchscreen 112 of electronic device 100). In some embodiments, indication 516 indicates the location on the touch-sensitive display that the stylus 203 will touch (or mark) before the stylus 203 touches the touch-sensitive display. In some embodiments, indication 516 is part of a mark drawn on the touch-sensitive display. In some embodiments, indication 516 is separate from the mark drawn on the touch-sensitive display and corresponds to a virtual “pen tip” or other element indicating the location on the touch-sensitive display where a mark will be drawn.

[0156] In some embodiments, the indicator 516 is displayed based on the position state of the stylus 203. For example, in some cases, the indicator 516 is shifted from the (x,y) position 504 (e.g., Figure 5A and Figure 5B As shown), and in other cases, indication 516 does not shift from (x,y) position 504 (e.g., when tilt 512 is zero degrees, indication 516 is displayed at or near (x,y) position 504). In some embodiments, indication 516 is displayed with varying colors, sizes (or radii or areas), opacities, and / or other characteristics depending on the positional state of stylus 203. In some embodiments, the displayed indication takes into account the thickness of the glass layer on the touch-sensitive display so that indication 516 is applied to the “pixels” of the touch-sensitive display, rather than displayed on the “glass” covering the pixels.

[0157] Figure 5B The image shows a stylus 203 relative to a touch-sensitive surface (e.g., a touchscreen 112 of an electronic device 100) as viewed from a side view, according to some embodiments. Figure 5B In the diagram, the z-axis 594 points in a direction perpendicular to the plane of the touchscreen 112, the x-axis 590 is parallel to the first edge (e.g., length) of the touchscreen 112, the y-axis 592 is parallel to the second edge (e.g., width) of the touchscreen 112, and the y-axis 592 is perpendicular to the x-axis 590.

[0158] Figure 5B The stylus 203 is shown to have a tilt 512. In some embodiments, the tilt 512 is an angle relative to the normal to the surface of the touch-sensitive surface (also simply referred to as the normal to the touch-sensitive surface) (e.g., normal 510). Figure 5B As shown, when the stylus is perpendicular to / perpendicular to the touch-sensitive surface (e.g., when the stylus 203 is parallel to the normal 510), the tilt 512 is zero, and as the stylus 203 tilts closer to being parallel to the touch-sensitive surface, the tilt 512 increases.

[0159] in addition, Figure 5BThe distance 514 between the stylus 203 and the touch-sensitive surface is shown. In some embodiments, the distance 514 is the distance from the tip of the stylus 203 to the touch-sensitive surface in a direction perpendicular to the touch-sensitive surface. For example, in Figure 5B In the middle, distance 514 is the distance from the tip of stylus 203 to position 504 at (x,y).

[0160] Although this document uses the terms "x-axis," "y-axis," and "z-axis" to illustrate certain directions in specific figures, it should be understood that these terms do not refer to absolute directions. In other words, the "x-axis" can be any corresponding axis, and the "y-axis" can be a specific axis different from the x-axis. Typically, the x-axis is perpendicular to the y-axis. Similarly, the "z-axis" is different from the "x-axis" and "y-axis," and is typically perpendicular to both the "x-axis" and "y-axis."

[0161] in addition, Figure 5B The rotation 518 is shown—a rotation about the length (major axis) of the stylus 203.

[0162] Now turn attention to an embodiment of the user interface (“UI”) optionally implemented on the portable multifunction device 100.

[0163] Figure 6A An exemplary user interface for a menu used on a portable multifunction device 100 according to some embodiments is shown. A similar user interface may optionally be implemented on an electronic device 300. In some embodiments, the user interface 600 includes the following elements or a subset or superset thereof: • One or more signal strength indicators 602 for one or more wireless communications such as cellular signals and Wi-Fi signals; • Time 604; • Bluetooth indicator 605; • Battery status indicator 606; • Tray 608 with icons for frequently used applications, such as: ○ The telephone module 138 has an icon 616 labeled "telephone", which optionally includes an indicator 614 indicating the number of missed calls or voicemails; ○ An icon 618 labeled "Mail" in the email client module 140, which optionally includes an indicator 610 for the number of unread emails; ○ The icon 620 labeled "Browser" in browser module 147; and ○ The icon 622 marked "iPod" for the video and music player module 152 (also known as the iPod® (Apple Inc. trademark) module 152); and Icons of other applications, such as: ○ Icon 624 labeled "Message" in IM module 141; ○ The icon 626 labeled "Calendar" in calendar module 148; ○ The icon 628 labeled "Photo" in the image management module 144; ○ The icon 630 of camera module 143, labeled "camera"; ○ Icon 632 of the online video module 155, labeled "Online Video"; ○ Icon 634 in the Stock Market Desktop Mini Program 149-2, labeled "Stock Market"; ○ Map module 154's icon 636 labeled "Map"; ○ The icon labeled "Weather" in the Weather desktop mini-program 149-1 is 638; ○ The icon labeled "Clock" in the alarm clock desktop mini-program 149-4 is 640; ○ The icon 642 of the fitness support module 142 is labeled "Fitness Support"; ○ The icon 644 labeled "Notepad" in Notepad module 153; and ○ An icon 646 is used to set up an application or module, which provides access to settings for the electronic device 100 and its various applications 136.

[0164] It should be noted that Figure 6A The icon labels shown are merely exemplary. For example, in some embodiments, the icon 622 of the video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a corresponding application icon includes the name of the application corresponding to that application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0165] Figure 6B A touch-sensitive surface 651 separate from the display 650 is shown (e.g., Figure 3 Electronic devices (e.g., tablets or touchpads 355) Figure 3 An exemplary user interface on device 300. Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more sensors 359) for detecting the intensity of contact on the tactile surface 651 and / or one or more tactile output generators 357 for generating tactile output for the user of device 300.

[0166] Figure 6BA touch-sensitive surface 651 separate from the display 650 is shown (e.g., Figure 3 Electronic devices (e.g., tablets or touchpads 355) Figure 3 An exemplary user interface on device 300. Although many examples of input on a reference touchscreen display 112 (which combines a touch-sensitive surface and a display) are given in the following examples, in some embodiments, the electronic device 100 detects input on a touch-sensitive surface separate from the display, such as Figure 6B As shown. In some embodiments, the touch-sensitive surface (e.g., Figure 6B The touch-sensitive surface 651 in the display (e.g., 650) has a main axis (e.g., on the display) that is aligned with the main axis on the display (e.g., on the display itself). Figure 6B The principal axis 653 in the middle corresponds to the principal axis (e.g., Figure 6B The main axis 652 in the middle). According to these embodiments, the electronic device 100 detects contact with the touch-sensitive surface 651 at a position corresponding to a corresponding position on the display (e.g., the main axis 652 in the display). Figure 6B Contacts 660 and 662 in (e.g., in Figure 6B In this way, contact 660 corresponds to position 668 and contact 662 corresponds to position 670. Thus, when the touch-sensitive surface (e.g., Figure 6B The touch-sensitive surface 651 in the device and the display of the electronic device 100 (e.g., Figure 6B When the display 650 is separated, user input detected by the electronic device 100 on the touch-sensitive surface (e.g., touches 660 and 662 and their movement) is used by the electronic device 100 to manipulate the user interface on the display. It should be understood that similar methods may optionally be used for other user interfaces described herein.

[0167] Additionally, while the examples below are primarily given with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture, etc.) and / or stylus input, it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the touch). Similarly, a tap gesture is optionally replaced by a mouse click when the cursor is above the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Likewise, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or mouse and finger (or stylus) touch are optionally used simultaneously.

[0168] User interface and related processes Now let’s turn our attention to electronic devices (such as the portable multifunction device 100 in Figure 1 or...) Figure 3 An embodiment of a user interface (“UI”) and associated processes implemented on an electronic device 300, wherein one or more input devices are used to detect various inputs (e.g., touch input, stylus input, mouse input, keyboard input, etc.), and a display device is used to manipulate the user interface based on the various inputs.

[0169] Figures 7A to 7CF These are examples of user interfaces for repositioning a drawing palette, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 10A to 10D The process in the process. Although some examples in the following examples will refer to input on a touchscreen display (where a touch-sensitive surface and a display are combined, such as on touchscreen 112), in some embodiments, electronic device 100 detects input on a touch-sensitive surface 651 separate from display 650, such as Figure 6B As shown.

[0170] like Figure 7A As shown, the electronic device 100 displays a first application interface 702, such as a drawing application interface or a word processing application interface. The first application interface 702 is defined by a first edge 701a, a second edge 701b, a third edge 701c, and a fourth edge 701d.

[0171] Electronic device 100 displays a first drawing palette 704 having a first appearance. The first drawing palette 704 is displayed along a fourth side 701d of a first application interface 702 (e.g., close to, anchored to, and / or substantially parallel to the fourth side). The first drawing palette 704 may include various indications (e.g., drawing tool indications, editing function indications, and / or color palettes) to facilitate content manipulation operations. For example, such as... Figure 7A As shown, the first drawing palette 704 includes an undo indicator 704a, a restore indicator 704b, a set of drawing tool indicator indicators 704c, a set of color palettes 704d, a text tool indicator 704e, a shape tool indicator 704f, and additional function indicator indicators 704g. Those skilled in the art will understand that the first drawing palette 704 may include any number and type of indicator indicators arranged in any manner.

[0172] An input pointing to the undo indicator 704a requests the electronic device 100 to undo a previous operation, such as erasing a previously drawn mark. An input pointing to the reset indicator 704b requests the electronic device 100 to reset a previously undone operation, such as redisplaying a previously erased mark.

[0173] A set of drawing tool power indicators 704c (from left to right) includes a pen power indicator, a marker power indicator (e.g., a highlighter power indicator), a pencil power indicator, a ruler power indicator, and an eraser power indicator. For example... Figure 7A As shown, the pencil indicator shows that the pencil is selected as the currently selected drawing tool. Input pointing to the corresponding drawing tool indicator sets that drawing tool as the currently selected drawing tool.

[0174] A set of color tanks 704d includes a top row color indicator for setting the currently selected color and a bottom row pattern indicator for setting the currently selected pattern associated with that color. For example... Figure 7A As shown, the current selection is black and solid pattern. Input pointing to the corresponding color indicator or the corresponding pattern indicator will set the corresponding color / pattern as the currently selected color / pattern.

[0175] The text tool enablement 704e enables the creation of text content within the first application interface 702. For example, after selecting the text tool enablement 704e, input pointing to the first application interface 702 causes the electronic device 100 to display a text box for receiving a text string, and causes the electronic device 100 to replace the text box with the text string entered into the text box.

[0176] The shape tool indicator 704f enables the placement of a specific shape within the first application interface 702. In some embodiments, for example, input pointing to the shape tool indicator 704f brings up a shape interface including various predetermined shapes (e.g., squares, circles, triangles). Subsequently, the electronic device 100 detects input corresponding to dragging a specific shape from the shape interface into the first application interface 702. In response, the electronic device 100 displays the specific shape at that location within the first application interface 702.

[0177] like Figure 7B As shown, the electronic device 100 detects drag input 708, which corresponds to a request to move a first drawing palette 704 within a first application interface 702. Drag input 708 corresponds to a request to move the first drawing palette 704 away from the fourth edge 701d of the first application interface 702 towards the first edge 701a of the first application interface 702. In some embodiments, the electronic device 100 detects drag input 708 on a touch-sensitive surface of the electronic device 100, such as detecting finger drag input or stylus drag input. In some embodiments, drag input 708 corresponds to mouse drag input (e.g., click and drag).

[0178] For illustrative purposes only, Figure 7BIt also includes a first threshold line 706 at a first distance 706a from the first edge 701a of the first application interface 702. It is worth noting that, as... Figure 7B As shown, the end point of dragging input 708 exceeds the first threshold line 706.

[0179] like Figure 7C As shown, drag input 708 from... Figure 7B The initial position is moved to a second position closer to the first edge 701a. Therefore, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709. The drawing tool indicator 709 includes a black-tipped pencil 710a, because... Figure 7A and Figure 7B As shown, a set of drawing tool indicators 704c indicates that the currently selected drawing tool is a pencil, and a set of color palettes 704d indicates that black is the currently selected color. Furthermore, the drawing tool indicator 709 is oriented upwards (e.g., towards the north), because... Figure 7A and Figure 7B In the diagram, a set of drawing tools indicates that the 704c is also oriented upwards.

[0180] like Figure 7D As shown, dragging input 708 continues to move closer to the first edge 701a, crossing the first threshold line 706. Electronic device 100 maintains the orientation of drawing tool indicator 709 because, as... Figure 7E As shown, a set of drawing tool indicators 704c within the repositioned first drawing palette 704 continues to face upwards. In response to the completion of the drag input 708, the electronic device 100 replaces the drawing tool indicators 709 (in the first drawing palette 704 with the first appearance) along the first edge 701a. Figure 7E middle).

[0181] like Figure 7F As shown, the electronic device 100 detects a tap input 711 directed at the first drawing palette 704. In some embodiments, the tap input 711 corresponds to a single-click or double-click input detected on the touch-sensitive surface of the electronic device 100. In some embodiments, the tap input 711 corresponds to a single mouse click or a double mouse click.

[0182] In response to the detection of tap input 711 (in) Figure 7F In the middle), the electronic device 100 moves the first drawing palette 704 to its previous position along the fourth edge 701d, as... Figures 7G to 7I As shown. That is, as Figure 7GAs shown, the electronic device 100 moves the first drawing palette 704 along line path 712 toward the fourth edge 701d. In some embodiments, the electronic device 100 moves the first drawing palette 704 according to an animation. During the transition from the first edge 701 back to the fourth edge 701d, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709, as shown. Figure 7H As shown. Similarly, the electronic device 100 keeps the drawing tool indicator 709 oriented upwards (e.g., towards the north), because as Figure 7I As shown, a set of drawing tools indicates that the 704c maintains an upward orientation. Figure 7I The transition is shown to be complete, with the electronic device 100 displaying a first drawing palette 704 with a first appearance along the fourth edge 701d.

[0183] like Figure 7I As further shown, the electronic device 100 detects drag input 713. However, with Figures 7B to 7D Unlike the previous drag input 708 shown, drag input 713 does not cross the first threshold line 706. Instead, drag input 713 ends at reference line 714, shown for illustrative purposes only. Figure 7J As shown, when the input 713 is dragged upwards away from the fourth edge 701d, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709. The drawing tool indicator 709 includes a pencil 710a with a black tip. Figure 7K As shown, in response to detecting that the release of drag input 713 did not exceed the first threshold line 706, the electronic device 100 moves the drawing tool indicator 709 back toward the fourth edge 701d, as indicated by line path 715. Finally, as Figure 7L As shown, the electronic device 100 displays a first drawing palette 704 with a first appearance along the fourth edge 701d.

[0184] like Figure 7MAs shown, in some embodiments, the electronic device 100 divides the device into seven regions 716a-716g, illustrated for illustrative purposes only. According to various embodiments, in response to detecting a specific input directed at the first drawing palette 704, the electronic device 100 moves the first drawing palette 704 from an initial position (e.g., along the fourth edge 701d) to an end position within a specific region of the seven regions 716a-716g. In some embodiments, when the first drawing palette 704 is located within a specific region of the seven regions 716a-716g, the electronic device 100 changes the appearance of the first drawing palette from a first appearance to a second appearance. In the following, in various embodiments, the drawing palette is moved to that specific region of the first application interface 702 when a large portion of the drawing palette is within the specific region or when the center of the drawing palette is within the specific region. Therefore, in various embodiments, the drawing palette can be moved to the specific region even if a portion of it is outside the specific region.

[0185] Specifically, the first region 716a corresponds to a first corner of the first application interface 702, which intersects with a first edge 701a and a second edge 701b. The second region 716b corresponds to a portion of the first application interface 702 extending along the second edge 701b. The third region 716c corresponds to a second corner of the first application interface 702, which intersects with the second edge 701b and a fourth edge 701d. The fourth region 716d corresponds to a third corner of the first application interface 702, which intersects with the first edge 701a and the third edge 701c. The fifth region 716e corresponds to a portion of the first application interface 702 extending along the third edge 701c. The sixth region 716f corresponds to a fourth corner of the first application interface 702, which intersects with the third edge 701c and the fourth edge 701d. The seventh region 716g corresponds to a portion of the first application interface 702 extending along the first edge 701a.

[0186] like Figure 7N As shown, please continue to refer to... Figure 7M Each of the seven regions 716a-716g is associated with one or more threshold lines located away from the corresponding edge. According to various embodiments, when a drag input pointing to the first drawing palette 704 crosses the corresponding threshold line, the electronic device 100 moves the first drawing palette 704 to a specific region among the seven regions 716a-716g. The threshold lines are shown for illustrative purposes only. Those skilled in the art will understand that the positions of the threshold lines may vary according to various embodiments.

[0187] That is, the seventh region 716g is associated with the first threshold line 706 a first distance 706a from the first edge 701a, as referenced above. Figures 7B to 7D and Figures 7I to 7K The first region 716a is associated with a second threshold line 718 at a second distance 718a from the first edge 701a and a third threshold line 717 at a third distance 717a from the second edge 701b. The second region 716b is associated with the third threshold line 717. The third region 716c is associated with the third threshold line 717 and a fourth threshold line 719 at a fourth distance 719a from the fourth edge 710d. The fourth region 716d is associated with a fifth threshold line 721 at a fifth distance 721a from the first edge 701a and a sixth threshold line 720 at a sixth distance 722a from the third edge 701c. The fifth region 716e is associated with the sixth threshold line 720. The sixth region 716f is associated with the sixth threshold line 720 and a seventh threshold line 722 at a seventh distance 722a from the fourth edge 701d.

[0188] According to various embodiments, in response to detecting an input that meets one or more movement criteria, electronic device 100 moves a first drawing palette 704 to a specific region among seven regions 716a-716g. For example, in some embodiments, in response to detecting a drag input pointing towards the first drawing palette 704 that crosses one or more threshold lines, electronic device 100 moves the first drawing palette 704 to a specific region among the seven regions 716a-716g. As another example, in some embodiments, in response to detecting a flick input pointing towards the first drawing palette 704 that meets a velocity threshold (e.g., a direction threshold and a magnitude threshold), electronic device 100 moves the first drawing palette 704 to a specific region among the seven regions 716a-716g.

[0189] like Figure 7O As shown, the electronic device 100 detects drag input 723, which requests to move the first drawing palette 704 to the fifth region 716e. Figure 7P As shown, before detecting that drag input 723 crosses the sixth threshold line 720 associated with the fifth region 716e, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 that includes an upward-facing, black-tipped pencil 710a. The black-tipped pencil 710a is upward-facing because, as... Figure 7O The set of drawing tools in the 704c indicates that the currently selected drawing tool corresponds to an upward-facing, pointed black pencil.

[0190] like Figure 7Q As shown, in response to detecting that the drag input 723 has crossed the sixth threshold line 720, the electronic device 100 rotates the black-tipped pencil 710a so that it is substantially perpendicular to and away from the third edge 701c. The reason why the electronic device 100 rotates the black-tipped pencil 710a in this way is that, as Figure 7RAs shown, the electronic device 100 reorients the first drawing palette 704 and a set of drawing tool indications 704c so that the set of drawing tool indications are substantially perpendicular to and away from the third edge 701c.

[0191] like Figure 7R As shown, in response to the detection of completion of drag input 723, electronic device 100 moves along the third edge 701c to, for example, Figure 7O Different orientations of the first drawing palette 704 with a first appearance are displayed. It is worth noting that the electronic device 100 displays the first drawing palette 704 along the third edge 701c, and rotates a set of drawing tool indications 704c so that they face away from the third edge 701c.

[0192] like Figure 7S As shown, electronic device 100 (e.g., via inertial measurement unit (IMU)) detects a first device rotation input 724 that causes electronic device 100 to rotate 90 degrees counterclockwise. Therefore, as... Figure 7T As shown, the positions of the edges 701a-701d of the electronic device 100 relative to Figure 7S Change. In response to detection Figure 7S In the first device rotation input 724, the electronic device 100 moves the first drawing palette 704 from along the third edge 701c (as shown in the image). Figure 7S (As shown) Move along the fourth edge 701d. In this way, the electronic device 100 fixes (e.g., anchors) the first drawing palette 704 to a specific side (e.g., the right side) of the first application interface 702.

[0193] like Figure 7U As shown, electronic device 100 detects a second device rotation input 725 that causes electronic device 100 to rotate an additional 90 degrees counterclockwise. Therefore, as... Figure 7V As shown, the positions of the edges 701a-701d of the electronic device 100 relative to Figure 7U Change. In response to the detection of a second device rotary input 725 (in Figure 7U In the middle), the electronic device 100 will move the first drawing palette 704 from the edge Figure 7U The fourth edge 701d is moved along the second edge 701b to fix the first drawing palette 704 to the right side of the first application interface 702.

[0194] like Figure 7W As shown, the electronic device 100 detects drag input 726, which requests to move the first drawing palette 704 to the fifth region 716e. Figure 7XAs shown, before detecting that the drag input 726 crosses the sixth threshold line 720 associated with the fifth region 716e, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 including a black-tipped pencil 710a. The electronic device 100 holds the black-tipped pencil 710a substantially perpendicular to and away from the second edge 701b.

[0195] like Figure 7Y As shown, in response to detecting that the drag input 726 has crossed the sixth threshold line 720, the electronic device 100 rotates the black-tipped pencil 710a so that it is substantially perpendicular to and away from the third edge 701c. The reason why the electronic device 100 rotates the black-tipped pencil 710a in this way is that, as Figure 7Z As shown, the electronic device 100 enables a set of drawing tools to reorient the representation 704c so that it is substantially perpendicular to and away from the third edge 701c.

[0196] like Figure 7Z As shown, in response to the detection of completion of drag input 726, the electronic device 100 displays a first drawing palette 704 with a first appearance along the third edge 701c. It is noteworthy that the electronic device 100 displays the first drawing palette 704 along the third edge 701c and rotates a set of drawing tool indications 704c so that they are substantially perpendicular to and away from the third edge 701c. Furthermore, in some embodiments, such as... Figure 7Z As shown, the electronic device 100 displays a display with corresponding Figure 7W The first drawing palette 704 in the image is a mirror image of the corresponding appearance of the first drawing palette 704.

[0197] like Figure 7AA As shown, electronic device 100 detects that the pen is set as the input 728 for the currently selected drawing tool. (As...) Figure 7AB As shown, the electronic device 100 displays a first drawing palette 704, and specifically, displays a set of drawing tool indications 704c indicating that the pen is the currently selected drawing tool.

[0198] like Figure 7AC As shown, the electronic device 100 detects a flick input 729 pointing to the first drawing palette 704. It is worth noting that, due to... Figure 7S The first device shown has rotary input 724 and Figure 7U The second device shown has a rotary input 725, compared to the previous figure. Figure 7AC The areas displayed in the text have different locations. For example... Figure 7ACAs shown, the flick input 729 includes a horizontal component 729a and a vertical component 729b. Although the flick input 729 is in the direction 730 toward the seventh region 716g, the flick input 729 does not cross the corresponding first threshold line 706. However, in response to determining that the flick input 729 meets a velocity threshold, the electronic device 100 moves the first drawing palette 704 to the seventh region 716g. For example, in some embodiments, the flick input 729 meets the velocity threshold when it is associated with a sufficient magnitude (e.g., rate and / or acceleration). As another example, in some embodiments, the flick input 729 meets the velocity threshold when it is associated with a sufficiently horizontal rate and / or acceleration.

[0199] like Figure 7AD As shown, when the input 729 is pressed forward, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 that includes a black-tipped pen 710b. The electronic device 100 displays the black-tipped pen 710b because the black-tipped pen drawing tool is currently selected. Furthermore, the electronic device 100 displays the black-tipped pen 710b substantially perpendicular to and away from the third edge 701c to match... Figure 7AC A set of drawing tools in the system can represent the orientation of 704c.

[0200] like Figure 7AE As shown, the electronic device 100 partially rotates the black-tipped pen 710b so that it faces the... Figure 7AD The orientation of the black tip of the 710b fountain pen is similar to... Figure 7AF The orientation (e.g., 45-degree angle) of the black-tipped pen 710b. Unlike the previous example involving drag input, in response to the detection of a flick input 729, the electronics 100 moves the first drawing palette 704 to the target area (seventh area 716g), regardless of whether the flick input 729 crosses the first threshold line 706. Therefore, the electronics 100 begins to rotate the black-tipped pen 710b before the drawing tool indicator 709 reaches the first threshold line 706.

[0201] like Figure 7AF As shown, when the electronic device 100 moves the drawing tool indicator 709 into the seventh area 716g, the electronic device 100 completes the rotation of the black-tipped pen 710b to match. Figure 7AG A set of drawing tools in the software can represent the orientation of 704c. For example... Figure 7AG As shown, the electronic device 100 displays a first drawing palette 704 having a first appearance, the first drawing palette including a set of drawing tool indications 704c that are substantially perpendicular to and opposite to the first edge 701a.

[0202] like Figure 7AH As shown, the electronic device 100 detects that the pencil is set as the input 732 of the currently selected drawing tool. (As...) Figure 7AB As shown, the electronic device 100 displays a first drawing palette 704, and specifically, displays a set of drawing tool indications 704c indicating that the pencil is the currently selected drawing tool.

[0203] like Figure 7AJ As shown, the electronic device 100 detects drag input 733, which requests to move the first drawing palette 704 to the second area 716b. Figure 7AK As shown, before the drag input 733 is detected crossing into the second region 716b, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 that includes a black pencil with a pointed tip 710a, because the black pencil with a pointed tip is the currently selected drawing tool. The black pencil with a pointed tip 710a is substantially perpendicular to and away from the first edge 701a in order to match... Figure 7AJ A set of drawing tools in the system can represent the orientation of 704c.

[0204] like Figure 7AL As shown, in response to detecting that the drag input 733 has crossed into the second region 716b, the electronic device 100 rotates the black-tipped pencil 710a so that it is substantially perpendicular to and away from the second edge 701b. The reason why the electronic device 100 rotates the black-tipped pencil 710a in this way is that, as Figure 7AM As shown, the electronic device 100 enables a set of drawing tools to reorient the representation 704c so that it is substantially perpendicular to and away from the second edge 701b.

[0205] like Figure 7AM As shown, in response to the detection of completion of drag input 733, electronic device 100 displays a first drawing palette 704 with a first appearance along the second edge 701b. Notably, electronic device 100 displays the first drawing palette 704 along the second edge 701b and rotates a set of drawing tool indications 704c so that they are substantially perpendicular to and away from the second edge 701b.

[0206] like Figure 7AN As shown, the electronic device 100 detects a drag input 734, which requests the first drawing palette 704 to be moved to the first area 716a (corner area). Figure 7AO As shown, after the drag input 734 is detected but before the drag input 734 is completed, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 that includes a black pencil 710a.

[0207] like Figure 7APAs shown, in response to the detection of completion of drag input 734, the electronic device 100 displays a first drawing palette 704 with a second appearance different from the first appearance. The second appearance includes a currently selected drawing tool indicator 735a, which is substantially perpendicular to and away from the second edge 701b, because... Figure 7AM The first appearance associated with a set of drawing tools indicates that the 704c is similarly oriented.

[0208] In some embodiments, the first appearance corresponds to a first drawing palette in a first expanded state, and the second appearance corresponds to a first drawing palette in a compressed state. In some embodiments, the second appearance includes fewer content manipulation capabilities compared to the first appearance. For example, such as Figure 7AP As shown, the first drawing palette 704 with a second appearance includes a single indicator, such as a currently selected drawing tool indicator 735a corresponding to the currently selected drawing tool, while the first drawing palette 704 with a first appearance includes Figure 7AM Multiple content manipulation capabilities are indicated in 704a-704g.

[0209] like Figures 7AQ to 7AU As shown, the electronic device 100 transforms the first drawing palette 704, which is in a compressed state, into a second extended state, different from the first extended state. For example... Figure 7AQ As shown, the electronic device 100 detects touch input 736 directed at the first drawing palette 704. In response to detecting touch input 736 (in... Figure 7AQ In the middle), electronic device 100 magnifies (e.g., expands or inflates) the first drawing palette 704 (in Figure 7AR (Middle). In some embodiments, touch input 736 corresponds to a touch input detected on a touch-sensitive surface for a duration of a first threshold amount. In some embodiments, Figure 7AR The enlarged first drawing palette 704 in the middle is relative to Figure 7AP and Figure 7AQ The first drawing palette 704 is in a second extended state. The enlarged first drawing palette 704 includes the indicator 735a of the currently selected drawing tool.

[0210] In some embodiments, such as Figure 7AS As shown, in response to a second threshold amount of time during which the detected touch input 736 continues to exceed the first threshold amount, the electronic device 100 replaces the first drawing palette 704 with a preview drawing palette 738. It is worth noting that the preview drawing palette 738 has a... Figure 7AR The enlarged first drawing palette 704 in the image is oriented to the left, the same as the one shown. Therefore, the electronic device 100 is relative to... Figure 7ARThe enlarged first drawing palette 704 in the preview drawing palette 738 is expanded vertically (e.g., upwards).

[0211] In some embodiments, the preview drawing palette 738 is relative to Figure 7AP and Figure 7AQ The first drawing palette 704 is in a second extended state. The preview drawing palette 738 includes optional drawing tool indications 738a-738d, wherein the pencil indication 738a with a black tip has focus because it is the currently selected drawing tool. Those skilled in the art will understand that the preview drawing palette 738 may include any number and type of content manipulation indications. For example, in some embodiments, the preview drawing palette 738 includes corresponding indications for different tools such as pencils, pens, highlighters, and erasers.

[0212] like Figure 7AT As shown, the electronic device 100 detects a drag input 740 that selects a gray pencil tool indicator 738c. In some embodiments, the drag input 740 originates from a point of previous touch input 736. In response to detection... Figure 7AT In the drag input 740, the electronic device changes the currently selected drawing tool indicator 735a from a black pencil tip to a gray pencil tip within the first drawing palette 704. Figure 7AU middle).

[0213] like Figure 7AV As shown, the electronic device 100 detects touch input 742 directed at the first drawing palette 704. In response to the detection... Figure 7AV Touch input 742, electronic device 100 magnifies first drawing palette 704 (in Figure 7AW middle).

[0214] like Figure 7AX As shown, the electronic device 100 detects a drag input 744 that requests to move the first drawing palette 704 to the sixth region 716f. In some embodiments, the drag input 740 originates from a point of previous touch input 742. Figure 7AY As shown, before detecting drag input 744 crossing into the sixth region 716f, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 comprising a gray-tipped pencil 710c, which is substantially perpendicular to and away from the second edge 710b. The reason for the gray-tipped pencil 710c being oriented in this way is that... Figure 7AX The indicator 735a of the currently selected drawing tool within the first drawing palette 704 is similarly oriented.

[0215] like Figure 7AZAs shown, in response to detecting a drag input 744 crossing into the sixth region 716f, the electronic device 100 rotates the pointed gray pencil 710c so that it is substantially perpendicular to and away from the first edge 701a. The reason the electronic device 100 rotates the pointed gray pencil 710c in this manner is that maintaining the orientation of the pointed gray pencil 710c facing left toward the target third edge 701c would result in an unintuitive and undesirable user experience. Figure 7BA As shown, in response to detecting the completion of drag input 744, electronic device 100 replaces drawing tool indicator 709 with a first drawing palette 704 having a second appearance, the first drawing palette including an upward-facing currently selected drawing tool indicator 735a.

[0216] like Figure 7BB As shown, electronic device 100 detects touch input 746. In response to the detection... Figure 7BB During the duration of the touch input 746 in the electronic device 100, which is a first threshold amount, the electronic device 100 magnifies the first drawing palette 704 (in...). Figure 7BC middle).

[0217] like Figure 7BD As shown, in response to a second threshold amount for a duration during which the detected touch input 746 continues to exceed a first threshold amount, the electronic device 100 uses... Figure 7BD Replacement of the preview drawing palette 738 in the middle Figure 7BC The first drawing palette 704 in the image. It is worth noting that the preview drawing palette 738 has the same... Figure 7BC The enlarged first drawing palette 704 in the image has the same upward orientation. Therefore, the electronic device 100 is relative to... Figure 7BC The enlarged first drawing palette 704 in the image horizontally (e.g., to the right) expands the preview drawing palette 738. This is consistent with... Figure 7AS The first drawing palette 704, vertically expanded and enlarged by the electronic device 100, creates a contrast. The preview drawing palette 738 indicates that the pointed gray pencil 710c is the currently selected drawing tool.

[0218] like Figure 7BE As shown, when detecting touch input 746, the electronic device 100 detects stylus tap input 750 made by a stylus 203 held by the user's hand 748. The stylus tap input 750 points towards a black-tipped pencil 738a. In response to the detection... Figure 7BE When the stylus is tapped to input 750, the electronic device 100 replaces the preview drawing palette 738 with a first drawing palette 704 having a second appearance. Figure 7BF (In the middle). The first drawing palette 704, and specifically the currently selected drawing tool indicator 735a, indicate that the black pencil tip is the currently selected drawing tool.

[0219] like Figure 7BG As shown, the electronic device 100 detects touch input 752 directed at the first drawing palette 704. In response to the detection... Figure 7BG Touch input 752, electronic device 100 magnifies first drawing palette 704 (in Figure 7BH (In the middle). The enlarged first drawing palette 704 includes a currently selected drawing tool indicator 735a corresponding to the currently selected black pencil tip.

[0220] like Figure 7BI As shown, electronic device 100 detects drag input 754, which requests to move the first drawing palette 704 toward the bottom edge 701a. In response to detecting drag input 754 and before detecting drag input 754 crossing the first threshold line 706, electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709 (in... Figure 7BJ (Middle). The drawing tool indicator 709 includes an upward-facing, pointed black pencil 710a for matching in Figure 7BI The orientation of the currently selected drawing tool indicator 735a is displayed within the first drawing palette 704.

[0221] like Figure 7BK As shown, in response to detecting that the drag input 754 has crossed the first threshold line 706, the electronic device 100 maintains the orientation of the black-tipped pencil 710a in order to match. Figure 7BL The corresponding set of drawing tools within the first drawing palette 704 in the diagram can indicate the orientation of 704c. For example... Figure 7BL As shown, in response to the detection of the completion of drag input 754, electronic device 100 replaces the first drawing palette 704 with a first appearance. Figure 7BK The drawing tool indicator 709 is included. The first drawing palette 704 includes a set of drawing tool indications 704c, which indicate that the currently selected drawing tool is a black pencil with a pointed tip.

[0222] Figures 7BM to 7CF The illustration shows an electronic device 100 displaying and manipulating multiple drawing palettes within a corresponding drawing application interface. For example... Figure 7BMAs shown, the electronic device 100 simultaneously displays a first application interface 702 including a first drawing palette 704 with a first appearance and a second application interface 758 including a second drawing palette 762 with a first appearance. The first application interface 702 includes content 767 (e.g., a triangle). The second drawing palette 762 includes various content manipulation indicators 762a-762g, which in some embodiments correspond to content manipulation indicators 704a-704g within the first drawing palette 704, respectively. In some embodiments, the first application interface 702 and the second application interface 758 correspond to different application windows of the same application (e.g., a notepad application). In some embodiments, the first application interface 702 is associated with a first application, which is different from the second application associated with the second application interface 758. Figure 7BM As further shown, the first application interface 702 is associated with a first width 764, which is equal to or substantially equal to the second width 766 associated with the second application interface 758. Based on this spatial relationship, as... Figures 7BM to 7BU As shown, the first drawing palette 704 and the second drawing palette 762 can be moved within the first application interface 702 and the second application interface 758, respectively.

[0223] like Figure 7BN As shown, the electronic device 100 detects a drag input 768, which requests to move the first drawing palette 704 to the top edge of the first application interface 702. Figure 7BO and Figure 7BP As shown, in response to the detection of drag input 768, but before the completion of drag input 768, the electronic device 100 replaces the first drawing palette 704 with a drawing tool indicator 709. Details about this type of transition are provided above. In response to the detection of the completion of drag input 768, the electronic device 100 displays the first drawing palette 704 with a first appearance along the top edge of the first application interface 702 (in... Figure 7BQ middle).

[0224] like Figure 7BR As shown, the electronic device 100 detects a drag input 770, which requests the second drawing palette 762 to be moved to the upper right corner of the second application interface 758. Figure 7BS and Figure 7BT As shown, in response to detecting drag input 770 but before the drag input 770 is completed, the electronic device 100 replaces the second drawing palette 762 with a second drawing tool indicator 772. Figure 7BU In response to the detection of completion of drag input 770, the electronic device 100 displays a second drawing palette 762 in the upper right corner, facing upwards. The second drawing palette 762 has... Figure 7BRThe second appearance of the second drawing palette 762 is different from the first appearance. That is, the second appearance is smaller than the first appearance and includes fewer energy representations than the first appearance.

[0225] like Figure 7BV As shown, the electronic device 100 detects drawing input 774 within the second application interface 758. In some embodiments, such as Figure 7BW As shown, when electronic device 100 detects drawing input 774 in the second application interface 758, electronic device 100 deems emphasis (e.g., fades in / out) on the first drawing palette 704 relative to the content 767 displayed in the first application interface 702. In response to the detection... Figure 7BV The drawing input 774 in the electronic device 100 displays the corresponding mark 775. In some embodiments, such as Figure 7BX As shown, in response to the detection of completion of drawing input 774, electronic device 100 re-emphasizes the first drawing palette 704.

[0226] like Figure 7BY As shown, the electronic device 100 detects drag input 778, which requests a reduction in the first width 764 associated with the first application interface 702 and an increase in the second width 766 associated with the second application interface 758. In response to the detection... Figure 7BY In the drag input 778, the electronic device 100 reduces the first width 764 and increases the second width 766, such as Figure 7BZ As shown.

[0227] Furthermore, because the electronic device 100 reduces the first width 764 to exceed a certain threshold, the electronic device 100 uses Figure 7BZ The toolbar 777, which includes various content manipulation capabilities, replaces the first drawing palette 704 within the first application interface 702. It is worth noting that, unlike the first drawing palette 704, the toolbar 777 is fixed to the corresponding edge 701a of the first application interface 702. Therefore, as... Figure 7CA and Figure 7CB As shown, in response to detecting a drag input 780 pointing to the toolbar 777, the electronic device 100 does not move the toolbar 777 or the content manipulation indication therein.

[0228] On the other hand, the second width 766 associated with the second application interface 758 has not yet decreased to below a certain threshold. Therefore, the electronic device 100 maintains the second drawing palette 762, as... Figure 7BZ As shown. Figures 7CC to 7CF As shown, the electronic device 100 detects a drag input 782, which requests the second drawing palette 762 to be moved to the bottom edge of the second application interface 758. In response to the detection... Figure 7CCIn the drag input 782, the electronic device 100 displays the second drawing tool indicator 772 (in Figure 7CD and Figure 7CE (in the middle), and finally displays a second drawing palette 762 with the first appearance along the bottom edge of the second application interface 758 (in Figure 7CF middle).

[0229] Figures 8A to 8AL These are examples of user interfaces for invoking and utilizing a screenshot editing interface, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 11A to 11C The process in the process. Although some examples in the following examples will refer to input on a touchscreen display (where a touch-sensitive surface and a display are combined, such as on touchscreen 112), in some embodiments, electronic device 100 detects input on a touch-sensitive surface 651 separate from display 650, such as Figure 6B As shown.

[0230] like Figure 8A As shown, the electronic device 100 displays a user interface having content 802 including text 806, a back indicator 803a, a forward indicator 803b, a share indicator 804a, a bookmark indicator 804b, and a tab indicator 804c. Those skilled in the art will understand that content 802 may include any type and / or kind of content, such as text, images, different indicators, etc.

[0231] like Figure 8B As shown, electronic device 100 detects input 808. In some embodiments, input 808 corresponds to a drag input or flick input detected on the touch-sensitive surface of electronic device 100. In some embodiments, input 808 corresponds to mouse drag input. In response to detection Figure 8B Input 808 in the electronic device 100, display interface 810 (in Figure 8C Interface 810 includes multiple application representations 810a corresponding to multiple applications. Interface 810 also includes a screenshot capture capability representation 810b. For example, in some embodiments, interface 810 corresponds to a taskbar including corresponding capability representations, which, when selected, cause the device to display a user interface corresponding to the application of the selected capability representation. As another example, in some embodiments, interface 810 corresponds to a control center including corresponding capability representations for performing various functions, such as changing the attributes of the electronic device (e.g., changing the screen brightness level, turning airplane mode on / off), and invoking features of the electronic device (e.g., launching the camera application, initiating screen mirroring)).

[0232] like Figure 8DAs shown, electronic device 100 detects a first screenshot capture input 811 pointing to screenshot capture enable display 810b. The first screenshot capture input 811 is a first input type. In response to detecting the first screenshot capture input 811, electronic device 100 captures content 802 as a screenshot image and displays a screenshot editing interface 812 including the screenshot image 813 (in...). Figure 8E (in the middle). The screenshot editing interface 812 also includes various display representations 814-816, 818, 820, 822a, 822b, 824, and 826 for editing (e.g., annotating) and manipulating the screenshot image 813. The screenshot editing interface 812 also includes a drawing tool indicator 828 that indicates the currently selected drawing tool is a black pencil 830 with a pointed tip.

[0233] According to some embodiments, in response to the activation of the power indicator 814 (e.g., completion of power indicator) (e.g., selection via touch), the electronic device 100 displays a completion interface that enables saving or discarding (e.g., deleting) the screenshot image 813, such as Figures 8P to 8R As shown.

[0234] According to some embodiments, in response to the activation of the power indicator 815 (e.g., a crop power indicator) (e.g., selection via touch), the electronic device 100 enters a cropping mode and enables cropping of the screenshot image 813. For example, the electronic device 100 displays a movable cropping handle around the corner of the screenshot image 813.

[0235] According to some embodiments, in response to the activation of the power indicator 816 (e.g., displaying all power indicators) (e.g., selection via touch), the electronic device 100 displays additional content within the screenshot image 813 that was not displayed on the display when the first screenshot capture input 811 was detected. (Refer to below) Figure 8X , Figure 8Y and Figures 8AI to 8AL An example of this function is provided.

[0236] According to some embodiments, in response to the activation of the power indicator 818 (e.g., an opacity level power indicator) (e.g., by touch or by dragging the opacity value indicator 818a), the electronic device 100 changes the opacity of the filter layer overlaid on the screenshot image 813. (Refer to below) Figures 8H to 8J , Figure 8Z and Figure 8AA An example of this function is provided.

[0237] According to some embodiments, in response to the activation of the power indicator 820 (e.g., sharing the power indicator) (e.g., selection via touch), the electronic device 100 displays a transmission interface overlaid on a screenshot editing interface 812, which is provided for sharing a screenshot image 813 via one or more communication methods (such as email, SMS, etc.) and / or for performing one of a variety of operations on a webpage, such as copying, printing, etc. References below... Figures 8K to 8O An example of this function is provided.

[0238] According to some embodiments, in response to the activation of a power indicator 822a (e.g., deactivating the power indicator) (e.g., selection via touch), the electronic device 100 restores one or more previously modified screenshot images 813 to their original state. According to some embodiments, in response to the activation of a power indicator 822b (e.g., restoring the power indicator) (e.g., selection via touch), the electronic device 100 reapplies one or more previously restored modifications to the screenshot images 813.

[0239] According to some embodiments, in response to the activation of one of the power indicators 824 (e.g., the group of drawing tool power indicators) (e.g., selection by touch), the electronic device 100 sets the currently selected drawing tool and, in various cases, changes the currently selected tool 830. According to some embodiments, in response to the activation of one of the power indicators 826 (e.g., a color tank) (e.g., selection by touch), the electronic device 100 sets the color of the currently selected drawing tool and, in various cases, changes the color of the currently selected tool 830 (e.g., changes the tool tip).

[0240] like Figure 8F As shown, electronic device 100 detects annotation input 832 that circles the word "Peleus" within the screenshot image 813. In response to the detection... Figure 8F When annotation 832 is entered, electronic device 100 displays the corresponding annotation 834 (in...). Figure 8G (In Chinese). The characteristics of note 834 are based on the currently selected drawing tool 824 (e.g., pencil), including its color 826 (e.g., black).

[0241] like Figure 8H As shown, the electronic device 100 detects a drag input 836 pointing to the opacity value indicator 818a of the opacity level indicator 818. The drag input 836 moves the opacity value indicator 818a to the corresponding opacity value, such as... Figure 8IAs shown. In some embodiments, in response to detecting a tap input pointing to a corresponding opacity value, the electronic device 100 moves the opacity value indicator 818a to the corresponding opacity value.

[0242] In some embodiments, in response to detection Figure 8H By dragging input 836, the electronic device changes the opacity of the first filter layer 838 overlaid on the screenshot image 813 to a corresponding opacity value, such as... Figure 8I As shown. It is worth noting that the electronic device 100 overlays a first filter layer 838 onto the screenshot image 813 and overlays annotation 834 onto the first filter layer 838, as... Figure 8I As shown.

[0243] In some embodiments, in response to detection Figure 8H By dragging input 836, the electronic device changes the opacity of the second filter layer 840 overlaid on the screenshot image 813 and annotation 834 to a corresponding opacity value, such as... Figure 8J As shown. It is worth noting that the electronic device 100 overlays the second filter layer 840 onto the screenshot image 813 and annotation 834, as... Figure 8J As shown.

[0244] like Figure 8K As shown, the electronic device 100 detects input 842 pointing to the shared power indicator 820. In response to the detection... Figure 8K Input 842, electronic device 100 displays transmission interface 844 overlaid on screenshot editing interface 812 (in Figure 8L (in the middle). The transmission interface 844 includes: a local sharing enablement 844a, which is provided for sharing a screenshot image 813 via a local interface (e.g., Bluetooth, NFC, WiFi, etc.); remote sharing enablement 844b-844e, which are provided for sharing a screenshot image 813 via a corresponding communication method (e.g., SMS, email, cloud storage, etc.); and operation enablement 844f-844i, which are provided for performing corresponding operations on the screenshot image 813.

[0245] like Figure 8M As shown, electronic device 100 detects input 846 pointing to the power indicator 844i (e.g., power indicator saved to file). In response to detection Figure 8M Input 846 in the middle, and the electronic device 100 displays the first save interface 848 (in Figure 8N (In the middle). The first save interface 848 provides various save locations for the screenshot image 813, the second filter layer 840, and the annotation 834. For example... Figure 8OAs shown, the electronic device 100 detects a subsequent input 850 requesting that the screenshot image 813 be saved to the "My Images" folder. Therefore, the electronic device 100 saves the screenshot image 813 to the "My Images" folder and stops displaying the first save interface 848 (in...). Figure 8P (in some embodiments, in response to detecting subsequent input 850, the electronic device 100 saves the screenshot image 813 and the second filter layer 840 as individually editable.)

[0246] like Figure 8P As shown, the electronic device 100 detects input 852 pointing to the completion indicator 814. In response to the detection... Figure 8P Input 852, electronic device 100 displays second save interface 853 (in Figure 8Q (In the middle). The second save interface 853 includes a "Save to Photo" option and a "Delete Screenshot" option. For example... Figure 8R As shown, the electronic device 100 detects subsequent input 854 pointing to the "Save to Photo" enable signal. Therefore, the electronic device 100 saves the screenshot image 813 to the "Photo" area and stops displaying the second save interface 853. In some embodiments, in response to detecting subsequent input 854, the electronic device 100 stores the screenshot image 813 and the second filter layer 840 as a flattened image.

[0247] Figures 8S to 8Y This illustrates updating a screenshot image in response to detecting a second input type of screenshot capture input to include additional content not displayed on the display when the electronic device 100 captured the screenshot image. In some embodiments, the electronic device 100 updates the captured screenshot image in response to detecting a first input type of screenshot capture input, such as... Figure 8D As shown. (Refer to...) Figure 8S The electronic device 100 displays content 802, which includes a partial list 856a of "Top Ten Cities to Visit in Europe". It is noteworthy that the remaining parts of list 856b (e.g., cities 7-10) are not included. Figure 8S It is displayed in the middle, but will become visible in response to scrolling down on scroll bar 857.

[0248] like Figure 8T As shown, the electronic device 100 detects second screenshot capture inputs 858 and 860 respectively pointed to the main button 204 and the press button 206. In general, the second screenshot capture inputs 858 and 860 are related to... Figure 8DThe first screenshot captures a second input type that is different from the first input type associated with input 811. Therefore, in some embodiments, the second input type corresponds to hardware input, such as pressing a hardware button (e.g., main button 204 and push-down button 206), while the first input type corresponds to touch input or mouse input pointing to a specific indication (e.g., ...). Figure 8D The middle finger indicates the first screenshot capture input 811 of the screen capture display 810b.

[0249] In response to detection Figure 8T The second screenshot capture inputs 858 and 860, and the electronic device 100 displays a thumbnail representation 862 of the first screenshot image 866 overlaid on the content 802 (in Figure 8U (In some embodiments, the electronic device 100 adjusts the size of the thumbnail representation 862 according to the size of the display of the electronic device 100. For example, the first screenshot image 866 is shrunk to generate the thumbnail representation 862 based on a predefined size, a predefined aspect ratio, and / or a predefined resolution.)

[0250] like Figure 8V As shown, electronic device 100 detects a first input 864 pointing to a thumbnail representation 862. In response to the detection... Figure 8V The first input 864 in the electronic device 100 displays a screenshot editing interface 812 including a first screenshot image 866 (e.g., as referenced above). Figure 8E The (in) Figure 8W (in Chinese). It is worth noting that, with Figure 8S The content displayed in the first screenshot image 866 is the same as that in 802. It includes a portion of list 856a but excludes the remainder of list 856b. In some embodiments, such as... Figure 8W As shown, the electronic device 100 displays a first drawing palette 704 with a first appearance. The first drawing palette 704 is movable within a screenshot editing interface 812, as described below. Figures 8AB to 8AE As stated above.

[0251] like Figure 8X As shown, electronic device 100 detects input 868 pointing to power indicator 816 (e.g., displaying all power indicators). In response to detection... Figure 8X Input 868, electronic device 100 displays second screenshot image 870 (in Figure 8Y (in the middle). Compared to the first screenshot image 866, the second screenshot image 870 includes additional content 856b corresponding to the rest of the list.

[0252] like Figure 8ZAs shown, the electronic device 100 detects input 872 pointing to the opacity value indicator 818a of the opacity level indicator 818. Input 872 moves the opacity value indicator 818a to the corresponding opacity value, such as... Figure 8AA As shown. In response to detection Figure 8Z Input 872, the electronic device 100 changes the opacity of the filter layer 874 overlaid on the second screenshot image 870 to a corresponding opacity value (in Figure 8AA middle).

[0253] like Figure 8AB As shown, the electronic device 100 detects a drag input 876 that requests to move the first drawing palette 704 along the right edge of the screenshot editing interface 812. The drag input 876 advances beyond a threshold line 877, which is a distance 877a from the third edge 701c. The threshold line 877 and the corresponding distance 877a are shown for illustrative purposes only.

[0254] In response to detection Figure 8AB In the drag input 876, the electronic device 100 replaces the first drawing palette 704 with the drawing tool indicator 878 (in Figure 8AC (In the middle). The drawing tool indicator 878 includes a black pencil tip 879 to indicate that the currently selected drawing tool is a black pencil tip. (As shown) Figure 8AC As shown, because the drag input 768 has not yet crossed the threshold line 877, the electronic device 100 displays the pointed black pencil 879 as facing upwards for matching. Figure 8AB The first drawing palette 704 in the diagram contains a set of drawing tools that can indicate the orientation of 704c.

[0255] exist Figure 8AD In response to detecting drag input 768 crossing... Figure 8AD At threshold line 877, electronic device 100 rotates the black-tipped pencil 879 to face left. Electronic device 100 rotates the black-tipped pencil 879 in this manner so that it moves along... Figure 8AE The third edge 701c in the first drawing palette 704 is matched with a set of drawing tools that indicate the orientation of 704c.

[0256] In response to the detection of completion of drag input 768, electronic device 100 along Figure 8AE The third edge 701c replaces the drawing tool indicator 878 with a first drawing palette 704 having a first appearance. Figure 8AB Compared to the first drawing palette 704 along the bottom edge of the center... Figure 8AE The first drawing palette 704 in the middle is rotated so that it is along (e.g., substantially parallel to) the third edge 701c.

[0257] like Figure 8AF As shown, electronic device 100 detects input 880 for the underlined annotation of the third city "London, England". Therefore, in Figure 8AG In the middle, the electronic device 100 displays the corresponding note 882 for "London, England" underlined, where note 882 reflects that the currently selected tool is a black pencil with a pointed tip.

[0258] Figure 8AH to Figure 8AL An updated screenshot image is illustrated according to some embodiments to include additional content not displayed on the display when corresponding screenshot capture input is detected. Figure 8AH As shown, the electronic device 100 displays a screenshot editing interface 812 including a first screenshot image 883, which includes a first list of "Cities in California" 884a.

[0259] like Figure 8AI As shown, the electronic device 100 detects input 885 pointing to the full power indicator 816. In response to the detection... Figure 8AI The input 885 is in Figure 8AJ In the screenshot editing interface 812, the electronic device 100 displays a second screenshot image 886, which includes a first list of a first group of cities 884a displayed in the first screenshot image 883 and a second list of a second group of cities 884b not displayed in the first screenshot image 883. Additionally, the electronic device 100 displays a scrolling interface 887. The scrolling interface 887 includes an optional content level indicator 887a, which allows the amount of content displayed by the electronic device 100 in the screenshot editing interface 812 to be changed (e.g., the number of listed cities).

[0260] like Figure 8AK As shown, the electronic device 100 displays an input 888 pointing to the content level indicator 887a. That is, input 888 moves the content level indicator 887a to the left (e.g., towards the "-" indicator) to request additional content to be displayed within the screenshot editing interface 812. In response to detection... Figure 8AK Upon input 888, the electronic device 100 displays a third screenshot image 890, which includes a first list of cities 884a in the first group, a second list of cities 884b in the second group, and... Figure 8AH The first screenshot image in the image is 883 or within the image. Figure 8AL The third list of cities 884c in the second screenshot image 886 is not displayed.

[0261] Figures 9A to 9PThese are examples of capturing screenshot images based on detected stylus input, according to some embodiments. The user interface in these figures is used to illustrate the process described below, including... Figures 11A to 11C The process in the process. Although some examples in the following examples will refer to input on a touchscreen display (where a touch-sensitive surface and a display are combined, such as on touchscreen 112), in some embodiments, electronic device 100 detects input on a touch-sensitive surface 651 separate from display 650, such as Figure 6B As shown.

[0262] like Figure 9A As shown, the electronic device 100 displays content 902 including a lion 906 (e.g., an image of a lion), as well as various power indicators, including a back power indicator 903a, a forward power indicator 903a, a share power indicator 904a, a bookmark power indicator 904b, and a tab power indicator 904c.

[0263] like Figure 9B As shown, the electronic device 100 detects a first stylus movement 908 of a stylus 203 held by a user's hand 748 on its touch-sensitive surface. The first stylus movement 908 moves away from the corresponding outer edge 701b of the touch-sensitive surface toward a release point within the touch-sensitive surface, as indicated by the end of the arrow. Figure 9B As shown, in some embodiments, the first stylus movement 908 originates outside a first region 910 at a first threshold distance 910a from the edges 701a-701d of the electronic device 100. For example... Figure 9B As further shown, in some embodiments, the first stylus movement 908 ends at a release point within a second region 912 (e.g., across the second region) of a second threshold distance 912a from the edges 701a-701d of the electronic device 100. The first region 910, the first threshold distance 910a, the second region 912, and the second threshold distance 912a are shown for illustrative purposes only.

[0264] like Figure 9C As shown, the first stylus movement 908 advances to a position closer to the release point. In response to detecting that the first stylus movement 908 originates from... Figure 9B Outside the first area 910, the electronic device 100 changes the display (in Figure 9C(In the middle). That is, the electronic device 100 replaces the content 902 with a screenshot preview interface 913. The screenshot preview interface 913 includes an outer region 914 (e.g., a plain gray area) defining the current position of the first stylus movement 908, and a screenshot preview area 916 associated with (e.g., continuously associated with) the outer region 914. The screenshot preview area 916 provides a preview of the screenshot image 918. On the other hand, in response to detecting that the stylus movement does not originate outside the first region 910, the electronic device 100 does not replace the content 902 with the screenshot preview interface 913.

[0265] As the first stylus moves 908 toward the second area 912, the electronic device 100 shrinks the screen preview area 916. For example, as... Figure 9D As shown, when with Figure 9C In contrast, when the first stylus moves 908 closer to the release point, the electronic device 100 correspondingly shrinks the screen preview area 916 and enlarges the outer area 914 (in... Figure 9D middle).

[0266] In response to determining that the release point of the first stylus movement 908 is within the second region 912, the electronic device 100 replaces the screenshot preview interface 913 with a screenshot editing interface 917. Figure 9E (in the middle). The screenshot editing interface 917 includes a screenshot image 918, display indicators 814-816, 818, 820, 822a, 822b, 824 and 826, and drawing tool indicators 828.

[0267] like Figure 9F As shown, when the electronic device 100 is displaying content 902, the electronic device 100 detects a second stylus movement 920 originating outside the first area 910. Figure 9G As shown, when the second stylus movement 920 moves closer to the second area 912, the electronic device 100 replaces the content 902 with a screenshot preview interface 913 that includes the outer area 914 and the screenshot preview area 916. However, because the release point of the second stylus movement 920 is not within the second area 912, the electronic device 100 does not display the screenshot editing interface 917. Instead, as... Figure 9H As shown, in response to the detection of the completion of the second stylus movement 920, the electronic device 100 displays content 902 without capturing a screenshot image.

[0268] like Figure 9I As shown, the electronic device 100 detects a first drag input 922 that is not a stylus drag input. For example, the electronic device 100 detects a first drag input 922 when a finger drags on the touch-sensitive surface of the electronic device 100. Figures 9I to 9K As shown, the first drag input 922 originates from Figure 9I The first drag input 922 is completed within the first region 910 and the second region 912. However, because the first drag input 922 is not a stylus drag input, in response to detecting the completion of the first drag input 922 within the second region 912, the electronic device 100 does not display the screenshot preview interface 913 and does not display the screenshot editing interface 917 as the first drag input 922 moves toward the end point. Instead, as Figure 9L As shown, the electronic device 100 displays content 902 in response to detecting the completion of the first drag input 922.

[0269] like Figure 9M As shown, the electronic device 100 detects a second drag input 924 that is not a stylus drag input. In some embodiments, the second drag input 924 is oriented inward toward the user interface from the bottom edge of the touch-sensitive surface (e.g., the fourth edge 701d).

[0270] In some embodiments, such as Figure 9M and Figure 9N As shown, in response to detecting a second drag input 924 crossing into the area between the first region 910 and the second region 912, the electronic device 100 displays an interface 810 (e.g., a taskbar or control center). The interface 810 includes multiple application representations 810a corresponding to multiple applications. The interface 810 also includes a screenshot capture capability representation 810b.

[0271] like Figure 9O As shown, electronic device 100 detects input 926 pointing to the screen capture display 810b. In response to the detection... Figure 9O Input 926 in the middle, electronic device 100 displays screen screenshot editing interface 918 (in Figure 9P middle).

[0272] In some embodiments, the electronic device 100 detects a third stylus movement of a stylus 203 held by a user's hand 748 on its touch-sensitive surface. The third stylus movement originates at a specific corner threshold distance from the touch-sensitive surface, toward a release point within the touch-sensitive surface. In some embodiments, the specific corner of the touch-sensitive surface corresponds to the location where the first edge 701a intersects the second edge 710b. Therefore, the threshold distance includes a vertical distance threshold from the first edge 701a and a horizontal distance threshold from the second edge 701b. Those skilled in the art will understand that the third stylus movement may originate at any of the four corner threshold distances. In some embodiments, the corner swipe gesture is limited to a subset of corners. For example, in some embodiments, when movement is detected at one of the top two corners, the movement triggers a corresponding response, while when movement is detected at one of the bottom two corners, the movement does not trigger a corresponding response. As another example, in some embodiments, when movement is detected at one of the bottom two corners, the movement triggers a corresponding response, while when movement is detected at one of the top two corners, the movement does not trigger a corresponding response. In some embodiments, corner swipe gestures are limited to gestures performed with a stylus rather than a finger (e.g., corresponding finger movement detected from a corner of the touchscreen display will not trigger a response). In some embodiments, corner swipe gestures are limited to gestures performed with a finger rather than a stylus (e.g., corresponding stylus movement detected from a corner of the touchscreen display will not trigger a response).

[0273] In some embodiments, as the third stylus moves closer to the release point, the electronic device 100 changes the display. That is, the electronic device 100 replaces content 902 with a screenshot preview interface 913 that includes an outer area 914 and a screenshot preview area 916.

[0274] In some embodiments, when the third stylus moves forward into the second region 912, the electronic device 100 shrinks the screenshot preview area 916. In response to the detection of the release of the third stylus from the second region 912, the electronic device 100 replaces the screenshot preview interface 913 with a screenshot editing interface 917.

[0275] In some embodiments, the electronic device 100 displays a screenshot capture menu instead of displaying a screenshot editing interface 917 based on stylus movement originating from a corner. In some embodiments, the electronic device 100 detects a fourth stylus movement of a stylus 203 held by a user's hand 748 on its touch-sensitive surface. The fourth stylus movement originates at a specific corner threshold distance from the touch-sensitive surface and moves toward a release point within the touch-sensitive surface.

[0276] In some embodiments, as the fourth stylus moves closer to the release point, the electronic device 100 changes the display. In some embodiments, the electronic device 100 abandons displaying the screenshot preview interface 913 and instead displays a stylus movement indicator. The stylus movement indicator indicates the distance the fourth stylus has moved from crossing into the second region 912. As described below, in response to detecting that the fourth stylus has moved into the second region 912, the electronic device 100 displays a screenshot capture menu 938. That is, in some embodiments, the stylus movement indicator includes approximately half the amount of color corresponding to the stylus movement indicator, because the fourth stylus has moved approximately halfway from its starting point to crossing into the second region 912. Those skilled in the art will understand that the stylus movement indicator may indicate the distance from crossing into the second region 912 in different ways.

[0277] In some embodiments, the electronic device 100 detects a fourth stylus movement crossing into the second region 912. Therefore, the electronic device 100 replaces the stylus movement indicator with a screenshot capture menu. In some embodiments, the screenshot capture menu includes a capture screenshot capability representation and an edit screenshot capability representation. In response to the detection of the fourth stylus movement being released within the second region 912, the electronic device 100 maintains the display of the screenshot capture menu.

[0278] In some embodiments, in response to detecting input pointing to a capture screenshot representation, the electronic device 100 captures a screenshot image without displaying the screenshot image or a screenshot editing interface. In some embodiments, the electronic device 100 pastes the captured screenshot in response to detecting subsequent paste input.

[0279] On the other hand, in some embodiments, in response to detecting input pointing to an editable screenshot interface, electronic device 100 displays a screenshot editing interface 917.

[0280] Figures 10A to 10D This is a flowchart of a method 1000 for repositioning a drawing palette according to some embodiments. In some embodiments, method 1000 is performed in an electronic device having one or more processors, non-transitory memory, input devices, and display devices (e.g., Figure 3 Electronic devices 300 or Figure 1A and Figures 7A to 7CF The operation is performed at the portable multifunction device 100. Some operations in method 1000 may be optionally combined, and / or the order of some operations may be optionally changed.

[0281] Electronic devices that reposition a drawing palette in response to a request to move it, and alter its appearance in various situations (e.g., resizing or reducing the number of content manipulation capabilities), improve operability. For example, repositioning the drawing palette results in a larger usable display area for drawing operations and other content modification operations. Furthermore, the electronic device is configured to perform one of multiple repositioning operations in response to a single input based on the target location requested by that input. Therefore, clutter in the user interface is reduced by avoiding the display of multiple different capabilities corresponding to multiple repositioning operations.

[0282] Compared to Figure 10A An electronic device displays (1002) a first drawing palette at a first location within a first application interface via a display device, wherein the first drawing palette has a first appearance at the first location, in which a representation of the currently selected drawing tool is displayed simultaneously with one or more representations of other available drawing tools. For example, the first application interface corresponds to a drawing application. As another example, the first drawing palette is anchored to a corner or edge of the first application interface.

[0283] In some embodiments, the electronic device displays a first drawing palette having a first appearance and positioned along the edge of a first application interface (e.g., substantially parallel to the edge of the first application interface). For example, refer to... Figure 7A The electronic device 100 displays a first drawing palette 704 along the fourth edge 701d of the first application interface 702. Continuing this example, compared to the second appearance of the first drawing palette 704, the first appearance of the first drawing palette 704 includes more content manipulation indications (e.g., 704a-704g) and, in various embodiments, is larger in size than the second appearance of the first drawing palette 704.

[0284] In some embodiments, the first appearance corresponds to (1004) a first drawing palette in a first extended state. When the first drawing palette is in the first extended state, the electronic device displays multiple content manipulation indications, thereby avoiding the need for input to retrieve multiple content manipulation indications. Therefore, the electronic device experiences less wear and tear and uses less battery and processing resources. As an example, refer to... Figure 7A The electronic device 100 displays a first drawing palette 704 in a first extended state, which includes multiple content manipulation display panels 704a-704g.

[0285] The electronic device detects (1006) a first input corresponding to a request to move the first drawing palette within the first application interface. For example, the first input corresponds to a drag input, such as... Figures 7B to 7DThe middle finger is the drag input 708 of the first drawing palette 704. As another example, the first input corresponds to a flick input, such as... Figure 7AC and Figure 7AD The middle finger flicks the first drawing palette 704. In some embodiments, the first input corresponds to touch input or stylus input detected on a touch-sensitive surface of the electronic device. As another example, the first input corresponds to mouse input, such as click and drag input.

[0286] In some embodiments, the electronic device displays (1008) a currently selected drawing tool indicator based on a first input. Displaying a representation of the path of the first input provides feedback to the user, ensuring that the drawing palette moves as indicated by the user, thereby reducing the possibility of further user interaction to provide different movements. Reducing user interaction with the electronic device reduces wear and tear on the device and, for battery-powered devices, extends battery life. For example, the drawing tool indicator includes a visual representation of the currently selected drawing tool, and in various embodiments, includes a visual representation of the color of the currently selected drawing tool (e.g., a circle containing a pencil photograph). As another example, the electronic device displays the drawing tool indicator when the first input is within a threshold distance from a corresponding edge or corner of the first application interface. In some embodiments, the electronic device rotates the drawing tool indicator based on a comparison between the corresponding orientation of the first drawing palette at the starting position and the corresponding orientation of the first drawing palette at the ending position.

[0287] As an example, see reference Figure 7C and Figure 7D As the input 708 is dragged toward the first edge 701a, the electronic device 100 displays a drawing tool indicator 709. The drawing tool indicator 709 includes a black-tipped pencil 710a to indicate that the black-tipped pencil is the currently selected drawing tool. As another example, see [reference needed]. Figure 7P and Figure 7Q When the corresponding drag input 723 crosses into the fifth region 716e, the electronic device 100 rotates the drawing tool indicator 709 to match the orientation of the first drawing palette 704 in the fifth region 716e. As yet another example, see [reference needed]. Figures 7AD to 7AF The electronic device 100 rotates the drawing tool indicator 709 to match the orientation of the first drawing palette 704 in the seventh area 716g. Because the first input 729 corresponds to a flick input type, therefore... Figures 7AD to 7AF In the middle, the electronic device 100 gradually rotates the drawing tool indicator 709.

[0288] In response to the detection of a first input: based on the determination that the first input corresponds to a request to move the first drawing palette to a second position within the first application interface, the electronic device displays (1010) a first drawing palette having a first appearance at the second position. For example, the electronic device moves the first drawing palette, which includes multiple content manipulation capabilities, from one edge of the first application interface to another edge of the first application interface. In some embodiments, the first drawing palette at the second position has the same orientation as the first drawing palette at the first position, such as... Figures 7B to 7E As shown. In some embodiments, the first drawing palette at the second position has a different orientation than the first drawing palette at the first position, such as... Figures 70 to 7R and Figures 7AC to 7AG As shown.

[0289] In response to the detection of a first input: based on the determination that the first input corresponds to a request to move the first drawing palette to a third position within the first application interface, different from the second position, the electronic device displays (1012) a first drawing palette with a second appearance different from the first appearance at the third position, wherein when the first drawing palette has the second appearance, the representation of the currently selected drawing tool is displayed in the first drawing palette, but representations of other drawing tools are not displayed in the first drawing palette. For example, the electronic device moves the first drawing palette, which includes multiple content manipulation capabilities, from one edge of the first application interface to a corner of the first application interface.

[0290] As an example, see reference Figures 7AN to 7AP In response to determining that drag input 734 corresponds to a request to move the first drawing palette 704 from the second region 716b to the first region 716a, the electronic device 100 displays the first drawing palette 704 with a second appearance in the first region 716a (in Figure 7AP (Middle). The first drawing palette 704, having a second appearance, includes an indicator 735a for the currently selected drawing tool, but does not include representations of other drawing tools. In some embodiments, such as Figures 7AN to 7AP As shown, Figure 7AN The orientation (e.g., facing left) of the first drawing palette 704 having a first appearance is consistent with... Figure 7AP The orientation of the first drawing palette 704 having a second appearance is matched. In embodiments where the source and destination locations do not share edges (e.g., seventh region 716g and third region 716c), the orientation of the first drawing palette having a first appearance may not match the orientation of the first drawing palette having a second appearance.

[0291] refer to Figure 10BIn some embodiments, in response to detecting a second input corresponding to a drawing operation on the first application interface, the electronic device stops (1014) displaying the first drawing palette. Stopping the display of the first drawing palette increases the available display area of ​​the first application interface.

[0292] In some embodiments, the electronic device simultaneously displays (1016) a second application interface and a first application interface via a display device, wherein the second application interface includes a second drawing palette. Based on the determination that the first application interface has a corresponding size characteristic that does not meet a threshold size, the electronic device sets (1016) the first drawing palette to be immovable within the first application interface and sets the second drawing palette to be movable within the second application interface. Furthermore, the electronic device detects (1016) a second input via an input device. In response to the detection of the second input: based on the determination that the second input corresponds to a request to move the first drawing palette to a fourth position within the first application interface, the electronic device maintains (1016) the current position of the first drawing palette; and based on the determination that the second input corresponds to a request to move the second drawing palette to a fifth position within the second application interface, the electronic device moves (1016) the second drawing palette to the fifth position. Maintaining the current position of the first drawing palette instead of moving it reduces the processing power and battery usage of the electronic device. For example, an electronic device may simultaneously display a first scaled-down representation of a first drawing palette within a first scaled-down representation of a first application interface and / or display a second scaled-down representation of a second drawing palette within a second scaled-down representation of a second application interface. As another example, the corresponding size characteristic corresponds to the width of the corresponding application interface, and the corresponding size characteristic satisfies a threshold size when the width is sufficiently large (e.g., the area occupied by the corresponding application interface exceeds 50% of the display area). In some embodiments, the first drawing application interface and the second drawing application interface are application windows of the same application. In some embodiments, the first drawing application interface and the second drawing application interface are application windows of different applications. In some embodiments, the first drawing application interface and the second drawing application interface are continuously associated with each other (e.g., sharing a boundary line) or adjacent to each other.

[0293] As an example, in response to detection Figure 7BY In the drag input 778, the electronic device 100 determines that the first application interface 702 has a corresponding size characteristic that does not meet the threshold size (e.g., a first width 764), and determines that the second application interface 758 has a corresponding size characteristic that meets the threshold size (e.g., a second width 766). Therefore, as Figure 7BZAs shown, the electronic device 100 replaces the first drawing palette 704 with a toolbar 777 in the fixed toolbar area of ​​the first application interface 702, and keeps the second drawing palette 762 in the second application interface 758. Continuing this example, in response to the detection Figure 7CA The middle finger drags the toolbar 777 to input 780, and the electronic device 100 maintains the current position of the toolbar 777 (in...). Figure 7CB (Middle). Continuing this example, in response to detection... Figure 7CC The middle finger drags the second drawing palette 762, and the electronic device 100 moves the second drawing palette 762 accordingly. Figures 7CD to 7CF middle).

[0294] In some embodiments, the electronic device detects (1018) a third input pointing to the first application interface via an input device, and in response to detecting the third input, deemphasizes (1018) the second drawing palette relative to the content displayed on the canvas of the second application interface. Deemphasing the second drawing palette indicates that the first drawing palette and the first application interface are currently active, thereby enhancing the operability of the electronic device and making the electronic device more efficient. For example, the number of erroneous user inputs is reduced, which also reduces power consumption and extends the battery life of the electronic device by enabling users to use it more quickly and efficiently.

[0295] For example, the third input corresponds to a drawing operation relative to the first application interface, such as selecting a display representation of specific content and / or drawing marks on the canvas of the first application interface. As another example, de-emphasis corresponds to reducing focus, such as decreasing brightness (e.g., darkening), increasing opacity, etc. In some embodiments, in response to detecting a fourth input directed to the second application interface, the electronic device restores focus on the second drawing palette, and in various embodiments, reduces focus on the first drawing palette relative to the content displayed on the canvas of the first application interface.

[0296] As an example, see reference Figure 7BV Electronic device 100 detects drawing input 774 directed to the second application interface 758. In response to detecting drawing input 774, electronic device 100 deems the first drawing palette 704 less emphasized relative to the content 767 within the first application interface 702, such as... Figure 7BW As shown. In response to the detection of completion of drawing input 774, electronic device 100 restores (e.g., re-emphasizes) the first drawing palette 704.

[0297] In some embodiments, based on determining that the first input corresponds to a first input type, the electronic device displays (1020) a first drawing palette having a first appearance at a second position based on a first position type and the first input type. By displaying the first drawing palette at the second position based on the first input type, the electronic device avoids moving the first drawing palette to the second position in response to an input that does not correspond to the first input type. Therefore, the electronic device utilizes less processing power and battery usage. For example, the first input type corresponds to a drag input performed within a specific predefined area of ​​the first application interface (e.g., sufficiently close to the edge or corner of the first application interface). As another example, the first input type corresponds to a flick input in a direction (e.g., toward the edge) within a specific predefined area, wherein the flick input satisfies a speed threshold.

[0298] As an example, see reference Figure 7W Electronic device 100 detects a drag input 726 pointing to a first drawing palette 704 at a first position along the second edge 701b. In response to determining that the drag input 726 is a drag input type and that the drag input 726 crosses into the fifth region 716e, electronic device 100... Figure 7Z The first drawing palette 704 is displayed at the second position of the third edge 701c. As another example, see [reference]. Figure 7AC Electronic device 100 detects a flick input 729 pointing to a first drawing palette 704 at a first position along the third edge 701c. In response to determining that the flick input 729 is a flick input type and that the flick input 729 meets a speed threshold, electronic device 100... Figure 7AG The first drawing palette 704 is displayed at the second position of the first edge 701a in the middle.

[0299] In some embodiments, determining that the first input corresponds to a request to move the first drawing palette to the second position includes (1022) determining that the second position corresponds to a first position type. By displaying the first drawing palette at the second position based on determining that the second position corresponds to the first position type, the electronic device avoids moving the first drawing palette to the second position in response to an input that is not a request to move to the first position type. Therefore, the electronic device saves processing power and battery usage. For example, the first position type corresponds to a position within a threshold distance from a corresponding edge of the display (e.g., top edge, bottom edge, side edge). As another example, the first position type corresponds to a position within a first threshold distance from a first edge of the display and within a second threshold distance from a second edge of the display, wherein the second edge intersects the first edge and is substantially perpendicular to the first edge. As an example, refer to... Figure 7BThe electronic device 100 determines the drag input 708 request to move the first drawing palette 704 to the first position by determining that the target position is an edge type (e.g., the first edge 701a) rather than a corner type.

[0300] In some embodiments, based on determining that the first input satisfies a first distance threshold (e.g., relative to a second position), the electronic device determines (1024) that the first input corresponds to a request to move the first drawing palette to the second position, and displays (1024) the first drawing palette having a first appearance at the second position. By moving the first drawing palette when the first input satisfies the first distance threshold, the electronic device avoids erroneously moving the first drawing palette in response to inputs not intended to move the first drawing palette (such as inputs that change the currently selected drawing tool within the first drawing palette). Therefore, the efficiency and operability of the electronic device are enhanced, thereby reducing processor utilization and battery usage. In some embodiments, the first input satisfies the first distance threshold when the release position of the first input is within a threshold distance from the corresponding edge. On the other hand, in some embodiments, when the release position of the first input is not within a threshold distance from the corresponding edge, the electronic device moves the first drawing palette back to the position where the first drawing palette was before the first input was detected.

[0301] As an example, see reference Figure 7W Electronic device 100 determines that the drag input 726 crosses a distance threshold 720 corresponding to a distance 720a from the third edge 701c. Therefore, electronic device 100 along... Figure 7Z The third edge 701c displays the first drawing palette 704. As a counterexample, in response to determining... Figure 7I If the drag input 713 does not exceed the distance threshold 706 corresponding to the distance 706a from the first edge 701a, the electronic device 100 does not move the first drawing palette 704 to the first edge 701a, but instead displays the first drawing palette 704 at its previous position (e.g., along the fourth edge 701d). Figure 7L middle).

[0302] In some embodiments, based on determining that a first input meets a speed threshold, the electronic device determines (1026) that the first input corresponds to a request to move a first drawing palette to a second position, and displays (1026) the first drawing palette having a first appearance at the second position. In some embodiments, the electronic device moves the first drawing palette based on a short-duration input (e.g., a flick input) that meets the speed threshold. Therefore, the electronic device avoids utilizing processing power and battery power that would otherwise be used for longer-duration inputs (e.g., drag inputs). For example, the first input corresponds to a flick input that is associated with a direction toward a specific area of ​​the application interface and with a sufficiently high magnitude (e.g., the rate or acceleration of the flick input) in that direction. As an example, refer to... Figure 7AC The electronic device 100 detects the flick input 729 and determines that the flick input 729 has a horizontal component 729a and a vertical component 729b that are jointly associated with the direction toward the seventh region 716g, as indicated by the corresponding explanatory direction line 730. Furthermore, the electronic device 100 determines that the flick input 729 is associated with a sufficiently high magnitude toward the seventh region 716g. Therefore, as... Figures 7AD to 7AG As shown, the electronic device 100 moves the first drawing palette 704 to the seventh region 716g.

[0303] refer to Figure 10CIn some embodiments, when a first drawing palette having a first appearance and a first orientation is displayed at a second position, where the second position corresponds to a first position type, the electronic device detects (1028) a second input via an input device corresponding to a request to move the first drawing palette to a fourth position within a first application interface, where the fourth position corresponds to the first position type. In response to the detection of the second input, based on determining that the fourth position is on the opposite side of the display compared to the second position, the electronic device displays (1028) the first drawing palette having the first appearance and the first orientation at the fourth position. Based on determining that the fourth position is not on the opposite side of the display compared to the second position, the electronic device displays (1028) the first drawing palette having the first appearance and a second orientation different from the first orientation at the fourth position. By changing or maintaining the orientation of the first drawing palette according to the spatial relationship between the initial and target positions of the first drawing palette, the electronic device increases the amount of usable display area that would otherwise be covered (and thus become unusable) by the first drawing palette. For example, the second position corresponds to a first edge of the first application interface, and the second input corresponds to a request to move the first drawing palette to a second edge different from the first edge. As another example, the second orientation of the first drawing palette is rotated approximately 90 degrees compared to the first orientation. As yet another example, the electronic device displays the first drawing palette according to the first orientation when it is located at the top or bottom edge, and according to the second orientation when it is located at a side edge (e.g., the left or right edge). In some embodiments, the electronic device adjusts the size of the first drawing palette at a fourth position because the second position of the first drawing palette corresponds to the shorter edge of the first application interface, and the fourth position of the first drawing palette corresponds to the longer edge of the first application interface.

[0304] As an example, see reference Figures 7B to 7E In response to determining a fourth position (e.g., along the first edge 701a) on the opposite side of the display compared to a second position having a first orientation (e.g., the fourth edge 701d), the electronic device 100 displays a first drawing palette 704 having a first appearance and a first orientation at the fourth position (in... Figure 7E (in Chinese). As an example, refer to... Figures 70 to 7R In response to determining that a fourth position (e.g., along the third edge 701c) is not on the opposite side of the display compared to a second position having a first orientation (e.g., the fourth edge 701d), the electronic device 100 displays a first drawing palette 704 at the fourth position having a first appearance and a second orientation different from the first orientation. Figure 7R (In the middle). That is, the second orientation is rotated approximately 90 degrees relative to the first orientation.

[0305] In some embodiments, the second appearance corresponds to (1032) a first drawing palette in a compressed state. The compressed first drawing palette covers less display space, thus providing a larger usable display area for the electronic device. For example, the compressed state corresponds to a shrunken drawing palette. As another example, the electronic device displays the compressed first drawing palette when it is anchored to or near a corner of the first application interface. In some embodiments, the compressed first drawing palette includes less content manipulation capability than the first drawing palette in other states. In some embodiments, the compressed first drawing palette is smaller than the first drawing palette in other states.

[0306] As an example, electronic device 100 displays a first drawing palette 704 in a compressed state (in... Figure 7AP In the middle), the first drawing palette 704 is displayed near the corresponding corner (e.g., the lower right corner). As another example, the electronic device 100 displays the first drawing palette 704 in a compressed state (in the middle). Figure 7BA (in the middle), so that the first drawing palette 704 is displayed near the corresponding corner (e.g., the upper left corner).

[0307] In some embodiments, based on determining that the first input corresponds to a second input type different from the first input type, the electronic device displays (1034) a first drawing palette with a second appearance at a third position based on the first position type and the second input type. By displaying the first drawing palette at the third position based on the second input type, the electronic device avoids moving the first drawing palette to the third position in response to an input that does not correspond to the second input type. Therefore, the electronic device saves processing power and battery usage. For example, the second input type corresponds to a flick or drag towards or to a corner.

[0308] As an example, in response to detecting drag input 734 corresponding to a drag input type toward the corresponding corner (in... Figure 7AN In the middle), the electronic device 100 displays the first drawing palette 704 near the corresponding corner (in Figure 7AP (in the middle). As another example, in response to detecting that drag input 744 corresponds to a drag input type toward the corresponding corner (in Figure 7AX In the middle), the electronic device 100 displays the first drawing palette 704 near the corresponding corner (in Figure 7BA middle).

[0309] In some embodiments, determining that the first input corresponds to a request to move the first drawing palette to a third position includes (1036) determining that the third position corresponds to a second position type different from the first position type. By displaying the first drawing palette at the third position based on determining that the third position corresponds to the second position type, the electronic device avoids moving the first drawing palette to the third position in response to an input that is not a request to move to the second position type. Therefore, the electronic device utilizes less processing power and battery usage. For example, the second position type corresponds to a corner.

[0310] As an example, in response to detecting drag input 734 corresponding to a drag input type toward the corresponding corner (in... Figure 7AN In the middle), the electronic device 100 displays the first drawing palette 704 near the corresponding corner (in Figure 7AP (in the middle). As another example, in response to detecting that drag input 744 corresponds to a drag input type toward the corresponding corner (in Figure 7AX In the middle), the electronic device 100 displays the first drawing palette 704 near the corresponding corner (in Figure 7BA middle).

[0311] In some embodiments, based on determining that the first input satisfies a second distance threshold different from a first distance threshold, the electronic device determines (1038) that the first input corresponds to a request to move the first drawing palette to a third position, and displays (1038) the first drawing palette with a second appearance at the third position. By moving the first drawing palette when the first input satisfies the second distance threshold, the electronic device avoids erroneously moving the first drawing palette in response to inputs not intended to move the first drawing palette (such as inputs that change the currently selected drawing tool within the first drawing palette). Therefore, the efficiency and operability of the electronic device are enhanced, thereby reducing processor utilization and battery usage. For example, the first input satisfies the second distance threshold when the release position of the first input is within a threshold distance from a corner.

[0312] As an example, in response to detection Figure 7AN The drag input 734 in the middle crosses a second distance threshold 718 corresponding to a distance 718a from the first edge 701a (e.g., within the first region 716a), and the electronic device 100 displays a first drawing palette 704 with a second appearance within the first region 716a. Figure 7AP (in the middle). As another example, in response to detection Figure 7AXWhen the drag input 744 in the middle crosses a distance threshold 722 corresponding to a distance 722a from the fourth edge 701d and a distance threshold 720 corresponding to a distance 720a from the third edge 701c (e.g., within the sixth region 716f), the electronic device 100 displays a first drawing palette 704 with a second appearance (in 7BA) within the sixth region 716f.

[0313] refer to Figure 10D In some embodiments, when a first drawing palette in a compressed state is displayed at a third location, the electronic device: detects (1040) touch input pointing to the first drawing palette via an input device; and in response to detecting the touch input, displays (1040) a first drawing palette in a second extended state, different from the first extended state, wherein the first drawing palette in the second extended state includes more drawing tools than the first drawing palette in the compressed state. By including more drawing tools within a single user interface, the electronic device avoids detecting additional input corresponding to the selection of a particular drawing tool. For example, the second extended state corresponds to a preview drawing palette that provides a preview of the available drawing tools and closes in response to the selection of a particular drawing tool. In some embodiments, the first drawing palette in the second extended state (e.g., Figure 7AS The preview drawing palette 738 in the preview drawing palette is different from the first drawing palette in the first extended state (e.g., Figure 7E The full-size drawing palette 704 includes fewer drawing tools. In some embodiments, the electronic device expands the first drawing palette in a compressed state according to the current orientation of the first drawing palette in a compressed state to display the first drawing palette in a second expanded state. In some embodiments, the electronic device displays the first drawing palette in the second expanded state in response to a duration of a threshold amount of detected touch input.

[0314] As an example, in response to detecting touch input 736 pointing to the first drawing palette 704 in a compressed state (in Figure 7AR In the middle), the electronic device 100 displays the first drawing palette 704 in the second extended state (in Figure 7AS (in Chinese). It is worth noting that, in Figure 7AR In this case, because the currently selected drawing tool indicator 735a is oriented to the left (e.g., pointing), the electronic device 100 extends the first drawing palette 704 upward to maintain the orientation, as shown below. Figure 7AS As shown. On the other hand, the electronic device 100 extends the first drawing palette 704 to the right (in Figure 7BD (in the middle), so as to maintain the upward orientation of the currently selected drawing tool indicator 735a (in Figure 7BC middle).

[0315] In some embodiments, the electronic device detects (1042) a second input pointing to a specific content manipulation indicator within a first drawing palette in a second extended state via an input device. In response to detecting the second input, the electronic device sets the current content manipulation settings associated with the first drawing palette according to the specific content manipulation indicator (1042). By enabling selection of a specific content manipulation indicator within the first drawing palette, the electronic device reduces input associated with opening and closing additional user interfaces to select a content manipulation indicator. Therefore, the electronic device consumes less processing and battery resources. In some embodiments, the second input corresponds to a drag input (e.g., finger drag) along the first drawing palette in the second extended state, where the release point of the drag input corresponds to the specific content manipulation indicator. In some embodiments, the second input corresponds to a stylus touch input pointing to a specific content manipulation indicator. For example, the specific content manipulation indicator corresponds to a specific tool, color, operation (e.g., undo / redo), etc.

[0316] As an example, when electronic device 100 is displaying a preview palette 738 (in... Figure 7AT In the middle), the electronic device 100 detects the drag input 740 of the selected gray-tipped pencil tool indicator 738c. Therefore, the electronic device 100 sets the gray-tipped pencil as the currently selected drawing tool and updates the currently selected drawing tool indicator 735a to the gray-tipped pencil (in the middle). Figure 7AU (in the middle). As another example, when electronic device 100 is displaying preview palette 738 (in Figure 7BE In the middle), the electronic device 100 detects a stylus tap input 750 indicating the selection of a black-tipped pencil indicator 738a. Therefore, the electronic device 100 sets the black-tipped pencil as the currently selected drawing tool and updates the currently selected drawing tool indicator 735a to the black-tipped pencil (in the middle). Figure 7BF middle).

[0317] In some embodiments, when the first drawing palette is displayed at a second or third position, the electronic device detects (1044) a tap input pointing to the first drawing palette via an input device. In response to detecting the tap input, the electronic device moves (1044) the first drawing palette to the first position. Moving the first drawing palette back to the first position in response to a tap input, rather than a drag or flick input, reduces processor and battery utilization due to the shorter duration of the tap input. In some embodiments, the tap input corresponds to either a single click or a double click. As an example, when the first drawing palette is in a first extended state, the tap input is a single click, such as... Figure 7FThe middle finger taps the first drawing palette 704 in the second position (input 711). Therefore, the electronic device 100 moves the first drawing palette 704 back to the first position, as... Figures 7G to 7I As shown. As another example, in response to detecting a pointer to a first drawing palette that is in a compressed state (e.g., ...), Figure 7AP Upon tapping the first drawing palette (704) in the device, the electronic device moves the first drawing palette to its previous position.

[0318] Figures 11A to 11C This is a flowchart of a method 1100 for invoking and utilizing a screenshot editing interface according to some embodiments. In some embodiments, method 1100 is implemented in an electronic device having one or more processors, non-transitory memory, input devices, and display devices (e.g., Figure 3 Electronic devices 300 or Figure 1A , Figures 8A to 8AL and / or Figures 9A to 9P The operation is performed at the portable multifunction device 100. Some operations in method 1100 may be optionally combined, and / or the order of some operations may be optionally changed.

[0319] Method 1100 includes: in response to detecting a single screenshot capture input, displaying a screenshot editing interface for editing a screenshot image or a thumbnail representation of a screenshot image based on the input type of the screenshot capture input. Therefore, Method 1100 provides additional control options without cluttering the user interface with additional displayed controls. Furthermore, Method 1100 provides an intuitive way to edit screenshot images. Method 1100 reduces the cognitive burden on users when editing screenshot images, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to edit screenshot images faster and more efficiently saves power and increases the time interval between battery charges.

[0320] Compared to Figure 11A In some embodiments, the electronic device detects (1102) drag input on a touch-sensitive surface. In response to the detection of the drag input, the electronic device displays (1102) a screenshot capture power indicator via a display device. In some embodiments, the electronic device displays the screenshot capture power indicator within a user interface that includes other power indicators, such as icons corresponding to the respective applications. Displaying a user interface with multiple power indicators including the screenshot capture power indicator reduces the number of inputs required to open different user interfaces that include multiple power indicators. Therefore, the electronic device utilizes fewer processing resources, thereby extending the battery life of the electronic device. For example, drag input corresponds to dragging upwards from the bottom of the display area or dragging downwards from the top of the display area. As an example, in response to the detection of... Figure 8BIn the drag input 808, the electronic device 100 displays a screenshot capture display 810b within an interface including multiple application displays 810a (in...). Figure 8C middle).

[0321] In some embodiments, the drag input points to the first taskbar, and displaying a screenshot capture power indicator includes (1104) replacing the first taskbar with a second taskbar that includes the screenshot capture power indicator. By stopping the display of the first taskbar, the electronic device saves processing resources and battery life, while providing a screenshot capture power indicator for capturing screenshot images within the second taskbar. For example, replacing the first taskbar with the second taskbar includes the electronic device expanding the first taskbar to display the second taskbar.

[0322] In some embodiments, the drag input moves away from the corresponding edge of the touch-sensitive surface, and the electronic device displays (1106) a screenshot capture power indication within the control interface. In some embodiments, the electronic device displays the control interface based on the drag input and expands the control interface from the corresponding edge. By expanding the control interface from the corresponding edge as the drag input moves forward, the electronic device provides feedback to avoid blurring (e.g., obscuring) content on the display. As an example, in response to detecting... Figure 8B The electronic device 100 displays an interface 810 (in the form of a screenshot capture display 810b) by dragging the input 808 upwards. Figure 8C middle).

[0323] When content is displayed via a display device, the electronic device detects (1108) a screenshot capture input. For example, the screenshot capture input points to a screenshot capture capability indicator, such as... Figure 8B As shown. As another example, in Figure 8T In this context, screenshot capture input is a combination of hardware inputs, such as the simultaneous pressing of a main button via a second screenshot capture input 858 and a press-down button 206 via a second screenshot capture input 860 detected by electronic device 100.

[0324] In response to detecting a screenshot capture input, the electronic device captures (1110) a screenshot image of the content displayed via the display device. In some embodiments, the electronic device captures the screenshot image and stores it in non-transitory memory (e.g., random access memory (RAM) or cache). As an example, refer to... Figure 8D In response to the detection of the first screenshot capture input 811, the electronic device captures screenshot images of contents 803a, 803b, 804a-804c and 806.

[0325] In response to the detection of screenshot capture input: based on determining that the screenshot capture input is a first input type, the electronic device displays (1112) a screenshot editing interface for editing a screenshot image via a display device, wherein the screenshot editing interface includes the screenshot image. For example, the first input type corresponds to selecting a screenshot capture capability representation displayed on the display. In some embodiments, the first input type corresponds to one of standard tap input (e.g., finger tap) or stylus tap input. As an example, in response to the detection of... Figure 8D The first screenshot capture input 811, the electronic device 100 displays a screenshot editing interface 812 including a screenshot image 813 (in Figure 8E middle).

[0326] In response to the detection of a screenshot capture input: based on the determination that the screenshot capture input corresponds to a second input type different from the first input type, the electronic device displays (1114) a thumbnail representation of the screenshot image overlaid on the content via a display device. For example, the second input type is a hardware-based input, such as the pressing of one or more hardware buttons on the electronic device. As an example, in response to the simultaneous detection of a corresponding hardware input 858 pointing to the main button 204 and a corresponding hardware input 860 pointing to the press button 206 (in Figure 8T In the middle), electronic device 100 displays a thumbnail representation of a screenshot image overlaid on content 862 (in Figure 8U middle).

[0327] refer to Figure 11B In some embodiments, the first input type includes (1116) movement of a stylus on the edge of a touch-sensitive surface of the electronic device away from the touch-sensitive surface (e.g., criteria for determining that the input is a first type of input that triggers capture of a screenshot and optionally enters the screenshot editing interface include the requirement that the input is a swipe gesture away from the edge of the touch-sensitive surface). By displaying the screenshot editing interface in response to the release of a stylus movement determined to be a screenshot capture input, the electronic device avoids displaying the screenshot editing interface in response to non-stylus movements (such as finger movements). Therefore, the electronic device saves processing power and battery by not displaying the screenshot editing interface in response to the detection of certain types of input. For example, in response to the detection of... Figures 9B to 9D The first stylus moves 908, and the electronic device 100 displays the screen screenshot editing interface 917 (in Figure 9E (in the middle). As a counterexample, in response to detection Figures 9I to 9K In the first non-stylus drag input 922 (e.g., finger drag input), the electronic device 100 abandons the display of the screenshot editing interface (in Figure 9L(in the middle). As another counterexample, in response to detecting a second non-pen drag input 924 originating from the bottom edge 701d of the display interface (e.g., finger drag input) (in Figure 9M In the middle), electronic device 100 abandons the display of the screenshot editing interface and displays an interface 810b including the screenshot capture capability display (in the middle). Figure 9N (In some embodiments, when the electronic device detects movement of the stylus, the electronic device displays an animation representing a corresponding contraction of the displayed content. For example, see reference...) Figures 9B to 9D The electronic device shrinks the screenshot preview area 916 according to the movement of the first stylus 908. In some embodiments, the animation includes the outer area, such as Figures 9B to 9D The outer region 914.

[0328] In some embodiments, the release point within the touch-sensitive surface is at a threshold distance (1118) from the target location on the touch-sensitive surface (e.g., criteria for determining that the input is a first type of input that triggers capture of a screenshot and optionally enters a screenshot editing interface include a requirement that the input is a swipe gesture starting from the edge of the touch-sensitive surface, and include a lift-off of contact that performs the gesture at least at a threshold distance from the target location on the touch-sensitive surface). By displaying the screenshot editing interface in response to determining the threshold distance between the release point and the target location, the electronic device avoids displaying the screenshot editing interface erroneously. For example, the electronic device avoids displaying the screenshot editing interface in response to stylus movements intended to add content (e.g., draw marks) within the canvas of the current application interface. Avoiding the erroneous display of the screenshot editing interface improves the operability of the electronic device. For example, the target location is at or near the center of the touch-sensitive surface.

[0329] As an example, in response to determining that the first stylus moves 908 across to a second region 912 at a second threshold distance 912a from the edge 701a-701d (in Figure 9D (in the middle), electronic device 100 display screen screenshot editing interface 917 (in Figure 9E (In the middle). As a counterexample, in response to determining that the second stylus movement 920 did not cross into the second region 912 (in Figure 9G In the middle), electronic devices 100 abandon the display of the screenshot editing interface (in Figure 9H middle).

[0330] In some embodiments, the first input type corresponds to stylus movement on the touch-sensitive surface of the electronic device away from a corner of the touch-sensitive surface (e.g., criteria for determining that the input is a first type of input that triggers the capture of a screenshot and optionally enters the screenshot editing interface include the requirement that the input is a swipe gesture originating from a corner of the touch-sensitive surface (such as the lower corner of the touch-sensitive surface). In some embodiments, the first input type corresponds (1120) to stylus movement on the touch-sensitive surface of the electronic device away from a corner of the touch-sensitive surface. By displaying the screenshot editing interface when the stylus movement originates at a threshold distance from the corresponding corner, the electronic device avoids erroneously displaying the screenshot editing interface in response to certain stylus movement inputs. Therefore, the electronic device saves processing power and battery by not displaying the screenshot editing interface in response to the detection of certain types of input. For example, as shown in FIG9Q, the electronic device 100 detects a third stylus movement 928 originating at a threshold distance 929 from the corresponding corner. Continuing with this example, in response to the detection of the release point of the third stylus movement 928 within the second region 912 (in Figure 9S), the electronic device 100 displays a screenshot editing interface 917 (in Figure 9T).

[0331] In some embodiments, the screenshot editing interface also includes a drawing palette (1122) at a first location within the screenshot editing interface, and in response to a first input pointing to the first drawing palette, the first drawing palette can be moved to a second location within the screenshot editing interface. Including a movable first drawing palette improves the operability of the electronic device. For example, moving the first drawing palette results in a larger usable display area for drawing operations and other content modification operations. As an example, in response to a detected request to move the first drawing palette 704 to a drag input 876 along the third edge 701c (in... Figures 8AB to 8AD In the middle), the electronic device 100 moves the first drawing palette 704 accordingly, such as Figure 8AE As shown. Therefore, Figure 8AB The bottom area of ​​the screenshot editing interface 812, covered by the first drawing palette 704, can be used Figure 8AE Editing operations within the program.

[0332] In some embodiments, when a screenshot editing interface including an opacity level indication is displayed, the electronic device detects (1124) a first input pointing to the opacity level indication, wherein the first input sets the opacity level indication to a corresponding opacity value. In response to detecting the first input, the electronic device changes (1124) the opacity of a filter layer overlaid on the screenshot image to the corresponding opacity value. Changing the opacity of the filter layer in response to detecting the first input without requiring further input enhances the operability of the electronic device by reducing processor and battery usage. For example, the filter layer corresponds to a semi-transparent layer. In some embodiments, the screenshot editing interface includes three layers, wherein the screenshot image corresponds to the bottom layer, the filter layer corresponds to the middle layer, and the marker (e.g., annotation) layer corresponds to the top layer.

[0333] As an example, such as Figure 8H As shown, the electronic device 100 detects a drag input 836 pointing to the opacity value indicator 818a of the opacity level indicator 818, which sets the opacity level indicator to the corresponding opacity value. In response to the detection... Figure 8H By dragging input 836, the electronic device 100 changes the opacity of the first filter layer 838 overlaid on the screenshot image 813 to a corresponding opacity value (in Figure 8I (in the middle). As another example, electronic device 100 detects a drag input 872 pointing to an opacity value indicator 818a of opacity level indicator 818, which sets the opacity level indicator to the corresponding opacity value (in the middle). Figure 8Z (in Chinese). In response to detection Figure 8Z By dragging input 872, the electronic device 100 changes the opacity of the filter layer 874 overlaid on the second screenshot image 870 to a corresponding opacity value (in Figure 8AA middle).

[0334] In some embodiments, in response to detecting a first input, the electronic device displays (1126) a filter layer overlaid on a caption of a screenshot image via a display device. Displaying the filter layer over the caption of the screenshot image results in filtering both the screenshot image and the caption simultaneously, thereby avoiding multiple filtering operations. Therefore, the electronic device utilizes less processing power and battery usage. As an example, in response to detecting a drag input 836 that sets the opacity level indicator to a corresponding opacity value (in... Figure 8H In the middle), the electronic device 100 displays a second filter layer 840 with corresponding opacity values ​​overlaid on the screenshot image 813 and annotation 834. Figure 8J middle).

[0335] In some embodiments, in response to detecting a first input, the electronic device displays (1128) an annotation of a screenshot image as an overlay on a filter layer via a display device. Displaying the annotation over the filter layer causes the filter layer to blur the annotation less (or not blur the annotation at all), resulting in greater visibility of the annotation. More visible annotations reduce filtering downtime, resulting in the electronic device consuming less processing and battery resources. As an example, in response to detecting a drag input 836 that sets the opacity level indicator to a corresponding opacity value (…), Figure 8H In the middle), the electronic device 100 displays annotation 834 as an overlay on the first filter layer 838, having a corresponding opacity value (in Figure 8I middle).

[0336] In some embodiments, in response to detecting a second input via an input device pointing to a completion indication included within the screenshot editing interface, the electronic device displays (1130) a save interface via a display device. Furthermore, the electronic device detects (1130) a third input via an input device pointing to the save interface. In response to detecting the third input, the electronic device stores (1130) the screenshot image and filter layer as a flattened image. The save interface provides the user with an efficient mechanism for managing storage, thereby reducing the amount of user interaction required to perform storage management operations. Reduced user interaction reduces wear and tear on the electronic device. Reduced user interaction also leads to faster initiation of storage management operations, and thus reduces power consumption during storage management operations, thereby extending the battery life of the electronic device. Furthermore, providing the user with an efficient mechanism for managing storage increases the likelihood that the user will perform such management and improves the performance of the electronic device. In some embodiments, the electronic device stores the flattened image in a pre-allocated memory location (e.g., a "photo" area). In some embodiments, the flattened image is editable.

[0337] As an example, in response to the detection of input 852 pointing to the completion indication representation 814 (in Figure 8P In the middle), the electronic device 100 displays a save interface (e.g., a second save interface 853) (in Figure 8Q (in the middle). Continuing this example, in response to detecting subsequent input 854 pointing to the "Save to Photo" enable signal within the second save interface 853 (in Figure 8R In the middle), the electronic device 100 stores the screenshot image 813 and the second filter layer 840 as a flattened image.

[0338] refer to Figure 11CIn some embodiments, the screenshot editing interface also includes a corresponding power indicator, and the electronic device detects (1132) a first input pointing to the corresponding power indicator via an input device. In response to the detection of the first input, the electronic device adds (1132) additional content that is not displayed on the display when the screenshot capture input is detected to the screenshot editing interface. By adding additional content to the screenshot editing interface, the electronic device avoids additional input that requests the display of additional content. For example, the electronic device avoids the need to detect input that requires closing the screenshot capture interface, navigating to the additional content, and / or reopening the screenshot capture interface. Therefore, the electronic device saves processing and battery resources. For example, for a paginated document, the electronic device captures all pages into a single screenshot. As another example, the electronic device captures all content, such as multiple presentation slides or a complete webpage, as a screenshot or PDF to accommodate extended content. As yet another example, the additional content corresponds to a portion of a document that is accessible by scrolling up or down on the display. In some embodiments, the additional content corresponds to a scaled-down representation of the content displayed when the electronic device captures a screenshot image. The scaled-down representation may include smaller text, images, etc. In some embodiments, electronic devices display scrolling elements to facilitate viewing additional content.

[0339] As an example, in response to detecting input 868 pointing to "Show all" enable display 816 (in Figure 8X In the middle), electronic device 100 displays additional content 856b that was not displayed when the electronic device detected a screenshot capture input (e.g., city 7-10). Figure 8Y (in the middle). That is, in Figure 8T In this case, electronic device 100 does not display additional content 856b. As another example, in response to detecting input 885 pointing to the "Show All" enable indicator 816 (in... Figure 8AI In the middle), electronic device 100 displays additional content 884b and also displays the brush interface 887 (in Figure 8AJ (in the middle). Furthermore, in response to detecting input 888 pointing to the brush interface 887 (in... Figure 8AK (in the middle), electronic devices display more additional content 884c (in Figure 8AL middle).

[0340] In some embodiments, the electronic device detects (1134) movement of a stylus on a touch-sensitive surface of the electronic device, wherein the movement is away from a corresponding corner of the touch-sensitive surface and originates at a threshold distance from the corresponding corner; and in response to the release of the detected stylus movement, a screenshot capture menu including a capture screenshot capability representation and an edit screenshot screenshot capability representation is displayed (1134) via a display device. In response to the detection of a first input pointing to the capture screenshot capability representation, a screenshot image of the captured content is captured. In response to the detection of a second input pointing to the edit screenshot capability representation, a screenshot editing interface for editing the screenshot image is displayed via a display device. Because the screenshot capture menu includes multiple capability representations, the electronic device does not need to detect multiple inputs, each opening a single capability representation. Therefore, the electronic device saves battery and processing resources. For example, refer to Figures 9U to... Figure 9P The electronic device 100 displays a screenshot capture menu 938 based on a fourth stylus movement 930, which moves away from the corresponding corner of the touch-sensitive surface and originates at a threshold distance 929 from the corresponding corner.

[0341] In some embodiments, when displaying a thumbnail representation of a screenshot image, the electronic device detects (1136) a first input pointing to the thumbnail representation of the screenshot image via an input device. In response to detecting the first input, the electronic device displays (1136) a screenshot editing interface via a display device. By displaying the screenshot editing interface in response to the first input pointing to the thumbnail representation, the electronic device avoids detecting other inputs for displaying additional interfaces that would enable the subsequent display of the screenshot editing interface. Therefore, the electronic device reduces processor and battery usage. In some embodiments, in response to detecting the first input, the electronic device stops displaying the thumbnail representation. As an example, in response to detecting a first input 864 pointing to the thumbnail representation 862 (in Figure 8V In the middle), the electronic device 100 displays a screenshot editing interface 812 including a first screenshot image 866. Figure 8W middle).

[0342] In some embodiments, in response to detecting a second input via an input device pointing to a sharing capability representation included within the screenshot editing interface, the electronic device displays (1138) the sharing interface via a display device. Furthermore, the electronic device detects (1138) a third input via the input device pointing to the sharing interface. In response to detecting the third input, the electronic device stores (1138) a screenshot image and a filter layer as an image file, wherein the screenshot image and the filter layer are individually editable. The sharing interface provides the user with an efficient mechanism for managing storage, thereby reducing the amount of user interaction required to perform storage management operations. Reduced user interaction reduces wear and tear on the electronic device. Reduced user interaction also leads to faster initiation of storage management operations, and thus reduces power consumption during storage management operations, thereby extending the battery life of the electronic device. Furthermore, providing the user with an efficient mechanism for managing storage increases the likelihood that the user will perform such management and improves the performance of the electronic device. For example, in some embodiments, the image file is editable. As another example, the image file is not flattened.

[0343] As an example, in response to detecting input 842 pointing to the shared power representation 820 (in Figure 8K In the middle), the electronic device 100 displays a transfer interface 844i including a "Save to File" indicator 844i (in Figure 8L (in the middle). Continuing this example, in response to detecting input 846 pointing to the "Save to file" enablement display 844i (in the middle). Figure 8M In the middle), the electronic device 100 displays the first save interface 848 (in Figure 8N (in the middle). Continuing this example, in response to detecting input 850 pointing to the location of the "My Images" folder within the first save interface 848 (in Figure 8O In the middle), the electronic device 100 stores the screenshot image 813 and the second filter layer 840 as an image file, wherein the screenshot image 813 and the second filter layer 840 are editable separately.

[0344] Figures 12A to 12AP These are examples of user interfaces for selectively erasing portions of an object, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 13A to 13D The process in the process. Although some examples in the following examples will refer to input on a touchscreen display (where a touch-sensitive surface and a display are combined, such as on touchscreen 112), in some embodiments, electronic device 100 detects input on a touch-sensitive surface 651 separate from display 650, such as Figure 6B As shown.

[0345] like Figure 12AAs shown, the electronic device 100 displays a drawing user interface 1202, such as a drawing application interface or a word processing application interface. The electronic device 100 displays a drawing palette 1204. The first drawing palette 1204 may include various indications (e.g., drawing tool indications, editing function indications, and / or color palettes) to facilitate content manipulation. Those skilled in the art will understand that the drawing palette 1204 may include any number and type of indications arranged in any manner.

[0346] The drawing palette 1204 includes an undo indicator 1204a and a redo indicator 1204b. An input to the undo indicator 1204a requests the electronic device 100 to undo a previous operation, such as erasing previously drawn marks. An input to the redo indicator 1204b requests the electronic device 100 to redo a previously undone operation, such as redisplaying previously erased marks.

[0347] The drawing palette 1204 includes a set of drawing tool indications, including a pen indication 1204c, a marker indication 1204d (e.g., a highlighter indication), a pencil indication 1204e, a ruler indication 1204f, and an eraser indication 1204g. Notably, the eraser indication 1204g has a first appearance (e.g., an "x" inside and near the top of the eraser indication 1204g) to indicate that the eraser tool associated with the eraser indication 1204g is in object erasure operation mode. The function of the eraser tool in object erasure operation mode is described below.

[0348] Furthermore, when a specific drawing tool's capability is selected from the set of drawing tool capability representations, input pointing to the drawing user interface 1202 causes the electronic device 100 to perform a corresponding content manipulation operation within the drawing user interface 1202. For example... Figure 12A As shown, the pencil indicator 1204e is currently selected, indicating that the pencil tool is the currently selected drawing tool. Input pointing to the corresponding drawing tool indicator sets that drawing tool as the currently selected drawing tool.

[0349] The drawing palette 1204 includes a set of color pots 1204h, which includes a top row color indicator for setting the currently selected color and a bottom row pattern indicator for setting the currently selected pattern associated with that color. For example... Figure 12A As shown, the current selection is black and solid pattern. Input pointing to the corresponding color indicator or the corresponding pattern indicator will set the corresponding color / pattern as the currently selected color / pattern.

[0350] The drawing palette 1204 includes a text tool display 1204i that enables the creation of text content within the drawing user interface 1202. For example, after selecting the text tool display 1204i, input to the drawing user interface 1202 causes the electronic device 100 to display a text box for receiving a text string, and causes the electronic device 100 to replace the text box with the text string entered into the text box.

[0351] The drawing palette 1204 includes a shape tool display 1204j that enables the placement of specific shapes within the drawing user interface 1202. In some embodiments, for example, input pointing to the shape tool display 1204j recalls a shape interface including various predetermined shapes (e.g., squares, circles, triangles). Subsequently, the electronic device 100 detects input corresponding to dragging a specific shape from the shape interface into the drawing user interface 1202. In response, the electronic device 100 displays the specific shape at that location within the drawing user interface 1202.

[0352] like Figure 12B As shown, the electronic device 100 detects an object insertion input 1206 corresponding to a request to insert an object 1208 into the drawing user interface 1202. In some embodiments, the object insertion input 1206 corresponds to a contact button. Figure 12B The shape shown is moving. In response to detection... Figure 12B The object insertion input 1206 in the electronic device 100 inserts the object 1208 into the drawing user interface 1202, such as... Figure 12C As shown.

[0353] like Figure 12D As shown, the electronic device 100 detects an input 1210 pointing to an eraser indicator 1204g. In some embodiments, input 1210 corresponds to a tap input. In response to the detection... Figure 12D Input 1210 in the input field, and the electronic device 100 sets the eraser tool as the currently selected tool, such as Figure 12E As shown.

[0354] like Figure 12F As shown, the electronic device 100 detects an input 1211 pointing towards the eraser indicator 1204g. In some embodiments, the input 1211 corresponds to a long touch input (e.g., a touch input that is detected to last for a duration greater than a threshold amount) or a force-sensitive touch input (e.g., a touch input associated with a level of force greater than a threshold amount). In response to the detection Figure 12F When inputting 1211, the electronic device 100 displays the eraser mode interface 1212, such as... Figure 12GAs shown. In some embodiments, the eraser mode interface 1212 at least partially covers the drawing palette 1204. The eraser mode interface 1212 includes a pixel eraser indication 1212a associated with a pixel eraser operation mode and an object eraser indication 1212b associated with an object eraser operation mode. Figure 12G As shown, the object eraser indicator 1212b is currently selected, indicating that the eraser tool is currently in object erasure operation mode. The eraser mode interface 1212 also includes multiple eraser thickness indicators 1212c, each associated with a multiple thickness level. Input pointing to a specific eraser thickness indicator among the multiple eraser indicator indicators 1212c sets the thickness associated with the eraser tool to the corresponding thickness level.

[0355] like Figure 12H As shown, the electronic device 100 detects input 1213 pointing to the pixel eraser indicator 1212a. In some embodiments, input 1213 corresponds to a tap input. In response to detection Figure 12H Input 1213 in the input field causes the electronic device 100 to change the eraser tool from object erasure operation mode to pixel erasure operation mode, as indicated by the selected pixel eraser indicator 1212a. Figure 12I As shown. Furthermore, as... Figure 12J As shown, the electronic device 100 stops displaying the eraser mode interface 1212 and changes the appearance of the eraser indicator 1204g from a first appearance to a second appearance (e.g., a shaded area inside and near the top of the eraser indicator 1204g). In some embodiments, the electronic device 100 stops displaying the eraser mode interface 1212 after a threshold amount of time has elapsed since the input 1213 was detected. The second appearance indicates that the eraser tool is in a pixel erasure operation mode, compared to the first appearance which indicates that the eraser tool is in an object erasure operation mode.

[0356] like Figure 12K As shown, the electronic device 100 detects a first pixel erase input 1214. In some embodiments, the first pixel erase input 1214 corresponds to a movement of the contact. The first pixel erase input 1214 defines a first path through the object 1208. Therefore, as Figure 12LAs shown, the electronic device 100 stops displaying (e.g., erasing) a first portion of object 1208 while maintaining the display of a second portion 1208a and a third portion 1208b of object 1208. It is noteworthy that the second portion 1208a is not connected to the third portion 1208b. In some embodiments, in response to detecting object erasure input defining a first path (e.g., when the eraser tool is in object erasure operation mode), the electronic device 100 stops displaying (e.g., erasing) the entire object 1208, rather than stopping displaying pixels within the first path.

[0357] like Figure 12M As shown, the electronic device 100 detects an input 1215 pointing towards the eraser indicator 1204g. In some embodiments, the input 1215 corresponds to a long touch input (e.g., a touch input that is detected to last for a duration greater than a threshold amount) or a force-sensitive touch input (e.g., a touch input associated with a level of force greater than a threshold amount). In response to the detection Figure 12M When inputting 1215, the electronic device 100 displays the eraser mode interface 1212, such as... Figure 12N As shown.

[0358] like Figure 12O As shown, the electronic device 100 detects input 1218 to the eraser indicator 1212b pointing to the object. In some embodiments, input 1218 corresponds to a tap input. In response to detection Figure 12O Input 1218 in the input field causes the electronic device 100 to change the eraser tool from pixel erasure operation mode to object erasure operation mode, as indicated by the selected object eraser display 1212b. Figure 12P As shown. Furthermore, as... Figure 12Q As shown, the electronic device 100 stops displaying the eraser mode interface 1212 and changes the appearance of the eraser indicator 1204g from the second appearance to the first appearance (e.g., an "x" at the top of the eraser indicator 1204g), thereby indicating that the eraser tool is in object erasure operation mode. In some embodiments, the electronic device 100 stops displaying the eraser mode interface 1212 after a threshold amount of time has elapsed since the input 1218 was detected.

[0359] like Figure 12R As shown, the electronic device 100 detects a first object erase input 1220. In various embodiments, the first object erase input 1220 corresponds to a movement of the contact. The first object erase input 1220 defines a second path that intersects with the third portion 1208b of the object 108 but does not intersect with the second portion 1208a of the object 1208. In response to the detection Figure 12RWhen the first object erase input 1220 is entered, the electronic device 100 stops displaying the third part 1208b of object 1208, but does not stop displaying the second part 1208a of object 1208, as shown below. Figure 12S As shown. In some embodiments, in response to detecting a pixel erase input defining the second path (e.g., when the eraser tool is in pixel erase operation mode), the electronic device 100 stops displaying pixels in the second path, instead of stopping displaying the entire third portion 1208b of the object 1208.

[0360] like Figure 12T As shown, the electronic device 100 detects an input 1222 pointing to the deactivation indicator 1204a. In some embodiments, input 1222 corresponds to a tap input. In response to the detection... Figure 12T Input 1222 in the input field, and the electronic device 100 cancels the stop display of the third part 1208b (e.g., redisplays the third part 1208b), as shown. Figure 12U As shown.

[0361] like Figure 12V As shown, the electronic device 100 detects a second object erasure input 1224. In some embodiments, the second object erasure input 1224 corresponds to a movement of the contact. The second object erasure input 1224 defines a third path that intersects with the second portion 1208a of the object 1208 but does not intersect with the third portion 1208b of the object 1208. In response to the detection Figure 12V When the second object erase input 1224 is entered, the electronic device 100 stops displaying the second part 1208a of object 1208, but does not stop displaying the third part 1208b of object 1208, as shown. Figure 12W As shown. In some embodiments, in response to detecting a pixel erase input defining a third path (e.g., when the eraser tool is in pixel erase operation mode), the electronic device 100 stops displaying pixels in the third path, instead of stopping displaying the entire second portion 1208a of the object 1208.

[0362] like Figures 12X to 12AP As shown, in some embodiments, the electronic device 100 stops displaying (e.g., deleting) the entire object 1208 in response to detecting an object erase input pointing to the fourth portion 1208c of the object 1208. It is worth noting that, in conjunction with the above reference... Figures 12A to 12W Compared to the example described, the fourth portion 1208c of object 1208 is connected to the remainder of object 1208 generated by the previous pixel erasure input.

[0363] like Figure 12X As shown, electronic device 100 responds to detecting Figure 12BObject 1208 is displayed when object 1206 is inserted into the input 1206. Additionally, the electronic device 100 detects input 1226 pointing to the eraser indicator 1204g. In some embodiments, input 1226 corresponds to a long touch input (e.g., a touch input that lasts for a duration greater than a threshold amount) or a force-sensitive touch input (e.g., a touch input associated with a level of force greater than a threshold amount). In response to the detection... Figure 12X When inputting 1226, the electronic device 100 displays the eraser mode interface 1212, as shown. Figure 12Y As shown.

[0364] like Figure 12Z As shown, the electronic device 100 detects an input 1228 pointing to the pixel eraser indicator 1212a. In some embodiments, the input 1228 corresponds to a tap input. In response to the detection Figure 12Z Input 1228 in the input field causes the electronic device 100 to change the eraser tool from object erasure operation mode to pixel erasure operation mode, as indicated by the selected pixel eraser indicator 1212a. Figure 12AA As shown. Furthermore, as... Figure 12AB As shown, the electronic device 100 stops displaying the eraser mode interface 1212 and changes the appearance of the eraser indicator 1204g from a first appearance to a second appearance, thereby indicating that the eraser tool is in pixel erasure operation mode. In some embodiments, the electronic device 100 stops displaying the eraser mode interface 1212 after a threshold amount of time has elapsed since the input 1228 was detected.

[0365] like Figure 12AC As shown, the electronic device 100 detects an input 1230 pointing towards the eraser indicator 1204g. In some embodiments, the input 1230 corresponds to a long touch input (e.g., a touch input that is detected to last for a duration greater than a threshold amount) or a force-sensitive touch input (e.g., a touch input associated with a level of force greater than a threshold amount). In response to the detection Figure 12AC Input 1230, electronic device 100 displays eraser mode interface 1212 (in Figure 12AD middle).

[0366] like Figure 12AE As shown, the electronic device 100 detects an input 1232 pointing to the thickest eraser thickness indicator among a plurality of eraser thickness indicator indicators 1212c. In some embodiments, input 1232 corresponds to a tap input. In response to detection Figure 12AE Inputting 1232, the electronic device 100 sets the thickness of the drawing tool to the thickest thickness level, such as... Figure 12AF As shown. Therefore, compared to the previous example, subsequent erase input will erase a larger portion of the content. Figure 12AGAs shown, the electronic device 100 stops displaying the eraser mode interface 1212. In some embodiments, the electronic device 100 stops displaying the eraser mode interface 1212 after a threshold amount of time has elapsed since the input 1232 was detected.

[0367] like Figure 12AH As shown, the electronic device 100 detects the second pixel erase input 1234. In some embodiments, the second pixel erase input 1234 corresponds to a movement of the contact. It is noteworthy that, compared to the example provided above with reference to the first pixel erase input 1214, the second pixel erase input 1234 defines a fourth path that does not divide the object 1208 into two separate parts. Instead, as... Figure 12 AI As shown, the second pixel erase input 1234 removes the bottom block of object 1208, leaving the fourth portion 1208c as a connected (e.g., continuous) drawing mark. In some embodiments, in response to detecting an object erase input defining the fourth path (e.g., when the eraser tool is in object erase operation mode), the electronic device 100 stops displaying (e.g., erasing) the entire object 1208, rather than stopping displaying pixels in the fourth path.

[0368] like Figure 12AJ As shown, the electronic device 100 detects an input 1236 pointing towards the eraser indicator 1204g. In some embodiments, the input 1236 corresponds to a long touch input (e.g., a touch input that is detected to last for a duration greater than a threshold amount) or a force-sensitive touch input (e.g., a touch input associated with a level of force greater than a threshold amount). In response to the detection Figure 12AJ When the input is 1236, the electronic device 100 displays the eraser mode interface 1212, such as... Figure 12AK As shown.

[0369] like Figure 12AL As shown, the electronic device 100 detects input 1238 to the eraser indicator 1212b pointing to the object. In some embodiments, input 1238 corresponds to a tap input. In response to detection Figure 12AL Input 1238 in the input field causes the electronic device 100 to change the eraser tool from pixel erasure operation mode to object erasure operation mode, as indicated by the selected object eraser display 1212b. Figure 12AM As shown. Furthermore, as... Figure 12AN As shown, the electronic device 100 stops displaying the eraser mode interface 1212 and changes the appearance of the eraser indicator 1204g from the second appearance to the first appearance, thereby indicating that the eraser tool is in object erasure operation mode. In some embodiments, the electronic device 100 stops displaying the eraser mode interface 1212 after a threshold amount of time has elapsed since the input 1238 was detected.

[0370] like Figure 12AO As shown, the electronic device 100 detects a third object erasure input 1240. In some embodiments, the third object erasure input 1240 corresponds to a movement of the contact. The third object erasure input 1240 defines a fifth path that intersects with the fourth portion 1208c of the object 1208. Figure 12AO As shown, the third object erase input 1240 passes through the section of the fourth part 1208c located to the right of the previously removed bottom block of object 1208. Figure 12 AP As shown, the third object erase input 1240 removes the entire object 1208 because the fifth path intersects with the fourth part 1208c of object 1208, and the fourth part 1208c of object 1208 corresponds to a single (e.g., not divided into multiple parts) drawing mark. This is consistent with the above reference. Figures 12R to 12W The two provided object erase input examples are opposite, wherein the previous pixel erase input 1214 divides object 1208 into two separate (e.g., separate) portions, such as a second portion 1208a and a third portion 1208b. In some embodiments, in response to detecting an object erase input that passes through a segment located to the left of the previously removed bottom block of object 1208 in the fourth portion 1208c, the electronics 100 removes the entire object 1208. In some embodiments, in response to detecting a pixel erase input defining a fifth path (e.g., when the eraser tool is in pixel erase operation mode), the electronics 100 stops displaying pixels in the fifth path, rather than stopping displaying the entire object 1208.

[0371] Figures 13A to 13D This is a flowchart of a method 1300 for selectively erasing portions of an object according to some embodiments. In some embodiments, method 1300 is used in an electronic device having one or more processors, non-transitory memory, input devices, and display devices (e.g., Figure 3 Electronic devices 300 or Figures 12A to 12AP The operation is performed at the portable multifunction device 100. Some operations in method 1300 may be optionally combined, and / or the order of some operations may be optionally changed.

[0372] Method 1300 includes: after dividing an object into multiple separate portions based on a pixel erase input, the electronic device, in response to the object erase input, stops displaying a specific separate portion and continues displaying the remaining portions. Therefore, the electronic device provides more functionality and control compared to an erase operation. Furthermore, the electronic device does not need to receive a drag erase input that spatially coexists with the separate portions of the object for erasing that portion. By utilizing the object erase input instead of the drag erase input to erase the separate portions, the electronic device reduces processing and battery consumption and experiences less wear and tear.

[0373] Compared to Figure 13A The electronic device displays (1302) a drawing user interface via a display device. For example, the drawing user interface includes content marked with objects (e.g., an annotated screenshot image). As another example, the drawing user interface includes a toolbar area that includes multiple drawing tool selection options associated with multiple drawing tools. As an example, see [reference]. Figure 12A The electronic device 100 displays a drawing user interface 1202.

[0374] When the drawing user interface is displayed, the electronic device detects (1304) an object insertion input corresponding to a request to insert an object into the drawing user interface. For example, the object may be a line drawing input, a text input, or a request to insert a predefined object such as a shape. As an example, refer to... Figure 12B The electronic device 100 detects the object insertion input 1206 corresponding to the request to insert a closed object into the drawing user interface 1202.

[0375] In response to the detection of the object insertion input, the electronic device inserts the corresponding object into the drawing user interface (1306). For example, the corresponding object corresponds to the object corresponding to a stroke defined by continuous user input within the drawing user interface when a drawing tool associated with a stroke operation is selected. As another example, the corresponding object corresponds to one of a shape object, a stroke object, a magnifying glass object, or a text object. As yet another example, the corresponding object is generated due to movement of a contact on the touch-sensitive surface of the electronic device, which ends when the contact is released. As yet another example, the corresponding object is generated due to a mouse click, followed by dragging the cursor on the display, which ends when the mouse click is released. As an example, in response to the detection of... Figure 12B The object is inserted into input 1206, and the electronic device 100 displays the corresponding object 1208 (in...). Figure 12C middle).

[0376] When the corresponding object is displayed in the drawing user interface, the electronic device detects the (1308) pixel erase input. As an example, refer to... Figure 12K Electronic device 100 detects a first pixel erase input 1214, which defines a corresponding path through object 1208 (e.g., dividing the object into two). As another example, see [reference needed]. Figure 12AH The electronic device 100 detects a second pixel erase input 1234, which defines a corresponding path for removing a segment of object 1208 instead of passing through it.

[0377] In response to the detection of the pixel erase input, the electronic device stops (1310) displaying the first portion of the corresponding object, but does not stop displaying the second portion of the corresponding object, nor does it stop displaying the third portion of the corresponding object. For example, in some embodiments, the second and third portions are separated from each other (e.g., not connected). For example, in response to the detection of... Figure 12K The first pixel erase input 1214, the electronic device 100 maintains the display of the second part 1208a and the third part 1208b (in Figure 12L In the example where the second part 1208a and the third part 1208b are not connected to each other, the second and third parts are, in some embodiments, consecutively associated (e.g., connected), as shown in the example. Figure 12AH and Figure 12 AI As shown.

[0378] The electronic device detects (1312) an object erase input pointing to a portion of the corresponding object. For example, the electronic device detects the object erase input after stopping the display of the first portion of the corresponding object without stopping the display of the second and third portions of the corresponding object. As an example, the electronic device 100 detects... Figure 12R The first object to erase is input 1220. As an example, electronic device 100 detects... Figure 12V The second object erase input is 1224. As an example, electronic device 100 detects... Figure 12AO The third object in the text is erased by input 1240.

[0379] In response to the detection of the object erase input: based on the determination that the object erase input points to the second part of the corresponding object and that the second part of the corresponding object is not connected to the third part of the corresponding object, the electronic device stops (1314) displaying the second part of the corresponding object, but does not stop displaying the third part of the corresponding object. As an example, in response to the detection of the object erase input: Figure 12R When the first object erase input 1220 is entered, the electronic device 100 stops displaying the third part 1208b of object 1208, but does not stop displaying the second part 1208a of object 1208 (in Figure 12S middle).

[0380] In response to the detection of the object erase input: based on the determination that the object erase input points to the third part of the corresponding object and that the third part of the corresponding object is not connected to the second part of the corresponding object, the electronic device stops (1316) displaying the third part of the corresponding object, but does not stop displaying the second part of the corresponding object. As an example, in response to the detection of the object erase input: Figure 12VWhen the second object erase input 1224 is entered, the electronic device 100 stops displaying the second part 1208a of object 1208, but does not stop displaying the third part 1208b of object 1208 (in Figure 12W middle).

[0381] refer to Figure 13B In some embodiments, when a corresponding object is displayed in the drawing user interface and before a pixel eraser input is detected: the electronic device performs the following operations: displays (1318) a drawing palette including multiple content manipulation capabilities within the drawing user interface; detects (1318) a first input pointing to an eraser capability among the multiple content manipulation capabilities via an input device, wherein the eraser capability is associated with an eraser tool; in response to the detection of the first input, displays (1318) an eraser mode interface including multiple eraser mode capabilities; detects (1318) a second input pointing to the first eraser mode capability among the multiple eraser mode capabilities via an input device; and in response to the detection of the second input, sets (1318) the eraser tool to a pixel eraser operation mode, wherein the pixel eraser input is detected when the eraser tool is in the pixel eraser operation mode. By displaying multiple eraser mode capabilities within the eraser mode interface, the electronic device does not need to detect multiple inputs for displaying the corresponding multiple eraser mode capabilities. Therefore, processor usage, battery usage, and wear and tear of the electronic device are reduced. For example, content manipulation indicators include pencil indicators, eraser indicators, lasso indicators, highlighter indicators, undo indicators, redo indicators, color palettes (e.g., hues and shadows), etc. As another example, the first input corresponds to a long press touch or a force-sensitive touch. In some embodiments, the eraser mode interface at least partially covers the drawing palette. In some embodiments, the eraser mode interface includes circles of different sizes indicating the corresponding thickness of the erasure operation. In some embodiments, the drawing palette is removable. In some embodiments, the drawing palette is fixed to the toolbar area. As an example, electronic device 100 detects input 1211 (in... Figure 12F (in the middle) and accordingly open the eraser mode interface 1212 (in Figure 12G (in the middle), and then detect the input 1213 pointing to the pixel eraser indicator 1212a (in Figure 12H (in Chinese). In response to detection Figure 12H Input 1213 in the input field indicates that the electronic device 100 sets the eraser tool to pixel erase operation mode. When the eraser tool is in pixel erase operation mode, the electronic device 100 detects the first pixel erase input 1214 (in the input field). Figure 12K middle).

[0382] In some embodiments, when the eraser tool is in pixel erase operation mode: the electronic device detects (1320) a third input pointing to a second eraser mode display among the plurality of eraser mode display representations via an input device; and in response to detecting the third input, the electronic device sets (1320) the eraser tool to object erase operation mode, wherein an object erase input is detected when the eraser tool is in object erase operation mode. For example, the eraser tool associated with a screenshot marker interface is in object erase operation mode. As another example, erasing an object in object erase operation mode corresponds to completely removing (e.g., deleting or stopping the display) the object. As yet another example, the electronic device erases the object when the rate of the erase input meets a rate threshold. Pixel erase mode is different from object erase mode. As an example, electronic device 100 detects input 1215 (in Figure 12M (in the middle) and accordingly open the eraser mode interface 1212 (in Figure 12N (in), and then detect the input 1218 of the eraser indicator 1212b pointing to the object (in Figure 12O (in Chinese). In response to detection Figure 12O Input 1218 in the input field, and the electronic device 100 sets the eraser tool to object erasure operation mode. When the eraser tool is in object erasure operation mode (in...), Figure 12K (In the middle), electronic device 100 detects the first object erasure input 1220.

[0383] In some embodiments, when the eraser tool is in object erase operation mode, the eraser indication has a first appearance (1322), and when the eraser tool is in pixel erase operation mode, the eraser indication has a second appearance (1322) different from the first appearance. By indicating the current erase operation mode, the electronic device detects fewer erroneous erase inputs pointing into the drawing interface, thereby reducing processor utilization and wear of the electronic device. For example, the first appearance includes an "X" near the top of the eraser indication, such as... Figure 12Q As shown. As another example, the second appearance includes a shaded area near the top of the eraser indicator, as... Figure 12J As shown.

[0384] In some embodiments, a first portion of the corresponding object lies within a first path defined by a pixel erase input (1324). By stopping the display of the first portion based on the first path defined by the pixel erase input, the electronic device provides an accurate erasure mechanism. For example, the length of the first path relative to the size of the object determines whether to delete a portion of the object or delete a portion of the object and split the object. As another example, if the length of the first path extends through the object, the electronic device deletes a portion of the object within the first path and splits the remaining portion of the object into a second and a third portion. As an example, in response to detecting a second pixel erase input 1234 that defines a corresponding path that does not extend through the object 1208, Figure 12AH In the middle), the electronic device 100 erases a segment of the object without dividing the object 1208 into two separate parts (in Figure 12 AI (in the middle), leaving the fourth part 1208c of object 1208.

[0385] In some embodiments, a first path defined by a pixel erase input traverses (1326) the corresponding object, creating a second portion of the corresponding object that is not connected to a third portion of the corresponding object. For example, the second portion corresponds to the left side of a segmented square, and the third portion corresponds to the right side of the segmented square. As an example, in response to detecting a first pixel erase input 1214 (in) defining a corresponding path traversing object 1208, Figure 12K In the middle), the electronic device 100 divides the object 1208 into (for example, into two unconnected parts) a second part 1208a and a third part 1208b (in Figure 12L middle).

[0386] refer to Figure 13C In some embodiments, in response to detecting the object erase input: based on determining that the object erase input points to the second part of the corresponding object and that the second part of the corresponding object is connected to the third part of the corresponding object, the electronic device stops (1328) displaying the second part of the corresponding object and stops displaying (1328) the third part of the corresponding object. By stopping the display of both the second and third parts of the corresponding object in response to a single object erase input, the electronic device does not need to detect multiple erase inputs for erasing these two parts, thereby reducing processor usage, battery usage, and wear and tear on the electronic device. For example, removing the first part of the corresponding object in response to a previous pixel erase input divides the corresponding object into a second and third part. As an example, in response to determining that the third object erase input 1240 points to the fourth part 1208c and that the fourth part 1208c corresponds to a connected drawing mark (in Figure 12AO In the middle), electronic device 100 removes the entire object 1208 (in Figure 12 AP middle).

[0387] In some embodiments, in response to detecting the object erase input: based on determining that the object erase input points to the third part of the corresponding object and that the third part of the corresponding object is connected to the second part of the corresponding object, the electronic device stops (1330) displaying the third part of the corresponding object and stops (1330) displaying the second part of the corresponding object. By stopping the display of both the second and third parts of the corresponding object in response to a single object erase input, the electronic device does not need to detect multiple erase inputs for erasing these two parts, thereby reducing processor usage, battery usage, and wear and tear on the electronic device. For example, removing the first part of the corresponding object in response to a previous pixel erase input divides the corresponding object into the second and third parts. As an example, in response to determining that the third object erase input 1240 points to the fourth part 1208c and that the fourth part 1208c corresponds to a connected drawing mark (in Figure 12AO In the middle), electronic device 100 removes the entire object 1208 (in Figure 12 AP middle).

[0388] In some embodiments, based on the determination that the object erase input defines a third path intersecting with both the second and third portions of the corresponding object, the electronic device stops (1332) displaying the second portion and the third portion of the corresponding object. By stopping the display of both the second and third portions of the corresponding object in response to a single object erase input, the electronic device does not need to detect multiple erase inputs for erasing these two portions, thereby reducing processor usage, battery usage, and wear and tear on the electronic device.

[0389] In some embodiments, in response to detecting the object erase input: based on determining that the object erase input points to the second part of the corresponding object and that the second part of the corresponding object is connected to the third part of the corresponding object, the electronic device stops (1334) displaying the second part of the corresponding object and stops (1334) displaying the third part of the corresponding object; and based on determining that the object erase input points to the third part of the corresponding object and that the se...

Claims

1. A method, the method comprising: In an electronic device having one or more processors, non-transitory memory, input devices, and display devices: A first drawing palette is displayed at a first position within a first application interface via the display device, wherein the first drawing palette has a first appearance at the first position, and in the first appearance, the first drawing palette includes a plurality of content manipulation indications, the plurality of content manipulation indications including an indication of the currently selected drawing tool, and the indication of the currently selected drawing tool is displayed simultaneously with one or more indications of other available drawing tools; The input device detects a first input corresponding to a request to move the first drawing palette within the first application interface; as well as In response to the detection of the first input: Based on the determination that the first input corresponds to a request to move the first drawing palette to a second position within the first application interface, the first drawing palette having the first appearance is displayed at the second position; as well as Based on the determination that the first input corresponds to a request to move the first drawing palette to a third position within the first application interface that is different from the second position, the first drawing palette with a second appearance different from the first appearance is displayed at the third position. In the second appearance, the first drawing palette includes a single content manipulation indicator that includes a representation of the currently selected drawing tool. The first drawing palette with the second appearance does not include representations of other drawing tools in the first drawing palette.

2. The method according to claim 1, wherein the first position corresponds to a first position type, the method further comprising: Based on determining that the first input corresponds to the first input type, the first drawing palette having the first appearance is displayed at the second position based on the first position type and the first input type; as well as Based on the determination that the first input corresponds to a second input type different from the first input type, the first drawing palette having the second appearance is displayed at the third position based on the first position type and the second input type.

3. The method of claim 1, wherein determining that the first input corresponds to the request to move the first drawing palette to the second position includes determining that the second position corresponds to a first position type, and wherein determining that the first input corresponds to the request to move the first drawing palette to the third position includes determining that the third position corresponds to a second position type different from the first position type.

4. The method according to claim 1, further comprising: When the first drawing palette having the first appearance and first orientation is displayed at the second position, wherein the second position corresponds to the first position type: The input device detects a second input corresponding to a request to move the first drawing palette to a fourth position within the first application interface, wherein the fourth position corresponds to the first position type; as well as In response to the detection of the second input: Based on the determination that the fourth position is on the opposite side of the display compared to the second position, the first drawing palette having the first appearance and the first orientation is displayed at the fourth position; as well as Based on the determination that the fourth position is not on the opposite side of the display compared to the second position, the first drawing palette having the first appearance and a second orientation different from the first orientation is displayed at the fourth position.

5. The method according to claim 1, further comprising: In response to detecting a second input corresponding to a drawing operation on the first application interface, the display of the first drawing palette is stopped.

6. The method of claim 1, wherein the first appearance corresponds to the first drawing palette in a first expanded state, and wherein the second appearance corresponds to the first drawing palette in a compressed state.

7. The method according to claim 6, further comprising: When the first drawing palette in the compressed state is displayed at the third position: Touch input directed at the first drawing palette is detected via the input device; as well as In response to the detection of the touch input, the first drawing palette is displayed in a second expanded state, which is different from the first expanded state, wherein the first drawing palette in the second expanded state includes more representations of drawing tools than the first drawing palette in the compressed state.

8. The method according to claim 7, further comprising: The input device detects a second input that manipulates a specific content within the first drawing palette in the second extended state; as well as In response to detecting the second input, the current editing settings associated with the first drawing palette are manipulated based on the specific content.

9. The method according to claim 1, further comprising: When the first drawing palette is displayed at the second or third position: The input device detects a tap input directed at the first drawing palette; as well as In response to detecting the tap input, the first drawing palette is moved to the first position.

10. The method according to claim 1, further comprising: Based on the determination that the first input satisfies the first distance threshold, it is determined that the first input corresponds to the request to move the first drawing palette to the second position, and the first drawing palette having the first appearance is displayed at the second position; as well as Based on the determination that the first input does not meet the first distance threshold, the first drawing palette with the first appearance is displayed at the first location.

11. The method according to claim 1, further comprising: Based on the determination that the first input meets the speed threshold, it is determined that the first input corresponds to the request to move the first drawing palette to the second position, and the first drawing palette having the first appearance is displayed at the second position; as well as Based on the determination that the first input does not meet the speed threshold, the first drawing palette with the first appearance is displayed at the first position.

12. The method according to claim 1, further comprising: The second application interface and the first application interface are simultaneously displayed via the display device, wherein the second application interface includes a second drawing palette; Based on the determination that the first application interface has a corresponding size characteristic that does not meet the threshold size, the first drawing palette is set to be unable to move within the first application interface, and the second drawing palette is set to be able to move within the second application interface. The second input is detected via the input device; as well as In response to the detection of the second input: Based on the determination that the second input corresponds to a request to move the first drawing palette to a fourth position within the first application interface, the current position of the first drawing palette is maintained; as well as Based on the determination that the second input corresponds to a request to move the second drawing palette to a fifth position within the second application interface, the second drawing palette is moved to the fifth position.

13. The method of claim 12, further comprising: The input device detects a third input directed to the first application interface; as well as In response to the detection of the third input, the second drawing palette is de-emphasized relative to the content displayed on the canvas of the second application interface.

14. An electronic device, the electronic device comprising: One or more processors; Non-transitory memory; Input devices; Display devices; and One or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any one of the methods according to claims 1 to 13.

15. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device having one or more processors, input devices, and display devices, cause the electronic device to perform any one of the methods according to claims 1 to 13.

16. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 13.

17. A method, the method comprising: In an electronic device having one or more processors, non-transitory memory, input devices, and display devices: When content is displayed via the display device, screen capture input is detected; and In response to the detection of the screenshot capture input: Capture a screenshot image of the content displayed via the display device; Based on the determination that the screenshot capture input is a first input type, a screenshot editing interface for editing the screenshot image is displayed via the display device, wherein the screenshot editing interface includes the screenshot image; as well as Based on the determination that the screenshot capture input corresponds to a second input type different from the first input type, a thumbnail representation of the screenshot image overlaid on the content is displayed via the display device.

18. The method of claim 17, further comprising: Drag input is detected on the touch-sensitive surface of the electronic device; as well as In response to detecting the drag input, a screenshot capture power indication is displayed via the display device; The first input type corresponds to the selection of the screenshot capture capability representation.

19. The method of claim 18, wherein the drag input points to the first taskbar, and wherein displaying the screenshot capture indication includes replacing the first taskbar with a second taskbar that includes the screenshot capture indication.

20. The method of claim 18, wherein the drag input moves away from the corresponding edge of the touch-sensitive surface, and wherein the screenshot capture indication is displayed within the control interface.

21. The method of claim 17, wherein the first input type includes movement of a stylus on the touch-sensitive surface of the electronic device away from the edge of the touch-sensitive surface.

22. The method of claim 21, wherein determining that the screenshot capture input is the first input type is at least in part based on determining that the release point of the movement within the touch-sensitive surface is a threshold distance from a target location on the touch-sensitive surface.

23. The method of claim 17, wherein the first input type corresponds to the movement of a stylus away from a corner on the touch-sensitive surface of the electronic device.

24. The method of claim 17, further comprising: The movement of the stylus on the touch-sensitive surface of the electronic device is detected, wherein the movement is away from a corresponding corner of the touch-sensitive surface and originates at a threshold distance from the corresponding corner; as well as In response to the release of the detected movement of the stylus, a screenshot capture menu including a capture screenshot function and an edit screenshot function is displayed via the display device, wherein in response to the detection of a first input pointing to the capture screenshot function, a screenshot image of the content is captured, and wherein in response to the detection of a second input pointing to the edit screenshot function, a screenshot editing interface for editing the screenshot image is displayed via the display device.

25. The method of claim 17, further comprising: When the thumbnail representation of the screenshot image is displayed, a first input pointing to the thumbnail representation of the screenshot image is detected via the input device; as well as In response to detecting the first input, the screenshot editing interface is displayed via the display device.

26. The method according to claim 24, further comprising: When the screenshot editing interface, which includes an opacity level indicator, is displayed: The input device detects a first input pointing to the opacity level indicator, wherein the first input sets the opacity level indicator to a corresponding opacity value; as well as In response to detecting the first input, the opacity of the filter layer overlaid on the screenshot image is changed to the corresponding opacity value.

27. The method according to claim 26, further comprising: In response to detecting the first input, the filter layer overlaid on the annotation of the screenshot image is displayed via the display device.

28. The method according to claim 26, further comprising: In response to detecting the first input, a note on the screenshot image overlaid on the filter layer is displayed via the display device.

29. The method according to claim 26, further comprising: In response to the detection of a second input via the input device pointing to a completion indication included in the screenshot editing interface, a save interface is displayed via the display device; The input device detects a third input pointing to the save interface; as well as In response to the detection of the third input, the screenshot image and the filter layer are stored as a flattened image.

30. The method according to claim 26, further comprising: In response to detecting a second input via the input device pointing to a sharing capability representation included in the screenshot editing interface, the sharing interface is displayed via the display device; The input device detects a third input directed to the sharing interface; as well as In response to the detection of the third input, the screenshot image and the filter layer are stored as an image file, wherein the screenshot image and the filter layer are editable separately.

31. The method of claim 17, wherein the screenshot editing interface further includes a corresponding display, and the method further includes: The first input, directed to the corresponding indicator, is detected via the input device; as well as In response to the detection of the first input, additional content that was not displayed on the display when the screenshot capture input was detected is added to the screenshot editing interface.

32. The method of claim 17, wherein the screenshot editing interface further includes a first drawing palette at a first position within the screenshot editing interface, and wherein, in response to a first input pointing to the first drawing palette, the first drawing palette is movable to a second position within the screenshot editing interface.

33. An electronic device, the electronic device comprising: One or more processors; Non-transitory memory; Input devices; Display devices; and One or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any one of the methods according to claims 17 to 32.

34. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device having one or more processors, input devices, and display devices, cause the electronic device to perform any one of the methods according to claims 17 to 32.

35. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices, the one or more programs comprising instructions for performing the method according to any one of claims 17 to 32.

Images