Gesture detection, list navigation, and item selection using a crown and sensors

Abstract

The present invention is entitled "Gesture Detection, List Navigation, and Item Selection Using Crown and Sensors." The present disclosure relates generally to methods and apparatus for detecting gestures at locations outside of a display on a reduced size electronic device, such as on the housing of the device or on the rotatable input mechanism (e.g., a digital crown) of the device, and responding to gestures by performing, for example, the following operations: navigating a list of items and selecting an item from the list; panning the display of an electronic document; or sending audio control data to an external audio device.

Description

[0001] This application is a divisional application of Chinese Patent Application 201780057626.8, filed on August 31, 2017, entitled "Gesture Detection, List Navigation and Item Selection Using a Crown and Sensor".

[0002] Cross-reference to related applications

[0003] This patent application claims priority to U.S. Provisional Patent Application 62 / 396,372, filed September 19, 2016, entitled “GESTURE DETECTION, LISTNAVIGATION, AND ITEM SELECTION USING ACROWN AND SENSORS,” the contents of which are incorporated herein by reference in their entirety. Technical Field

[0004] This disclosure relates generally to computer user interfaces, and more specifically to techniques for navigating lists and selecting items using digital crowns and sensors. Background Technology

[0005] Compared to other types of electronic devices, scaled-down electronic devices have smaller screen sizes, which can present usability challenges. For example, some scaled-down devices with touchscreen interfaces require users to select items from a list by touching items on the touchscreen. However, with scaled-down screens, users may find it difficult to provide sufficiently precise touch input. Furthermore, providing touch input on a small screen can obscure a relatively large portion of the displayed content, increasing the likelihood of inaccurate or unwanted input and responses. Devices and user interfaces that allow users to provide input outside the display can improve the device's efficiency and usability. Summary of the Invention

[0006] Miniaturized devices that allow users to input from outside the display can alleviate the problem of content being obscured on small screens. However, there are challenges in providing alternative input modes, such as those that do not require the user to touch the display. For example, miniaturized devices are often too small to accommodate a physical keyboard. Some miniaturized devices include one or more physical buttons, but these may not provide users with enough variety of input modes for an efficient interface. For miniaturized devices, especially wearable miniaturized devices such as smartwatches, other external input devices such as mice or trackballs are not attractive solutions. Existing user interfaces, often based on touchscreens, can lead to frequent erroneous device responses because users cannot easily look at the screen while providing input, wasting user time and device power. This latter consideration is particularly important in battery-powered devices.

[0007] Therefore, this technology provides faster and more efficient methods and interfaces for miniaturized electronic devices to implement list navigation, item selection, and other control options. Such methods and interfaces optionally complement or replace other methods used for list navigation and item selection. These methods and interfaces reduce the cognitive burden on the user and result in a more efficient human-machine interface. These methods can reduce the input required to navigate lists and select items (or provide other control inputs), thereby reducing the device's processing and memory usage requirements. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charges.

[0008] According to some embodiments, a method is performed on an electronic device having a display, a housing, and a rotatable input mechanism. The method includes: receiving a request to display items; in response to receiving the request, displaying a user interface comprising a plurality of items on the display, wherein a focus selector is associated with one or more of the plurality of items; detecting contact at a location on the device corresponding to the rotatable input mechanism; in response to the detected contact being at a first position relative to the rotatable input mechanism, performing a process on the one or more items associated with the focus selector; and in response to the detected contact being at a second position relative to the rotatable input mechanism, abandoning the performance of the process on the one or more items associated with the focus selector, wherein the second position is different from the first position.

[0009] According to some embodiments, a method is performed on an electronic device having a display and a rotatable input mechanism. The method includes: displaying a user interface on the display comprising a plurality of items, wherein a focus selector is associated with one or more items in a first group of the plurality of items; detecting user input corresponding to rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism; in response to detecting rotation, scrolling the plurality of items and associating the focus selector with one or more items in a second group based on the rotation; determining whether contact has been removed from the rotatable input mechanism when the second group of one or more items is associated with the focus selector; performing a process on the second group of one or more items based on the determination that contact has been removed when the second group of one or more items is associated with the focus selector; and abandoning the process on the second group of one or more items based on the determination that contact has not been removed when the second item is in focus.

[0010] According to some embodiments, a method is performed on an electronic device having a display and a rotatable input mechanism. The method includes: displaying a first user interface screen on the display; while displaying the first user interface screen, detecting a swipe gesture originating from a position corresponding to the rotatable input mechanism; determining the direction of the swipe relative to the rotatable input mechanism; and replacing the display of the first user interface screen with the display of a second user interface screen based on the determined direction of the swipe relative to the rotatable input mechanism.

[0011] According to some implementations, a method is performed on an electronic device having a display and a rotatable input mechanism. The method includes: displaying a portion of an electronic document on the display; detecting a swipe gesture at a position on the device corresponding to the rotatable input mechanism; determining the direction of the swipe gesture relative to the rotatable input mechanism; and translating the display of the document according to the determined direction of the swipe gesture.

[0012] According to some embodiments, a method is performed in an electronic device having a display, a housing, a rotatable input mechanism, and an audio output communicator. The method includes: detecting a contact on the device at a position corresponding to the rotatable input mechanism when the display is inactive and the device is in an audio control mode, wherein the contact does not activate the display; and providing first audio control data to the audio output communicator in response to the detected contact being at a first position relative to the rotatable input mechanism, wherein the first audio control data is associated with an audio function.

[0013] According to some embodiments, the device includes: a display; a housing; a rotatable input mechanism; one or more processors; a memory; and one or more programs stored in the memory, including instructions that, when executed by the one or more processors, cause the device to: receive a request to display an item; in response to receiving the request, display a user interface comprising a plurality of items on the display, wherein a focus selector is associated with one or more of the plurality of items; detect contact on the device at a position corresponding to the rotatable input mechanism; in response to the detected contact being at a first position relative to the rotatable input mechanism, perform a process on the one or more items associated with the focus selector; and in response to the detected contact being at a second position relative to the rotatable input mechanism, abandon the execution of the process on the one or more items associated with the focus selector, wherein the second position is different from the first position.

[0014] According to some embodiments, the device includes: a display; a rotatable input mechanism; one or more processors; a memory; and one or more programs stored in the memory, including instructions that, when executed by the one or more processors, cause the device to: display a user interface comprising a plurality of items on the display, wherein a focus selector is associated with a first group of one or more items in the plurality of items; detect user input corresponding to rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism; in response to detecting rotation, scroll the plurality of items and associate the focus selector with a second group of one or more items according to the rotation; determine whether contact has been removed from the rotatable input mechanism when the second group of one or more items is associated with the focus selector; perform a process on the second group of one or more items based on the determination that contact has been removed when the second group of one or more items is associated with the focus selector; and abandon the execution of the process on the second group of one or more items based on the determination that contact has not been removed when the second item is in focus.

[0015] According to some embodiments, the device includes: a display; a rotatable input mechanism; one or more processors; a memory; and one or more programs stored in the memory, including instructions that, when executed by the one or more processors, cause the device to: display a first user interface screen on the display; while displaying the first user interface screen, detect a swipe gesture starting from a position corresponding to the rotatable input mechanism; determine the direction of the swipe relative to the rotatable input mechanism; and replace the display of the first user interface screen with the display of a second user interface screen according to the determined direction of the swipe relative to the rotatable input mechanism.

[0016] According to some embodiments, the device includes: a display; a rotatable input mechanism; one or more processors; a memory; and one or more programs stored in the memory, including instructions that, when executed by the one or more processors, cause the device to: display a portion of an electronic document on the display; detect a swipe gesture on the device at a position corresponding to the rotatable input mechanism; determine the direction of the swipe gesture relative to the rotatable input mechanism; and translate the display of the document according to the determined direction of the swipe gesture.

[0017] According to some embodiments, the device includes: a display; a housing; a rotatable input mechanism; an audio output communicator; one or more processors; a memory; and one or more programs stored in the memory, including instructions that, when executed by the one or more processors, cause the device to: detect a contact on the device at a position corresponding to the rotatable input mechanism when the display is inactive and the device is in an audio control mode, wherein the contact does not activate the display; and, in response to the detected contact being at a first position relative to the rotatable input mechanism, provide first audio control data to the audio output communicator, wherein the first audio control data is associated with an audio function.

[0018] According to some embodiments, a non-transitory computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display, a housing, and a rotatable input mechanism. The one or more programs include instructions, when executed by the one or more processors, to cause the electronic device to: receive a request to display an item; in response to receiving the request, display a user interface comprising a plurality of items on the display, wherein a focus selector is associated with one or more of the plurality of items; detect contact on the device at a position corresponding to the rotatable input mechanism; in response to the detected contact being at a first position relative to the rotatable input mechanism, perform a process on the one or more items associated with the focus selector; and in response to the detected contact being at a second position relative to the rotatable input mechanism, abandon the process on the one or more items associated with the focus selector, wherein the second position is different from the first position.

[0019] According to some embodiments, a non-transitory computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs including instructions, when executed by the one or more processors, to cause the electronic device to: display a user interface comprising a plurality of items on the display, wherein a focus selector is associated with a first group of one or more items in the plurality of items; detect user input corresponding to rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism; in response to detecting rotation, scroll the plurality of items and associate the focus selector with a second group of one or more items according to the rotation; determine whether contact has been removed from the rotatable input mechanism when the second group of one or more items is associated with the focus selector; perform a process on the second group of one or more items based on the determination that contact has been removed when the second group of one or more items is associated with the focus selector; and abandon the execution of the process on the second group of one or more items based on the determination that contact has not been removed when the second item is in focus.

[0020] According to some embodiments, a non-transitory computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs including instructions, when executed by the one or more processors, to cause the electronic device to: display a first user interface screen on the display; while displaying the first user interface screen, detect a swipe gesture starting from a position corresponding to the rotatable input mechanism; determine the direction of the swipe relative to the rotatable input mechanism; and replace the display of the first user interface screen with the display of a second user interface screen according to the determined direction of the swipe relative to the rotatable input mechanism.

[0021] According to some embodiments, a non-transitory computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs including instructions, when executed by the one or more processors, to cause the electronic device to: display a portion of an electronic document on the display; detect a swipe gesture on the device at a position corresponding to the rotatable input mechanism; determine the direction of the swipe gesture relative to the rotatable input mechanism; and translate the display of the document according to the determined direction of the swipe gesture.

[0022] According to some embodiments, a non-transitory computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display, a housing, a rotatable input mechanism, and an audio output communicator. The one or more programs include instructions, when executed by the one or more processors, to cause the electronic device to: detect a contact on the device at a position corresponding to the rotatable input mechanism when the display is inactive and the device is in an audio control mode, wherein the contact does not activate the display; and, in response to the detected contact being at a first position relative to the rotatable input mechanism, provide first audio control data to the audio output communicator, wherein the first audio control data is associated with an audio function.

[0023] According to some embodiments, a transient computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display, a housing, and a rotatable input mechanism. The one or more programs include instructions, when executed by the one or more processors, to cause the electronic device to: receive a request to display an item; in response to receiving the request, display a user interface comprising a plurality of items on the display, wherein a focus selector is associated with one or more of the plurality of items; detect contact on the device at a position corresponding to the rotatable input mechanism; in response to the detected contact being at a first position relative to the rotatable input mechanism, perform a process on the one or more items associated with the focus selector; and in response to the detected contact being at a second position relative to the rotatable input mechanism, abandon the process on the one or more items associated with the focus selector, wherein the second position is different from the first position.

[0024] According to some embodiments, a transient computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs including instructions, when executed by the one or more processors, to cause the electronic device to: display a user interface comprising a plurality of items on the display, wherein a focus selector is associated with a first group of one or more items in the plurality of items; detect user input corresponding to rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism; in response to detecting rotation, scroll the plurality of items and associate the focus selector with a second group of one or more items according to the rotation; determine whether contact has been removed from the rotatable input mechanism when the second group of one or more items is associated with the focus selector; perform a process on the second group of one or more items based on the determination that contact has been removed when the second group of one or more items is associated with the focus selector; and abandon the execution of the process on the second group of one or more items based on the determination that contact has not been removed when the second item is in focus.

[0025] According to some embodiments, a transient computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs including instructions, when executed by the one or more processors, to cause the electronic device to: display a first user interface screen on the display; while displaying the first user interface screen, detect a swipe gesture starting from a position corresponding to the rotatable input mechanism; determine the direction of the swipe relative to the rotatable input mechanism; and replace the display of the first user interface screen with the display of a second user interface screen according to the determined direction of the swipe relative to the rotatable input mechanism.

[0026] According to some embodiments, a transient computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs including instructions, when executed by the one or more processors, to cause the electronic device to: display a portion of an electronic document on the display; detect a swipe gesture on the device at a position corresponding to the rotatable input mechanism; determine the direction of the swipe gesture relative to the rotatable input mechanism; and translate the display of the document according to the determined direction of the swipe gesture.

[0027] According to some embodiments, a transient computer-readable storage medium includes one or more programs executable by one or more processors of an electronic device having a display, a housing, a rotatable input mechanism, and an audio output communicator. The one or more programs include instructions, when executed by the one or more processors, to cause the electronic device to: detect a contact on the device at a position corresponding to the rotatable input mechanism when the display is inactive and the device is in an audio control mode, wherein the contact does not activate the display; and, in response to the detected contact being at a first position relative to the rotatable input mechanism, provide first audio control data to the audio output communicator, wherein the first audio control data is associated with an audio function.

[0028] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0029] Therefore, faster and more efficient methods and interfaces are provided for list navigation and item selection, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods used for list navigation and item selection. Attached Figure Description

[0030] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0031] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.

[0032] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0033] Figure 2A portable multi-functional device with a touchscreen is shown according to some embodiments.

[0034] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0035] Figure 4A An exemplary user interface for a menu on an application on a portable multifunction device, according to some implementation schemes, is shown.

[0036] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0037] Figure 5A A personal electronic device according to some implementation schemes is shown.

[0038] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0039] Figures 6A to 6V An exemplary user interface according to some implementation schemes is shown.

[0040] Figure 6W An exemplary device block diagram according to some implementation schemes is shown.

[0041] Figure 6X An exemplary user interface according to some implementation schemes is shown.

[0042] Figures 7 to 9 A flowchart is shown illustrating an exemplary method for detecting gestures, navigation lists, and selecting items according to some implementation schemes.

[0043] Figure 10 A flowchart of an exemplary method for detecting gestures and navigating electronic documents, according to some implementation schemes, is shown.

[0044] Figure 11 A flowchart of an exemplary method for detecting gestures and controlling audio functions according to some implementation schemes is shown.

[0045] Figures 12 to 16 A functional block diagram of an electronic device according to some implementation schemes is shown. Detailed Implementation

[0046] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0047] Electronic devices requiring reduced size need to provide efficient methods and interfaces for navigating lists and selecting one or more items without requiring the user to touch a touchscreen. For example, as mentioned above, for a reduced-size device with a small display, it may be advantageous to allow the user to navigate a list and select one or more items without having to touch the display and possibly obscure the items on the list. Another advantage of allowing the user to provide input outside the display is that, in some cases, the device can receive user input and respond to it without activating the display, thus saving power.

[0048] under, Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5B Provides functions for gesture detection, list navigation, item selection, and about. Figures 6A to 6X and Figures 7 to 11 Description of exemplary devices for other control technologies described. Figure 6A middle Figure 6X An exemplary user interface is shown for gesture detection, list navigation, item selection, document panning, and audio function control. Figures 7 to 11 This is a flowchart illustrating methods for gesture detection, list navigation and item selection, document panning, and external audio device control according to some implementation schemes. Figures 6A to 6X The user interface and diagrams in the document are used to illustrate the processes described below, including Figures 7 to 11 The process in.

[0049] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first touch may be named a second touch and similarly, a second touch may be named a first touch, without departing from the scope of the various described embodiments. Both the first touch and the second touch are touches, but they are not the same touch.

[0050] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “includes”, “including”, “comprises”, and / or “comprising”, when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0051] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "at," or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if determination..." or "if detection [the stated condition or event]" may optionally be interpreted as meaning "in response to determination..." or "in response to detection [the stated condition or event]."

[0052] This document describes implementations of electronic devices, user interfaces for such devices, and related processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, those from Apple Inc. (Cupertino, California). Devices, iPod Equipment, and Device. Optionally, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0053] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0054] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0055] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0056] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as a touch-sensitive display system. Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, an antenna 194, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0057] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values ​​that includes at least four different values ​​and more typically hundreds of different values ​​(e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or variation of the contact area detected on the touch-sensitive surface, the capacitance and / or variation of the touch-sensitive surface near the contact, and / or the resistance and / or variation of the touch-sensitive surface near the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold (e.g., the intensity threshold is described in units corresponding to the substitute measurement) has been exceeded. In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold (e.g., the intensity threshold is a pressure threshold measured in units of pressure) has been exceeded. Using the intensity of the contact as an attribute of user input allows the user to access additional device functions that would otherwise be inaccessible to the user on smaller devices with limited physical space, such smaller devices being used (e.g., on a touch-sensitive display) to display power indications and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0058] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement does not move. For example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or sensed by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user.

[0059] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0060] Memory 102 optionally includes one or more computer-readable storage media. The computer-readable storage media are optionally tangible and non-transitory. The computer-readable storage media are optionally transient. Memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls other components of device 100 to access memory 102.

[0061] Peripheral interface 118 can be used to couple the device's input and output peripherals to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In other embodiments, they are optionally implemented on separate chips.

[0062] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems (including antenna 194), RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates with networks and other devices wirelessly, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as through near-field communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0063] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset with both output (e.g., a mono or binaural headphone) and input (e.g., a microphone).

[0064] I / O subsystem 106 couples input / output peripherals on device 100, such as touchscreen 112 and other input control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / send electrical signals to other input control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click dials, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing devices such as a mouse. One or more buttons (e.g., ... Figure 2 Optionally, 208) includes volume up / down buttons for volume control of speaker 111 and / or microphone 113. One or more buttons optionally include push-button buttons (e.g., Figure 2 (206 in the middle).

[0065] A quick press of the push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of the push button (e.g., 206) optionally powers the device 100 on or off. The function of one or more buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0066] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0067] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0068] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that from Apple Inc. (Cupertino, California). and iPod The technology used.

[0069] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles a multi-touch touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0070] In some embodiments, the touchscreen 112 has a touch-sensitive display as described in the following patent applications: (1) U.S. Patent Application 11 / 381,313, filed May 2, 2006, entitled “Multipoint Touch Surface Controller”; (2) U.S. Patent Application 10 / 840,862, filed May 6, 2004, entitled “Multipoint Touchscreen”; (3) U.S. Patent Application 10 / 903,964, filed July 30, 2004, entitled “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application 11 / 048,264, filed January 31, 2005, entitled “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application 18, 2005, entitled “Mode-Based Graphical User Interfaces For Touch Sensitive”. U.S. Patent Application No. 11 / 038,590 entitled “Input Devices”; (6) U.S. Patent Application No. 11 / 228,758 entitled “Virtual Input Device Placement On A Touch Screen User Interface”, filed September 16, 2005; (7) U.S. Patent Application No. 11 / 228,700 entitled “Operation Of A Computer With A Touch Screen Interface”, filed September 16, 2005; (8) U.S. Patent Application No. 11 / 228,737 entitled “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard”, filed September 16, 2005; and (9) U.S. Patent Application No. 11 / 367,749 entitled “Multi-Functional Hand-Held Device”, filed March 3, 2006. The entire contents of all these applications are incorporated herein by reference.

[0071] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally interact with touchscreen 112 using any suitable object or accessory such as a stylus, finger, etc. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.

[0072] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0073] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0074] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 is shown coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensor within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0075] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor 165 is shown coupled to a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0076] The device 100 may optionally also include one or more proximity sensors 166. Figure 1AA proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally perform as described in the following U.S. patent applications: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, the proximity sensor is turned off and the touchscreen 112 is disabled when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).

[0077] The device 100 may optionally also include one or more tactile output generators 167. Figure 1A A haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components; and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymerizers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be sensed by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., a haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0078] The device 100 optionally also includes one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally perform as described in the following U.S. patent publications: U.S. Patent Publication 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and U.S. Patent Publication 20060017692, entitled "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. In addition to one or more accelerometers 168, device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the position and orientation (e.g., longitudinal or lateral) of device 100.

[0079] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application program (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the diagram. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, which indicates what applications, views or other information occupy various areas of the touchscreen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and positional information about the device's position and / or orientation.

[0080] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0081] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is connected to… (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.

[0082] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0083] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0084] The touch / motion module 130 optionally detects the user's gesture input. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0085] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other display, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual characteristics). As used herein, the term "graphics" includes any object that can be displayed to a user, and non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0086] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like to specify the graphic to be displayed, and, if necessary, also receives coordinate data and other graphic attribute data, and then generates screen image data for output to the display controller 156.

[0087] The haptic feedback module 133 includes various software components for generating instructions that are used by one or more haptic output generators 167 to produce haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0088] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).

[0089] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to phone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather desktop apps, local yellow pages desktop apps, and map / navigation desktop apps).

[0090] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0091] • Contacts module 137 (sometimes called address book or contact list);

[0092] • Telephone module 138;

[0093] • Video conferencing module 139;

[0094] • Email client module 140;

[0095] • Instant Messaging (IM) module 141;

[0096] Fitness support module 142;

[0097] • Camera module 143 for still images and / or video images;

[0098] • Image management module 144;

[0099] • Video player module;

[0100] Music player module;

[0101] • Browser module 147;

[0102] • Calendar module 148;

[0103] • Desktop mini-program module 149, which optionally includes one or more of the following: weather desktop mini-program 149-1, stock market desktop mini-program 149-2, calculator desktop mini-program 149-3, alarm clock desktop mini-program 149-4, dictionary desktop mini-program 149-5, and other desktop mini-programs obtained by the user, and desktop mini-programs created by the user 149-6;

[0104] • Desktop app creator module 150 for creating user-created desktop apps 149-6;

[0105] • Search module 151;

[0106] • Video and music player module 152, which combines a video player module and a music player module;

[0107] • Notepad module 153;

[0108] • Map module 154; and / or

[0109] • Online video module 155.

[0110] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0111] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting one or more names from the address book; associating one or more phone numbers, one or more email addresses, one or more physical addresses, or other information with a name; associating an image with a name; categorizing and classifying names; providing a phone number or email address to initiate and / or facilitate communication via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0112] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify entered telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0113] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0114] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0115] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing a received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0116] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for: creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0117] In conjunction with the touchscreen 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0118] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0119] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.

[0120] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0121] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, desktop applet module 149 is optionally a micro-application downloaded and used by a user (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) or a user-created micro-application (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop applet includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop applet).

[0122] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, the desktop applet creator module 150 can optionally be used by the user to create desktop applets (e.g., converting user-specified portions of a webpage into desktop applets).

[0123] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0124] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0125] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0126] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is optionally used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0127] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following operations: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Further descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0128] Each of the above modules and applications corresponds to an executable instruction set for performing one or more of the functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods as described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A(e.g., video and music player module 152). In some embodiments, memory 102 optionally stores subgroups of the above-described modules and data structures. Additionally, memory 102 optionally stores other modules and data structures not described above.

[0129] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on device 100 can be optionally reduced.

[0130] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, home menu, or root menu. In such embodiments, a "menu button" is implemented using a touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0131] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 137 to 151, 155, 380 to 390).

[0132] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0133] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0134] Event monitor 171 receives event information from peripheral interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripheral interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0135] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other implementations, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0136] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0137] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0138] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0139] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0140] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0141] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0142] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0143] In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit (not shown) or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0144] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies events from the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0145] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device pose).

[0146] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event (187) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on the displayed object for a predetermined duration, movement of the touch on the touch-sensitive display 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0147] In some implementations, event definition 187 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0148] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0149] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0150] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0151] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some implementations, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with that tag retrieves the tag and executes a predefined procedure.

[0152] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined procedure.

[0153] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on touch-sensitive display.

[0154] In some implementations, one or more event handlers 190 include or have access to a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, the data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0155] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0156] Figure 2A portable multifunction device 100 with a touchscreen 112 is shown according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0157] Device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0158] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0159] Figure 3This is a block diagram of an exemplary multifunctional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 167 and sensors 359 (e.g., optical sensors, accelerometers, proximity sensors, touch sensors, and / or contact intensity sensors similar to those mentioned above) are described. Figure 1A The one or more contact strength sensors 165). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from one or more CPUs 310. In some embodiments, memory 370 stores data with portable multifunction device 100 (…). Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0160] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each of the modules described above corresponds to an instruction set for performing the functions described above. The modules or programs (e.g., instruction sets) described above need not be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subgroup of the modules and data structures described above. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0161] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.

[0162] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is shown. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof:

[0163] • One or more signal strength indicators 402 for wireless communications such as cellular signals and Wi-Fi signals;

[0164] • Time 404;

[0165] Bluetooth indicator 405;

[0166] • Battery status indicator 406;

[0167] • Tray tray 408 with icons for commonly used applications, such as:

[0168] ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 for the number of missed calls or voicemails;

[0169] ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails;

[0170] ○ The icon 420 labeled "Browser" in browser module 147; and

[0171] ○ The icon 422 marked "iPod" for the video and music player module 152 (also known as the iPod (a trademark of Apple Inc.) module 152); and

[0172] • Icons of other applications, such as:

[0173] ○The icon 424 of the IM module 141 marked as "Message";

[0174] ○The calendar module 148 has an icon 426 labeled "Calendar";

[0175] ○ The icon 428 of the image management module 144 is labeled "Photo".

[0176] ○ The icon 430 of camera module 143, which is labeled "camera";

[0177] ○ The icon 432 of the online video module 155, which is labeled "Online Video";

[0178] ○ The icon 434 in the Stock Market Desktop Mini Program 149-2 that is marked as "Stock Market";

[0179] ○The icon 436 of the map module 154 that is labeled "map";

[0180] ○ The icon 438 in the Weather desktop mini-program 149-1 that is marked as "Weather";

[0181] ○ The icon 440 in the alarm clock desktop mini-program 149-4 that is marked as "clock";

[0182] ○ The icon 442 of the fitness support module 142 is labeled "fitness support";

[0183] ○ The icon 444 labeled "Notepad" in Notepad module 153; and

[0184] ○ The icon 446, labeled "Settings," is used to set the settings of the device 100 and its various applications 136.

[0185] It should be pointed out that, Figure 4A The icon labels shown are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0186] Figure 4B A touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)) is shown. Figure 3 Devices (e.g., tablets or touchpads 355) Figure 3An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451, and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.

[0187] While some examples of input on a reference touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations the device detects input on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450)). Figure 4B The spindle corresponding to 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to the corresponding position on the display (e.g., in the example). Figure 4B In the middle, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, on touch-sensitive surfaces (e.g., Figure 4B 451) and the display of the multi-functional device ( Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on the touch-sensitive surface (e.g., touches on 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0188] Additionally, while the examples below are primarily given with reference to finger input (e.g., finger touch, single-finger tap, finger swipe), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the swipe path (e.g., instead of movement of the touch). Similarly, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Likewise, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0189] Figure 5AAn exemplary personal electronic device 500 is illustrated. Device 500 includes a housing 502 for enclosing a portion of device 500. Housing 502 may include a border of device 500 (if present). In some embodiments, device 500 may include components relative to devices 100 and 300 (e.g., Figures 1A to 4B Some or all of the features described herein. In some embodiments, device 500 has a touch-sensitive display 504, referred to below as touchscreen 504. Touchscreen 504 is not part of housing 502. As an alternative to or complement to touchscreen 504, device 500 has a display and a touch-sensitive surface; the display and touch-sensitive surface (if included) are not part of housing 502. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., touch). One or more intensity sensors of touchscreen 504 (or touch-sensitive surface) can provide output data representing the intensity of the touch. The user interface of device 500 can respond to touches based on the intensity of the touch, meaning that touches of different intensities can invoke different user interface operations on device 500.

[0190] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related patent applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application,” published under WIPO Patent Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships,” published under WIPO Patent Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0191] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) may be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0192] Figure 5B An exemplary personal electronic device 500 is illustrated. In some embodiments, device 500 may include a reference. Figure 1A , Figure 1B and Figure 3 Some or all of the components described herein. Device 500 has a bus 512 that operatively couples I / O section 514 to one or more computer processors 516 and memory 518. I / O section 514 may be connected to touchscreen 504, which may have touch-sensitive components 522 and optionally have intensity sensors 524 (e.g., contact intensity sensors). Furthermore, I / O section 514 may be connected to a communication unit 530 including receiver 544 and / or transmitter 546 for receiving and / or transmitting application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. In some embodiments, receiver 544 and transmitter 546 are contained in a single component such as a transceiver. In some embodiments, communication unit 530 may include RF circuitry 108, such as relative to… Figure 1A The device 500 may include input mechanisms 506 and / or 508. Input mechanism 506 is optionally a rotatable input mechanism or a pressable and rotatable input mechanism, such as a digital crown on a watch. In some examples, input mechanism 508 is optionally a button.

[0193] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.

[0194] In some embodiments, device 500 includes one or more contact sensors 542 configured to detect contacts on device 500 other than touchscreen 504, such as contacts on housing 502, contacts on input mechanism 506, and / or contacts on input mechanism 508. In some embodiments, contact sensor 542 is configured to detect the position of the contact relative to input mechanism 506. For example, in some embodiments, contact sensor 542 is configured to detect a touch, tap, or double tap received on housing 502 located above or below input mechanism 506.

[0195] In some embodiments, the contact sensor 542 is configured to detect a swipe gesture on the housing 502 or the input mechanism 506. In some embodiments, the contact sensor 542 is configured to determine the direction of the swipe gesture relative to the input mechanism 506.

[0196] In some embodiments, the contact sensor 542 is configured to detect contact at any location on the housing 502. In some embodiments, the contact sensor 542 is configured to detect contact on a specific area of ​​the housing 502, such as a contact on the portion of the housing 502 located on the same side of the device as the input mechanism 506.

[0197] In some embodiments, the contact sensor 542 includes a capacitive sensor. In some embodiments, the contact sensor 542 includes an optical sensor. In some embodiments, the contact sensor 542 includes an accelerometer such as an accelerometer 534. In some embodiments, the contact sensor 542 includes an antenna such as... Figure 1A Antenna 194 in the middle.

[0198] The contact sensor 542 may be located at various locations on the device 500. In some embodiments, the contact sensor 542 is located below the touchscreen 502. In some embodiments, the contact sensor 542 is located below the housing 502. In some embodiments, the contact sensor 542 is located within the input mechanism 506.

[0199] In some implementations, the physical location of the contact sensor 542 on the device 500 does not specify a particular area on the device 500 where the contact sensor 542 can detect contact; for example, the contact sensor 542 may be physically located below the touch screen 504 and configured to detect contact on the housing 502 and / or the input mechanism 506.

[0200] In some implementations, in addition to being configured to detect contacts on housing 502 and / or input mechanism 506, contact sensor 542 is also configured to detect contacts on touchscreen 502.

[0201] The memory 518 of the personal electronic device 500 may be a non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause the computer processor to perform, for example, processes 700 to 1100. Figures 7 to 11 The computer-executable instructions may also be stored and / or transmitted in any non-transitory computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a processor-included system, or other system that may retrieve and execute instructions from and execute instructions. For the purposes of this document, a “non-transitory computer-readable storage medium” can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices 500 are not limited to Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.

[0202] As used herein, the term "power indication" refers optionally to the power indication in devices 100, 300, and / or 500 ( Figures 1A to 1B , Figure 3 and Figures 5A to 5B A user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.

[0203] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of enabling direct interaction with user interface elements on the touchscreen display) Figure 1A The touch-sensitive display system 112 or Figure 4AIn some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0204] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted away, before or after contact begins to move, before contact ends, before or after contact intensity is detected to increase and / or before or after contact intensity decreases). The characteristic intensity of the contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half maximum value of the contact intensity, the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some implementations, the feature intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a feature intensity not exceeding the first threshold results in a first action, contact with a feature intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a feature intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the feature intensity and one or more thresholds is used to determine whether to perform one or more actions (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0205] In some implementations, a portion of the gesture is identified to determine the characteristic intensity. For example, a touch-sensitive surface optionally receives a series of swipe contacts that transition from a starting position to an ending position, where the contact intensity increases. In this example, the characteristic intensity of the contact at the ending position is optionally based only on a portion of the series of swipe contacts, rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some implementations, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: unweighted moving average smoothing algorithm, triangular smoothing algorithm, median filter smoothing algorithm, and / or exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contact to achieve the purpose of determining the characteristic intensity.

[0206] Optionally, the contact intensity on a touch-sensitive surface is characterized relative to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device performs an operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device performs an operation different from the operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, when a contact with a characteristic intensity lower than the light press intensity threshold (e.g., and higher than the nominal contact detection intensity threshold, where contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector based on the movement of the contact on the touch-sensitive surface without performing the operation associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent across different groups of user interface figures.

[0207] An increase in contact intensity from below a light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in contact intensity from below a deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in contact intensity from below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in contact intensity from above a contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a contact being lifted off the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0208] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the intensity of the contact (or multiple contacts) increases to above a press input intensity threshold. In some embodiments, the corresponding operation is performed in response to detecting that the intensity of the corresponding contact increases to above a press input intensity threshold (e.g., a "downward stroke" of the corresponding press input). In some embodiments, the press input includes the intensity of the corresponding contact increasing to above a press input intensity threshold and the intensity of the contact subsequently decreasing to below the press input intensity threshold, and the corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the press input threshold (e.g., an "upward stroke" of the corresponding press input).

[0209] In some implementations, the device employs intensity hysteresis to avoid unintended inputs sometimes referred to as "jitter," wherein the device defines or selects a hysteresis intensity threshold that has a predefined relationship with a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the press input intensity threshold). Therefore, in some implementations, a press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of that contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., an "upstroke" of the corresponding press input). Similarly, in some implementations, a press input is detected only when the device detects that the contact intensity increases from an intensity equal to or below the hysteresis intensity threshold to an intensity equal to or above the press input intensity threshold and optionally the contact intensity subsequently decreases to an intensity equal to or below the hysteresis intensity threshold, and a corresponding operation is performed in response to detecting a press input (e.g., an increase or decrease in contact intensity depending on the environment).

[0210] For ease of explanation, optionally, the description of an operation triggered in response to a press input associated with a press input strength threshold or in response to a gesture including a press input is provided in response to detecting any of the following conditions: the contact strength increases to above the press input strength threshold, the contact strength increases from below a hysteresis strength threshold to above the press input strength threshold, the contact strength decreases to below the press input strength threshold, and / or the contact strength decreases to below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in the example where the operation is described as being performed in response to detecting a decrease in contact strength below the press input strength threshold, the operation is optionally performed in response to detecting a decrease in contact strength below a hysteresis strength threshold corresponding to and less than the press input strength threshold.

[0211] As used herein, "installed application" refers to a software application that has been downloaded to an electronic device (e.g., device 100, 300, and / or 500) and is ready to be launched on the device (e.g., become open). In some implementations, the downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the computer system's operating system.

[0212] As used herein, the terms "open application" or "running application" refer to a software application that maintains state information (e.g., as part of device / global internal state 157 and / or application internal state 192). An open or running application is optionally any of the following types of applications:

[0213] The active application currently displayed on the screen of the device that is using the application;

[0214] Background applications (or background processes) that are not currently displayed but whose one or more processes are being handled by one or more processors; and

[0215] A suspended or hibernating application that is not running but has state information stored in memory (both volatile and non-volatile) that can be used to resume the execution of the application.

[0216] As used herein, the term "closed application" refers to a software application that does not retain state information (e.g., the state information of a closed application is not stored in the device's memory). Therefore, closing an application includes stopping and / or removing the application's process and removing the application's state information from the device's memory. Generally, opening a second application while the first application is running does not close the first application. When the second application is displayed and the first application stops displaying, the first application becomes a background application.

[0217] Now let’s turn our attention to the implementation of user interfaces (“UIs”) and associated processes on electronic devices (such as portable multifunction devices 100, 300 or 500).

[0218] Figures 6A to 6K The device 500 is shown to perform a process that enables a user to navigate a list of items (or other groups) and select one or more items in the list by touching a location outside the display of the device 500; for example, by touching the device 500 on the housing 502 or the rotatable input mechanism 506. In some embodiments, the housing 502 and the rotatable input mechanism 506 of the device 500 provide touch-sensitive areas on which a user can provide touch input to the device 500 without having to touch the touchscreen 504 and without causing mechanical actuation of any input mechanisms on the device 500 such as input mechanisms 506, 508. In some embodiments, the device 500 uses a contact sensor 542 to detect contact on the housing 502 and the rotatable input mechanism 506, as previously described. Figure 5B As mentioned above.

[0219] Figures 6A to 6K The user interface depicted in the image is used to illustrate the following: Figure 7 The process described below is the process in the middle.

[0220] As will be discussed in more detail below, in some embodiments, device 500 may respond differently to contacts on housing 502 or on rotatable input mechanism 506, depending on the position of the contacts relative to rotatable input mechanism 506. For example, device 500 may respond differently to contacts on housing 502 located above rotatable input mechanism 506 than to contacts on housing 502 located below rotatable input mechanism 506. Similarly, device 500 may respond differently to contacts on housing 502 than to contacts on rotatable input mechanism 506.

[0221] Figure 6A An exemplary user interface 606 is depicted, which is optionally displayed by device 500 in response to a user request that a display item is detected. For example, a user request may be a touch on touchscreen 504 (e.g., selection of an icon), activation of button 508, rotation of rotatable input mechanism 506, or movement of device 500 such as the user raising device 500 into their field of vision (detected, for example, by an accelerometer on device 500).

[0222] User interface 606 includes an item list 608 and a focus selector 604 associated with one of the items. In the exemplary user interface 606, the item list 608 is a list of replies from which a user can select a reply to send as an email to another user. Figure 6A In this context, focus selector 604 is associated with the "Yes" response.

[0223] When displaying list 608, in response to detecting a contact corresponding to the rotatable input mechanism 506 at a first position on device 500 relative to the rotatable input mechanism 506, device 500 sends a response associated with focus selector 604 as an electronic message to an external device, such as... Figure 6B The device 600 shown is an example. For instance, an external device could be a device associated with another user. Figure 6A In the example depicted, when... Figure 6A When viewed in the orientation depicted, the contact corresponding to the rotatable input mechanism 506 is a finger touch on the housing 502 at a position 602 above the rotatable input mechanism 506 (e.g., closer to the top of the device than the bottom). Therefore, the contact corresponding to the rotatable input mechanism 506 does not need to be on the rotatable input mechanism 506 itself.

[0224] In some implementations, the contact corresponding to the rotatable input mechanism is part of a gesture that includes a tap. For example, the contact may include a single tap or a double tap.

[0225] like Figure 6AAs shown in the examples, in some embodiments, a contact corresponding to the rotatable input mechanism 506 does not activate a mechanically actuated input mechanism on the device 500 (e.g., it does not cause a press of the button 508 or a rotation of the rotatable input mechanism 506). That is, the contact can be a touch, tap, double-tap, or swipe on the housing 502, or a touch, tap, double-tap, or swipe on the input mechanisms 506, 508, which does not cause mechanical actuation of the input mechanisms 506, 508.

[0226] like Figure 6A As shown in the examples, in some embodiments, the contact corresponding to the rotatable input mechanism does not include any contact with the touchscreen 504; for example, the contact is completely detached from the display. In some embodiments, the contact may include a portion of the contact on the display and a portion of the contact outside the display, such as a swipe that begins on the touchscreen 504 and ends on the housing 502.

[0227] Back Figure 6A As previously described, in response to detecting contact on the housing 502 at a first position relative to the rotatable input mechanism 506 (in this case, at position 602 above the rotatable input mechanism 506), the device 500 sends a response (“Yes”) associated with the focus selector 604 as an electronic message to the external device 600. Therefore, Figures 6A to 6B The device 500 is shown to perform a process that allows a user to navigate a list and select an item to process. In this case, processing the item includes sending the item as an electronic message to an external device.

[0228] In some implementations, the device 500 may perform different processes depending on the position of contact corresponding to the rotatable input mechanism 506. For example, when displaying the user interface 606, if the device 500 detects contact at a second position relative to the rotatable input mechanism 506 (rather than as...), Figure 6A If the device 500 is in position 602 as shown in the figure, then the device 500 may abandon sending the response associated with the focus selector 604.

[0229] like Figure 6C As shown, in some embodiments, in response to detecting contact at a second position 612 relative to the rotatable input mechanism 506, the device 500 performs a process different from the process performed by the device 500 in response to detecting contact at a first position 602. Figure 6C In the example, the second process includes replacing the display of user interface 606 with the display of user interface 610.

[0230] Therefore, in some implementations, if a user touches the housing of device 500 at a first position relative to the rotatable input mechanism 506 while device 500 is displaying list 608, device 500 responds by performing a first process on one or more items associated with focus selector 604, such as... Figures 6A to 6B As shown in the diagram. However, if the user touches the housing in a second position (different from the first position) relative to the rotatable input mechanism 506, the device 500 responds by performing a second process, such as... Figures 6C to 6D As shown in the image.

[0231] In this example, the second process is independent of one or more items associated with the focus selector. However, in some implementations, device 500 performs the second process on one or more items associated with focus selector 604. For example, device 500 may alternatively perform the second process on the items associated with focus selector 604, such as... Figures 6E to 6F As shown, instead of using Figure 6C The display of user interface 610 replaces the display of user interface 606. In this example, in response to a detected contact at position 612, the device performs a second process that includes deassigning the item (“Yes”) associated with focus selector 604 by removing it from the displayed list, and changing the item associated with focus selector 604 to “No”.

[0232] In some implementations, the first and / or second processes include deselecting items not associated with the focus selector (e.g., in...). Figure 6E In this context, uncheck "No" and "I'll call you," instead of unchecking "Yes," as mentioned above. For example, unchecking an item could include removing it from a list, removing it from the display, or deleting it.

[0233] In some implementations, the first and / or second processes include selecting one or more items associated with the focus selector for further processing; for example, placing the item in a text field, displaying the item in a full-screen view, or opening an application associated with the item. For example, if the list of items is a list of email addresses, the first and / or second processes may include placing the email address associated with the focus selector in a recipient text field or launching an email application.

[0234] As previously mentioned, in some embodiments, performing the second process includes displaying a second user interface; for example, by using a second user interface such as Figures 6C to 6DThe image depicts a replacement (first) user interface, where user interface 606 is replaced by user interface 610. In some embodiments, the second user interface is associated with an open application, such as an application running in the background of device 500. In some embodiments, the second user interface is associated with an application launched as part of performing a second process.

[0235] Back Figures 6G to 6I In some implementations, in response to detecting scroll input while displaying user interface 606, device 500 scrolls the item list and changes one or more items associated with the focus selector based on the scroll input. In this example, the scroll input is the rotation of the rotatable input mechanism 506, and as... Figures 6G to 6H As shown, the scrolling list changes the item associated with focus selector 604 from "Yes" to "No". In some implementations, the scrolling input can be a swipe on touchscreen 504 or a swipe on housing 502.

[0236] In this example, device 500 allows the user to scroll through a list of items by rotating the rotatable input mechanism 506, and then select the item to process (“No”) by touching the housing 502, as shown. Figure 6H As shown. In response to detecting a contact at position 602 on housing 502, device 500 sends an item (“No”) associated with focus selector 604 as an electronic message to an external device, such as Figure 6I The device 600 shown is shown.

[0237] exist Figures 6A to 6H In the illustrated example, user interface 606 includes multiple text replies displayed in a list format. In other examples, such as device 500, multiple items may be displayed in a different group format, such as a carousel or array. The multiple items displayed may include multiple text strings (as in user interface 606), or multiple images, messages, menus, contacts, icons, applications, etc. Therefore, refer to... Figures 6A to 6H The aforementioned process applies to navigating any number of displayed items and selecting an item.

[0238] exist Figures 6A to 6I In the example depicted, focus selector 604 is an ellipse enclosing items in the list. In other embodiments, the focus selector may be another form of visual indicator, such as a highlight, outline, etc., associated with one or more displayed items.

[0239] Figures 6J to 6K The above reference is shown Figures 6A to 6I Describe a similar process. In Figure 6JIn the example, device 500 displays multiple items in user interface 626, where in this case, the multiple items are a list of notifications 618, 620, 622, and 624. In this example, the focus selector is highlighted in gray, and the item associated with the focus selector is notification 620. In response to detecting a contact on the rotatable input mechanism 506 corresponding to the rotatable input mechanism 506, device 500 launches an application associated with the highlighted text message, such as... Figure 6K As shown. In this example, the contact is a double-click on the rotatable input mechanism 506, which does not cause mechanical actuation (such as rotation) of the rotatable input mechanism 506.

[0240] Figure 7 This is a flowchart illustrating a process for navigating a list of items and selecting one or more items for processing using a scaled-down electronic device, according to some embodiments. Method 700 is performed at a device (e.g., 100, 300, 500) having a display, housing, and rotatable input mechanism. Some operations in method 700 are optionally combined; the order of some operations is optionally changed, and some operations are optionally omitted.

[0241] As described below, process 700 provides users with an intuitive way to navigate a list of items (or multiple items) and select one or more items for processing on a scaled-down device. This method creates a more efficient human-computer interface by reducing the cognitive burden on users when navigating the list and selecting items by allowing them to provide a series of inputs without obscuring the display. For battery-powered computing devices, this enables users to efficiently navigate lists and select items, saving power and increasing the time interval between battery charges.

[0242] At box 702, the device receives a request to display an item. For example, the request could be a touch on touchscreen 504 (such as the selection of an icon), a touch at another location on the device, movement of the device detected by an accelerometer, or activation of a mechanically actuated input mechanism such as a button or a rotatable input mechanism.

[0243] At box 704, in response to a received request, the device displays a user interface comprising multiple items, wherein a focus selector is associated with one or more of those items. (See above regarding...) Figures 6A to 6I As described, these multiple items can be a reply list, a notification list, an array of images, a set of icons representing the application, a set of menus, etc. For example, a focus selector could be a line surrounding one or more items, such as... Figure 6A As shown, or highlighted around the project, such as Figure 6J As shown.

[0244] At frame 706, the device detects a contact on the device at a location corresponding to the rotatable input mechanism. As previously mentioned, the contact corresponding to the rotatable input mechanism does not need to be located on the rotatable input mechanism. In some embodiments, this contact is an external contact to the display, such as a contact on the housing or on the rotatable input mechanism, e.g. Figures 6A to 6I As shown.

[0245] At box 708, in response to the detected contact, the position is located at a first position relative to the rotatable input mechanism (e.g., on the housing above the rotatable input mechanism, such as...). Figure 6A As observed in the device orientation depicted in the diagram, or on a rotatable input mechanism, the device performs a process on one or more items associated with the focus selector. As previously discussed... Figures 6A to 6I The process discussed may include, for example, sending the project as an electronic message to an external device, placing the project in a text field, or launching an application associated with the project.

[0246] At box 710, in response to the detected contact being located at a second position relative to the rotatable input mechanism (e.g., on the housing below the rotatable input mechanism), the device abandons the process for one or more items associated with the focus selector.

[0247] Optionally, at box 712, in response to detecting that the contact location is in a second position, the device performs a second process on one or more items associated with the focus selector, such as referencing... Figures 6C to 6D and Figures 6E to 6F As described. The second process differs from the first process.

[0248] It should be noted that the process described above is relative to method 700 (e.g. Figure 7 The details of ) also apply in a similar manner to the methods described below. For example, method 800 optionally includes one or more characteristics of the various processes described above with reference to method 700. For example, device 500 can respond to contact on housing 502 (such as regarding Figure 7 (as described) and the ability to respond to determining that contact has been removed from the rotatable input mechanism (as per [the description]). Figure 8 The details described herein will not be repeated below for the sake of brevity.

[0249] According to some implementation plans Figure 12 An exemplary functional block diagram of an electronic device 1200 configured according to the principles of various described embodiments is shown. According to some embodiments, functional blocks of the electronic device 1200 are configured to perform the techniques described above. The functional blocks of the device 1200 are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of the various examples described. Those skilled in the art will understand that... Figure 12 The functional blocks described herein may optionally be combined or separated into sub-blocks to implement the principles of the various examples described. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0250] like Figure 12 As shown, the electronic device 1200 includes: a display unit 1202 configured to display a graphical user interface; optionally, a touch-sensitive surface unit 1204 configured to receive contact; a rotatable input mechanism unit 1206 configured to receive user input; a housing unit 1208 on which contact can be detected; and a processing unit 1210 coupled to the display unit 1202, the rotatable input mechanism unit 1206, the housing unit 1208, and optionally the touch-sensitive surface unit 1204. In some embodiments, the processing unit 1210 includes: a receiving unit 1212, a display enabling unit 1214, a detection unit 1216, an execution unit 1218, a sending unit 1220, a deactivation unit 1222, and a placement unit 1224.

[0251] The processing unit 1210 is configured to: (e.g., using the receiving unit 1212) receive a request to display an item; in response to receiving the request, enable (e.g., using the display enabling unit 1214) to display a user interface including multiple items on the display unit, wherein a focus selector is associated with one or more of the multiple items; (e.g., using the detection unit 1216) detect contact on the device at a position corresponding to the rotatable input mechanism; in response to the detected contact being at a first position relative to the rotatable input mechanism unit, execute a process on one or more items associated with the focus selector (e.g., using the execution unit 1218); and in response to the detected contact being at a second position relative to the rotatable input mechanism, abandon the execution of the process on one or more items associated with the focus selector, wherein the second position is different from the first position.

[0252] In some implementations, the first position is located on the housing unit above the rotatable input mechanism unit.

[0253] In some implementations, the first position is located on the rotatable input mechanism unit.

[0254] In some implementations, the input mechanism is not activated by contact.

[0255] In some implementations, the contact does not include any contact with the display unit.

[0256] In some implementations, the one or more items are one or more text strings.

[0257] In some implementations, the processing unit is further configured to, in response to a detected contact position at a first position relative to the rotatable input mechanism, (e.g., using the sending unit 1220) send one or more items associated with the focus selector as electronic messages to an external device, wherein performing the process includes sending one or more items.

[0258] In some implementations, the processing unit is further configured to release other items not associated with the focus selector in response to a detected contact position at a first position relative to the rotatable input mechanism unit, wherein performing the process includes releasing other items.

[0259] In some implementations, the processing unit is further configured to place one or more items in a text field in response to a detected contact position at a first position relative to the rotatable input mechanism unit, wherein performing the process includes placing one or more items in the text field.

[0260] In some implementations, contact is part of a gesture that includes a tapping motion.

[0261] In some implementations, the display unit is a touch-sensitive display unit.

[0262] In some implementations, the device includes one or more sensors configured to detect contact on the housing.

[0263] In some implementations, the device includes one or more sensors configured to detect contact on the rotatable input mechanism.

[0264] In some implementations, the one or more sensors include one or more capacitive sensors.

[0265] In some implementations, the processing unit is further configured to perform a second process in response to the detected contact being located at a second position (e.g., using execution unit 1218).

[0266] In some implementations, the second process includes displaying a second user interface.

[0267] In some implementations, a second process is performed on one or more items associated with the focus selector.

[0268] In some implementations, the contact is located outside the display on the device.

[0269] Earlier reference Figure 7 The described operation is optionally provided by Figures 1A to 1B , Figures 5A to 5B or Figure 12The components depicted are used to implement this. For example, the receive operation 702, the detect operation 706, and the execute operation 708 are optionally implemented by an event classifier 170, an event recognizer 180, and an event handler 190. An event monitor 171 in the event classifier 170 detects a contact corresponding to the input mechanism 506, and an event dispatcher module 174 delivers the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with a corresponding event definition 186 and determines whether the contact corresponds to a predefined event or sub-event, such as a request for process execution. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes a data updater 176 or an object updater 177 to update the application's internal state 192. In some embodiments, the event handler 190 accesses a corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand how it can be based on Figure 1A-Figure 1B The components depicted in the diagram are used to perform other processes.

[0270] Figures 6L to 6N The diagram illustrates a process that device 500 can perform in response to detecting user input on rotatable input mechanism 506. As will be discussed in more detail below, after detecting rotation of the rotatable input mechanism, device 500 can determine whether the user has removed contact from the rotatable input mechanism 506 (e.g., whether the user has removed one or more of their fingers from the rotatable input mechanism after rotation), and thus respond differently. The user interface depicted in these figures is used to illustrate including... Figure 8 The process described below is the process in the middle.

[0271] Figure 6L An exemplary user interface 636 is depicted, which includes a vertical slideshow of images 630, 632, and 634 and two focus selectors 638 associated with images 632 and 634. In this example, the focus selector is a coarse outline around the image.

[0272] In response to detecting user rotation of the rotatable input mechanism 506, the device 500 vertically scrolls the image and changes the focus selector to be associated with the second set of images 634, 640 according to the rotation, such as Figure 6M As shown. In this example, the second set of images includes one of the images (634) included in the first set.

[0273] Users can "pinch" the rotatable input mechanism 506 between their fingers and thumb (e.g., ...). Figure 6L(As shown) and rotate it to rotate the rotatable input mechanism 506, or the user can rotate the rotatable input mechanism using, for example, a single finger. In both cases, the user contacts the rotatable input mechanism 506 to rotate it.

[0274] In some implementations, after detecting rotation of the rotatable input mechanism 506, device 500 determines whether contact has been removed from the rotatable input mechanism 506 when selecting the second set of images 634, 640; for example, device 500 determines whether the user has removed one or more of their fingers from the rotatable input mechanism 506. If device 500 determines that contact has been removed, device 500 performs a process on the item associated with focus selector 638. In this example, device 500 sends the second set of images 634, 640 as an electronic message to an external electronic device, such as... Figure 6N The device 600 shown is shown.

[0275] Conversely, if device 500 determines that contact has not been removed from rotatable input mechanism 506 (e.g., the user continues to touch or pinch rotatable input mechanism 506), device 500 does not send a second set of images. Therefore, in some embodiments, in response to determining that contact has not been removed, device 500 abandons the process for performing procedures on one or more items in a second set associated with focus selector 604.

[0276] In some implementations, in response to determining that the contact has not been removed, device 500 performs various processes, such as returning to the home screen, unlinking one or more items associated with the focus selector, or opening the associated application of one or more items associated with the focus selector.

[0277] As previously mentioned Figures 6A to 6K The process discussed above for navigating a list of items and selecting one or more items for processing can be applied to any type of project and to projects displayed in any format. Although Figures 6L to 6M The examples depicted focus on selecting and processing a set of two items (e.g., images), and in some implementations, the first and / or second set may include only one item.

[0278] Figure 8 This is a flowchart illustrating a method for navigating a list and selecting items using a scaled-down electronic device, according to some embodiments. Method 800 is performed at a device (e.g., 100, 300, 500) having a display and a rotatable input mechanism. Some operations in method 800 are optionally combined; the order of some operations is optionally changed, and some operations are optionally omitted.

[0279] As described below, method 800 provides users with an intuitive way to navigate a list (or multiple items) and select one or more items for processing. This method reduces the cognitive burden on users navigating lists and selecting items, thus creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to navigate lists and select items more efficiently, saving power and increasing the time interval between battery charges.

[0280] At box 802, the device displays a user interface including the plurality of items, where a focus selector is associated with one or more items in a first group of the plurality of items, such as Figure 6L As depicted in the text.

[0281] At box 804, the device detects user input corresponding to rotation of the rotatable input mechanism, which includes contact with the rotatable input mechanism. In some embodiments, for example, the user input corresponds to a user touching or pinching the rotatable input mechanism and rotating it, as per [reference to...]. Figure 6L As mentioned above.

[0282] At box 806, for example, in response to detecting user input corresponding to a rotation of the rotatable input mechanism, device 500 scrolls through multiple displayed items and changes the focus selector according to the rotation to associate it with one or more items in a second group, as per the description of... Figure 6G As described.

[0283] At box 808, the device determines whether contact has been removed from the rotatable input mechanism when one or more items in the second group are associated with the focus selector.

[0284] For example, in response to determining that user contact has been removed, at box 810, the device performs a process on one or more items in the second group, such as regarding Figure 6M As described.

[0285] In response to determining that user contact has not been removed, at box 812, the device abandons the process for one or more items in the second group. In this case, the device 500 may not perform any process, or it may perform different processes (such as removing items, e.g., regarding...) for one or more items in the second group. Figure 6F (as described), or can perform different processes independent of the second group of one or more items (such as returning to the home screen, as described). Figure 6D (As described).

[0286] It should be noted that the above is relative to method 800 (for example, Figure 8The details of the process described above also apply in a similar manner to the methods described above and below. For example, method 700 optionally includes one or more features of the various methods described above with reference to method 800. As previously mentioned, for example, device 500 can respond to contact on housing 502 (as per [reference to...]). Figure 7 (as described) and the ability to respond to determining that contact has been removed from the rotatable input mechanism (as per [the description]). Figure 8 The details described herein will not be repeated below for the sake of brevity.

[0287] According to some implementation plans Figure 13 An exemplary functional block diagram of an electronic device 1300 configured according to the principles of various described embodiments is shown. According to some embodiments, functional blocks of the electronic device 1300 are configured to perform the techniques described above. The functional blocks of the device 1300 are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of the various described examples. Those skilled in the art will understand that... Figure 13 The functional blocks described herein may optionally be combined or separated into sub-blocks to implement the principles of the various examples described. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0288] like Figure 13 As shown, the electronic device 1300 includes a display unit 1302 configured to display a user interface; optionally, a touch-sensitive surface unit 1304 configured to receive contact; a rotatable input mechanism unit 1306 configured to receive user input; and a processing unit 1308 coupled to the display unit 1302, the rotatable input mechanism unit 1306, and optionally the touch-sensitive surface unit 1304. In some embodiments, the processing unit 1308 includes a display enable unit 1310, a detection unit 1312, a scrolling unit 1314, a change unit 1316, a determination unit 1318, an execution unit 1320, a release unit 1322, a placement unit 1324, and a sending unit 1326.

[0289] Processing unit 1308 is configured to: enable (e.g., using display enable unit 1310) display a user interface comprising multiple items on display unit 1302, wherein a focus selector is associated with a first group of one or more items among the multiple items; (e.g., using detection unit 1312) detect user input corresponding to rotation of rotatable input mechanism unit 1306, wherein the rotation includes contact with rotatable input mechanism unit; in response to detecting user input corresponding to rotation, (e.g., using scrolling unit 1314) scroll multiple items and associate the focus selector with a second group of one or more items according to rotation (e.g., using association unit 1316); (e.g., using determination unit 1318) determine whether user contact has been removed from rotatable input mechanism when the second group of one or more items is associated with focus selector; based on the determination that contact has been removed when the second group of one or more items is associated with focus selector, (e.g., using execution unit 1320) perform a process on the second group of one or more items; and based on the determination that contact has not been removed when the second item is in focus, abandon the process for the second group of one or more items.

[0290] In some implementations, the processing unit is further configured to: release items not in the second group of one or more items based on the determination that contact has been removed when the second group of one or more items is associated with the focus selector (e.g., using release unit 1322), wherein the execution process includes releasing the items.

[0291] In some implementations, the processing unit is further configured to place one or more items from a second group into a text field, based on the determination that the contact has been removed (e.g., using placement unit 1324), wherein performing the process includes placing the items into the text field.

[0292] In some implementations, the processing unit is further configured to send a second set of one or more items in an electronic message to an external device based on a determination that the contact has been removed (e.g., using sending unit 1326), wherein performing the process includes sending the items.

[0293] In some implementations, the processing unit is further configured to, based on the determination that the contact has not been removed when a second group of one or more items is associated with the focus selector, and while continuing to detect user input corresponding to the rotation of the rotatable input mechanism, (e.g., using scrolling unit 1314) scroll the items to display items that were not previously visible on the display unit, and associate the focus selector with a third group of one or more items based on the rotation (e.g., using association unit 1316).

[0294] The above is for reference only. Figure 8 The described operation is optionally provided by Figures 1A to 1B , Figures 5A to 5B or Figure 13 The components depicted are used to implement this. For example, the detection operation 804, the determination operation 808, and the execution operation 810 are optionally implemented by the event classifier 170, the event recognizer 180, and the event handler 190. The event monitor 171 in the event classifier 170 detects user input on the rotatable input mechanism 506, and the event dispatcher module 174 delivers the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with the corresponding event definition 186 and determines whether the user input corresponds to a predefined event or sub-event, such as rotation or removal of contact from the rotatable input mechanism. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes the data updater 176 or the object updater 177 to update the application's internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand how it can be based on Figure 1A-Figure 1B The components depicted in the diagram are used to perform other processes.

[0295] Figures 60 to 6R The process shown is that device 500 can perform in response to detecting a swipe gesture at a location other than the display on device 500; for example, a swipe gesture on housing 502 or on rotatable input mechanism 506. As previously discussed... Figures 6A to 6L As discussed, in some embodiments, housing 502 and / or rotatable input mechanism 506 may provide a touch-sensitive area on which a user can perform swipe gesture input to device 500 without having to touch touchscreen 504. The user interface depicted in these figures is used to illustrate including... Figure 9 The process described below is the process in the middle.

[0296] As will be discussed in more detail below, device 500 may respond differently to swipe gestures outside the display, depending on the position and / or orientation of the swipe gesture relative to the rotatable input mechanism 506. For example, device 500 may respond differently to a swipe gesture along a first direction on the upper housing of the rotatable input mechanism 506 than to a swipe gesture along a second (different) direction on the upper housing of the rotatable input mechanism 506.

[0297] Figure 6OAn exemplary user interface 644, optionally displayed by device 500, is depicted. User interface 644 is a map screen associated with a map drawing application. In response to detecting a swipe gesture corresponding to rotatable input mechanism 506, device 500 determines the direction of the swipe relative to rotatable input mechanism 506. The swipe gesture corresponding to rotatable input mechanism 506 does not need to be on rotatable input mechanism 506; it may be located, for example, on housing 502. In some embodiments, at least a portion of the swipe gesture is not on touchscreen 504; for example, it is outside the display.

[0298] exist Figure 6O In the example depicted, the swipe gesture corresponding to the rotatable input mechanism 506 is located on the housing 502 above the rotatable input mechanism 506 (as shown along...). Figure 6O (As shown in the device orientation). In this example, the direction of the swipe gesture is along an axis perpendicular to the display; for example, the “back-to-forehead” orientation of device 500 on the housing 502 of device 500, where the front of device 500 is considered to be the side with touchscreen 504.

[0299] In some implementations, the direction from back to front can be determined by contact sensor 542; for example, by comparing the signal strength of the contact as the contact moves vertically toward or away from the display.

[0300] In this example, if device 500 determines that the swipe gesture is in a "back-to-forward" direction relative to the rotatable input mechanism 506, such as Figure 6O As depicted, device 500 replaces the display (map screen) of user interface 644 with the display of user interface 646. User interface 646 is an audio control screen with transmission control options for controlling audio functions.

[0301] In some implementations, the device 500 can replace the display of the map user interface screen with the display of the audio control user interface screen by separately replacing one user interface with another, or by smoothly scrolling one user interface off the display while scrolling another user interface onto the display, so that portions of the two user interfaces are temporarily displayed simultaneously.

[0302] like Figure 6P As shown, in some embodiments, if device 500 determines that a swipe gesture corresponding to the rotatable input mechanism 506 is in a direction different from (e.g., opposite to) the directions described above, such as a "front-to-back" direction on the upper housing 504 of the rotatable input mechanism 506, then device 500 replaces the display of user interface 644 with a different user interface, namely user interface 648, instead of replacing user interface 644 with user interface 646. User interface 648 is a weather user interface associated with a weather application. Therefore, as shown in 60 to Figure 6P As shown, in some implementations, device 500 enables users to navigate the user interface using swipe gestures on housing 502.

[0303] In some implementations, a swipe gesture on housing 502 in a first direction causes device 500 to replace the first user interface with a second user interface, and a subsequent swipe gesture on housing 502 in the opposite direction causes device 500 to replace the second user interface with the first user interface. This type of navigation may be intuitive for users who previously navigated using swipe gestures on a touchscreen, but it offers the significant benefit of not obscuring the touchscreen during navigation. Furthermore, by responding to swipe gestures on surfaces other than the touchscreen, device 500 can provide a wider range of user inputs and a more robust and flexible user interface.

[0304] Despite about Figures 60 to 6P The described swipe gesture is shown on the housing 502 located above the rotatable input mechanism 506 (e.g., closer to the top of the device than the bottom, as along...). Figures 60 to 6P As shown in the orientation), in some embodiments, it can be on the rotatable input mechanism 506 (e.g. Figure 6Q (as shown) or on the housing 502 below the rotatable input mechanism 506 (as shown) Figure 6R (As shown) a swipe gesture is detected. In some embodiments, the orientation of device 500 (e.g., top and bottom, left and right, and / or front and rear) is defined by a rotatable input mechanism 506 and / or a touchscreen 504. In some embodiments, device 500 responds to a swipe gesture above the rotatable input mechanism 506 differently than a swipe gesture below or on the rotatable input mechanism 506.

[0305] Figure 6S Another example is depicted where device 500 can display a user interface in response to detecting a swipe gesture on housing 502. Device 500 can display user interface 650, which includes a notification list. Figure 6S In the illustrated example, device 500 detects a swipe gesture on housing 502 below rotatable input mechanism 506. The swipe gesture includes contact with a vertical upward movement toward housing 502 of rotatable input mechanism 506 in a direction perpendicular to the horizontal axis of rotatable input mechanism 506.

[0306] In response to detecting a swipe gesture and determining the direction as vertical towards the rotatable input mechanism 506, device 500 scrolls the user interface 650 to display additional notifications, such as... Figure 6T As shown.

[0307] In some implementations, device 500 may respond differently depending on the position of the swipe gesture and the orientation of the swipe gesture relative to the rotatable input mechanism 506. For example, if device 500 detects a vertically downward direction (e.g., along with) on the housing 502 above the rotatable input mechanism 506... Figure 6S A swipe gesture (in the opposite direction shown in the diagram) allows device 500 to scroll user interface 650 in the opposite direction.

[0308] Figure 9 This is a flowchart illustrating a method for navigating a user interface screen using a scaled-down electronic device, according to some embodiments. Method 900 is performed at a device (e.g., 100, 300, 500) having a display and a rotatable input mechanism. Some operations in method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0309] As described below, method 900 provides users with an intuitive way to navigate the user interface screen without obscuring the display by detecting swipe gestures outside the display, such as on the housing or a rotatable input mechanism. This method reduces the cognitive burden on users navigating the user interface screen, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling users to navigate user interface screens and documents more efficiently saves power and increases the time interval between battery charges.

[0310] At box 902, the device displays a first user interface. The first user interface can be an interface associated with a specific application, such as... Figures 60 to 6R As shown, for example, or as part of an electronic document.

[0311] When the first user interface is displayed, at box 904, the device detects a swipe gesture originating from the position corresponding to the rotatable input mechanism. In some implementations, the position corresponding to the rotatable input mechanism is a location outside the display, such as on the device's housing (e.g., Figure 6O , Figure 6P and Figure 6R (as shown) or on a rotatable input mechanism (such as Figure 6Q (As shown).

[0312] Optionally, a swipe gesture includes horizontal movement of a contact on a rotatable input mechanism, such as... Figure 6Q As shown in the image.

[0313] Optionally, a swiping gesture includes a vertical movement of the contact on the casing, such as... Figure 6S As shown in the image.

[0314] Optionally, a swipe gesture includes a movement of the contact on the housing along an axis perpendicular to the display, such as... Figure 6RAs shown in the image.

[0315] At box 906, the device determines the orientation of the swipe gesture relative to the rotatable input mechanism. In some implementations, for example, the device uses contact sensor 542 to determine the orientation of the swipe gesture.

[0316] At box 908, based on the direction of the swipe gesture determined at box 906, the device replaces the display of the first user interface with the display of the second user interface. In some embodiments, the device replaces the display of the first user interface screen with the display of the second user interface screen, as per [reference to...]. Figures 60 to 6R As described.

[0317] It should be noted that the above is relative to method 900 (for example, Figure 9 The details of the processes described above also apply in a similar manner to the methods described above and below. For example, method 800 optionally includes one or more characteristics of the various processes described above with reference to method 900. For example, device 500 may respond to a swipe gesture starting from a position corresponding to the rotatable input mechanism, as described above with respect to method 900, and to user input corresponding to rotation of the rotatable input mechanism 506, as described with respect to method 800. For the sake of brevity, these details will not be repeated.

[0318] According to some implementation plans Figure 14 An exemplary functional block diagram of an electronic device 1400 configured according to the principles of various described embodiments is shown. According to some embodiments, functional blocks of the electronic device 1400 are configured to perform the techniques described above. The functional blocks of the device 1400 are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of the various examples described. Those skilled in the art will understand that... Figure 14 The functional blocks described herein may optionally be combined or separated into sub-blocks to implement the principles of the various examples described. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0319] like Figure 14 As shown, the electronic device 1400 includes a display unit 1402 configured to display a user interface, optionally a touch-sensitive surface unit 1404 configured to receive contact, a rotatable input mechanism unit 1406 configured to receive user input, a housing unit 1408 on which contact can be detected, and a processing unit 1410 coupled to the display unit 1402, the rotatable input mechanism unit 1406, and optionally the touch-sensitive surface unit 1404 and the housing unit 1408. In some embodiments, the processing unit 1410 includes a display enabling unit 1412, a detection unit 1414, a determination unit 1416, and an execution unit 1418.

[0320] The processing unit 1410 is configured to: enable (e.g., using the display enabling unit 1412) to display a first user interface screen on the display unit; when the first user interface screen is displayed on the display unit, (e.g., using the detection unit 1414) detect a swipe gesture starting from a position corresponding to the rotatable input mechanism unit; (e.g., using the determining unit 1416) determine the direction of the swipe gesture relative to the rotatable input mechanism; and based on the determined direction of the swipe gesture relative to the rotatable input mechanism, enable the display of a second user interface screen to replace the display of the first user interface screen on the display unit.

[0321] In some implementations, a swipe gesture includes contact with a horizontal movement on a rotatable input mechanism unit.

[0322] In some implementations, the swipe gesture includes a vertical movement of contact on the housing unit.

[0323] In some implementations, a swipe gesture includes contact movement on the housing unit along an axis perpendicular to the display unit.

[0324] In some implementations, the swipe is positioned above the rotatable input mechanism unit.

[0325] In some implementations, the swipe gesture is a first swipe gesture, and the processing unit is further configured to: (e.g., using detection unit 1414) detect a second swipe gesture starting from a position corresponding to the rotatable input mechanism and in the opposite direction to the first swipe gesture; and (e.g., using display enable unit 1412) replace the display of the second user interface screen on the display unit with the display of the first user interface screen on the display unit.

[0326] In some implementations, at least a portion of the swipe gesture is detected at a location outside the display unit on the device.

[0327] The above is for reference only. Figure 9 The described operation is optionally provided by Figures 1A to 1B , Figures 5A to 5B or Figure 14The components depicted are used to implement this. For example, the detection operation 904, the determination operation 906, and the replacement operation 908 are optionally implemented by the event classifier 170, the event recognizer 180, and the event handler 190. The event monitor 171 in the event classifier 170 detects a swipe gesture corresponding to the rotatable input mechanism 506, and the event dispatcher module 174 delivers the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with the corresponding event definition 186 and determines whether the direction of the swipe gesture corresponds to a predefined event or sub-event, such as a request to replace the display of the first user interface screen with the display of the second user interface screen. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes the data updater 176 or the object updater 177 to update the internal state 192 of the application. In some implementations, event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will readily understand how this can be done based on... Figure 1A-Figure 1B The components depicted in the diagram are used to perform other processes.

[0328] Figures 6U to 6V The diagram illustrates a process that device 500 can perform in response to detecting a swipe gesture outside the display while an electronic document is being displayed. As will be discussed in more detail below, device 500 can respond to a swipe gesture outside the display by translating the display of the electronic document according to the direction and / or position of the swipe gesture relative to the rotatable input mechanism 506. The user interface depicted in these figures is used to illustrate including Figure 10 The process described below is the process in the middle.

[0329] As previously mentioned Figures 6A to 6T As discussed, in some embodiments, housing 502 and / or rotatable input mechanism 506 may provide a touch-sensitive area on which a user may provide swipe gesture input to device 500 without having to touch touchscreen 504.

[0330] Figure 6U An exemplary user interface 654, optionally displayed by device 500, is depicted. User interface 654 is a map associated with a map drawing application. In response to detecting a swipe gesture corresponding to rotatable input mechanism 506, device 500 determines the direction of the swipe relative to rotatable input mechanism 506.

[0331] As previously stated, the swipe gesture corresponding to the rotatable input mechanism does not need to be on the rotatable input mechanism 506; it may be located, for example, on the housing 502. In some embodiments, at least a portion of the swipe gesture is not on the touchscreen 504; for example, it is outside the display.

[0332] like Figure 6U As shown, in response to determining the direction of a swipe gesture relative to the rotatable input mechanism 506, device 500 translates the map display according to the determined direction of the swipe gesture. For example, device 500 can translate (e.g., shift or tile) the map up or down, left or right, according to the direction of the swipe gesture to display portions of the map that were not previously displayed. In this example, the swipe gesture is from left to right on housing 502 (note that housing 502 includes the border of device 500), and the map is translated from left to right on the display. In this way, device 500 allows the user to translate (e.g., shift or tile) the display of an electronic document without requiring the user to touch the display and obscure a portion of the content.

[0333] Figure 6V An exemplary user interface optionally displayed by device 500 is depicted. In this example, device 500 displays user interface 658, which includes a document containing text. In response to detecting a swipe gesture on housing 506 and determining that the swipe gesture is in a vertical direction relative to rotatable input mechanism 506, device 500 vertically translates the display of the text document upwards.

[0334] While the above examples describe panning the display of a map or text document, those skilled in the art will recognize that the above techniques can be used to pan the display of any type of electronic document or screen, including images, web pages, lists, menus, etc.

[0335] Figure 10 This is a flowchart illustrating a method for translating the display of an electronic document using a scaled-down electronic device, according to some embodiments. Method 1000 is performed at a device (e.g., 100, 300, 500) having a display and a rotatable input mechanism. Some operations in method 1000 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0336] As described below, method 1000 provides users with an intuitive way to pan (e.g., shift or tile) the display of electronic documents. This method reduces the cognitive burden on users navigating documents, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to pan the display of electronic documents more efficiently saves power and increases the time interval between battery charges.

[0337] At box 1002, the device displays a portion of an electronic document. This electronic document can be a map, image, item list, text document, webpage, etc., such as an information about... Figures 6U to 6V As described.

[0338] At box 1003, when displaying a portion of an electronic document, the device detects a swipe gesture originating from a position corresponding to the rotatable input mechanism. This position may be on the housing or on the rotatable input mechanism, for example, as per [reference to...]. Figures 6U to 6V As described.

[0339] Optionally, a swipe gesture includes horizontal movement of a contact on a rotatable input mechanism, such as regarding Figure 6Q shown.

[0340] Optionally, a swipe gesture includes a vertical movement of the contact on the casing, as in the case of... Figure 6V shown.

[0341] Optionally, a swipe gesture includes a movement of the contact on the housing along an axis substantially perpendicular to the display, as described above. Figure 6R shown.

[0342] At frame 1006, the device determines the orientation of the swipe gesture relative to the rotatable input mechanism.

[0343] At box 1008, the device pans the display of the electronic document according to the direction determined by the swipe gesture. Figure 6U For example, the device determines that a swipe gesture is from left to right on the casing and accordingly pans the display of an electronic document from left to right on the display.

[0344] It should be noted that the above is relative to method 1000 (for example, Figure 10 The details of the process described above also apply in a similar manner to the methods described above and below. For example, method 900 optionally includes one or more characteristics of the various processes described above with reference to method 1000. For example, in some embodiments, device 500 can provide, as about Figure 7 and Figure 8 The aforementioned list navigation and item selection functions allow users to select and open electronic documents, and then, in response to detecting information about... Figure 9 The swipe gesture described above is used to pan the display of electronic documents. For the sake of brevity, these details will not be repeated.

[0345] According to some implementation plans Figure 15An exemplary functional block diagram of an electronic device 1500 configured according to the principles of various described embodiments is shown. According to some embodiments, functional blocks of the electronic device 1500 are configured to perform the techniques described above. The functional blocks of the device 1500 are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of the various described examples. Those skilled in the art will understand that... Figure 15 The functional blocks described herein may optionally be combined or separated into sub-blocks to implement the principles of the various examples described. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0346] like Figure 15 As shown, the electronic device 1500 includes a display unit 1502 configured to display a user interface, optionally a touch-sensitive surface unit 1504 configured to receive contact, a rotatable input mechanism unit 1506 configured to receive user input, a housing unit 1508 on which contact can be detected, and a processing unit 1510 coupled to the display unit 1502, the rotatable input mechanism unit 1506, and optionally the touch-sensitive surface unit 1504 and the housing unit 1508. In some embodiments, the processing unit 1510 includes a display enabling unit 1512, a detection unit 1514, a determination unit 1516, and a translation unit 1518.

[0347] The processing unit 1510 is configured to: enable (e.g., using the display enable unit 1512) display a portion of an electronic document on the display unit; (e.g., using the detection unit 1514) detect a swipe gesture on the device at a position corresponding to the rotatable input mechanism; (e.g., using the determination unit 1516) determine the direction of the swipe gesture relative to the rotatable input mechanism unit; and translate the display of the document on the display unit according to the determined direction of the swipe gesture (e.g., using the translation unit 1518).

[0348] In some implementations, translating a document includes displaying a second portion of the document that is different from the first portion.

[0349] In some implementations, at least a portion of the swipe gesture is located on the housing.

[0350] In some implementations, at least a portion of the swipe gesture is detected at a location outside the display on the device.

[0351] The above is for reference only. Figure 10 The described operation is optionally provided by Figures 1A to 1B , Figures 5A to 5B or Figure 15The components depicted are used to implement this. For example, the detection operation 1004 and the determination operation 1006 are optionally implemented by an event classifier 170, an event recognizer 180, and an event handler 190. An event monitor 171 in the event classifier 170 detects a swipe gesture corresponding to the rotatable input mechanism 506, and an event dispatcher module 174 delivers the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with a corresponding event definition 186 and determines whether the swipe gesture corresponds to a predefined event or sub-event, such as a request to pan a document. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes a data updater 176 or an object updater 177 to update the application's internal state 192. In some embodiments, the event handler 190 accesses a corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly understand how it can be based on Figure 1A-Figure 1B The components depicted in the diagram are used to perform other processes.

[0352] Figure 6X The process shown is that the device 500 can perform in response to the detection of a contact at a location other than the display on the device 500; for example, a tap or swipe gesture on the housing 502 or on the rotatable input mechanism 506. Figure 6X The user interface depicted in the image is used to illustrate the following: Figure 11 The process described below is the process in the middle.

[0353] In some embodiments, device 500 may operate in an audio control mode. In some embodiments, the audio control mode is associated with a user interface for receiving user input associated with controlling audio functions. For example, such audio functions may include changing the volume of played audio content. For example, such audio functions may include changing the track or playlist of played audio content. For example, such audio functions may include starting, stopping, pausing, fast-forwarding, or rewinding audio recording. Those skilled in the art will understand that the disclosed techniques can be used to control other audio functions, and that the disclosed techniques can also be applied to control other multimedia functions such as video functions.

[0354] In some implementations, when device 500 is operating in audio control mode and the display is inactive, device 500 can respond to touch outside the display by sending audio control data associated with the audio function to an external audio device without activating the display. For example, device 500 can enable users to control the audio functions of external audio devices (such as wireless headphones or earbuds) from device 500 without activating the display on device 500, thereby reducing battery usage.

[0355] In some implementations, the user interface associated with the audio control mode can be activated when the display is off or dimmed.

[0356] In some implementations, the user interface associated with the audio control mode can be active when the display shows a user interface for different applications, such as a mapping application. Therefore, in some implementations, the user interface associated with the audio control mode does not require a display.

[0357] In some embodiments, device 500 may enter an audio control mode, for example, in response to detecting user input such as a touch on touchscreen 502, a touch on housing 506, or a touch on rotatable input mechanism 506. In some embodiments, device 500 may enter an audio control mode, for example, in response to detecting mechanical actuation (e.g., rotation or pressing) of user input such as rotatable input mechanism 506 or button 508. In some embodiments, device 500 may automatically enter an audio control mode after launching an audio application.

[0358] In order for device 500 to communicate with external audio devices, in some implementations, such as Figure 6W As shown, device 500 includes an audio output communicator 662, such as a bus 664 or a transmitter 666, for communicating with external audio devices. For example, the audio output communicator may include a bus for communicating with wired earphones. For example, the audio output communicator may include a transmitter for communicating wirelessly with wireless headphones.

[0359] In some embodiments, when device 500 is operating in audio control mode and touchscreen 502 is inactive or dimmed, device 500 can detect and respond to touch outside touchscreen 502 without activating or turning on touchscreen 502. In some embodiments, when device 500 is operating in audio control mode, if device 500 detects a contact at a location corresponding to rotatable input mechanism 506 (e.g., on housing 504 or on rotatable input mechanism 506), device 500 provides audio control data associated with the audio function to audio output communicator 662. For example, device 500 can provide audio control data to audio output communicator by providing audio control data to bus 664 or transmitter 666.

[0360] Figure 6X An example of a device 500 operating in audio control mode when the touchscreen 504 is inactive or dimmed is depicted. In response to detecting a swipe gesture on the housing 502 below the rotatable input mechanism 506, wherein the swipe gesture includes movement toward the rotatable input mechanism 506, the device 500 provides audio control data to an audio output communicator 662. The audio output communicator 662 then sends the audio control data to external audio devices such as wired earphones 668, wireless headphones 670, speakers 672, or portable multifunction devices 600 to control the audio functions of these external audio devices. For example, if the device 500 detects an upward swipe on the housing 506, the device 500 can respond by providing audio control data to the audio output communicator, causing the wireless headphones 608 to increase the volume of the audio content being played.

[0361] In some implementations, contact corresponding to the rotatable input mechanism 506 does not activate the touchscreen 502; for example, contact on the housing 502 or the rotatable input mechanism 506 does not cause the touchscreen 502 to be turned on or to actively display content.

[0362] In some implementations, if the touchscreen 502 is displaying a user interface associated with another application, contact with the rotatable input mechanism 506 will not cause the touchscreen 502 to display different content.

[0363] In some embodiments, device 500 can provide different audio control data to the audio output communicator based on the position of the contact relative to the rotatable input mechanism. For example, in response to detecting that the contact is in a first position relative to the rotatable input mechanism (e.g., on the housing below the rotatable input mechanism), device 500 can send audio control data associated with an audio function that reduces audio volume. In response to detecting that the contact is in a second (different) position relative to the rotatable input mechanism (e.g., on the housing above the rotatable input mechanism), device 500 can send audio control data associated with an audio function that increases the volume of audio content being played. Therefore, in some embodiments, the audio control data is based on the position of the contact relative to the rotatable input mechanism 506.

[0364] In some implementations, device 500 can provide different audio control data to the audio output communicator based on the direction of movement of the contact. For example, in response to detecting an upward swipe (e.g., on the housing 502 toward the rotatable input mechanism 506)... Figure 6W As shown, device 500 can provide audio control data associated with an audio function that switches to the next track in the playlist. In response to detecting a downward swipe on housing 502 below the rotatable input mechanism 506, device 500 can provide audio control data associated with an audio function that switches to the previous track in the playlist. Therefore, in some embodiments, the audio control data is based on the direction of movement of the contact relative to the rotatable input mechanism 506 (e.g., the direction of the swipe gesture).

[0365] In some implementations, the contact does not activate mechanically actuated inputs such as button 508 or rotatable input mechanism 506. For example, the contact does not cause button 508 to be pressed or rotatable input mechanism 506 to be rotated.

[0366] In some implementations, if device 500 detects contact with touchscreen 504 while in audio control mode, device 500 activates touchscreen 504 by, for example, turning on the screen or displaying a user interface.

[0367] In some implementations, device 500 may provide audio control data to an audio output communicator in response to the detection of rotation of the rotatable input mechanism 506. For example, in response to the detection of rotation of the rotatable input mechanism 506 in a first direction, device 500 may provide audio control data associated with increasing the volume of audio content. In response to the detection of rotation of the rotatable input mechanism 506 in the opposite direction, device 500 may provide audio control data associated with an audio function that decreases the volume of audio content. Similarly, in some implementations, device 500 may provide audio control data associated with an audio function that switches tracks or playlists in response to the detection of rotation of the rotatable input mechanism 506. Thus, in some implementations, a user may control an external audio device by tapping or swiping the housing of device 500, by tapping or swiping the rotatable input mechanism, and / or by rotating the rotatable input mechanism—none of which requires activation of the display.

[0368] Figure 11 This is a flowchart illustrating a method for controlling the audio function of an external device using a scaled-down electronic device according to some embodiments. Method 1100 is performed at a device (e.g., 100, 300, 500) having a display and a rotatable input mechanism. Some operations in method 1100 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0369] As described below, method 1100 provides users with an intuitive way to control the audio functions of external audio devices without activating the display. This method reduces the cognitive burden on users controlling audio, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to control audio functions without activating the display saves power and increases the time interval between battery charges.

[0370] At frame 1102, when the display is inactive and the device is operating in audio control mode, the device detects a contact on the device at a location corresponding to the rotatable input mechanism, wherein this contact does not activate the display. In some embodiments, the contact is located on the housing, such as... Figure 6X As shown. In some embodiments, the contact is located on a rotatable input mechanism.

[0371] At block 1104, in response to a detected contact position at a first position relative to the rotatable input mechanism, the device provides first audio control data to the audio output communicator, wherein the first audio control data is associated with an audio function. In some embodiments, the device provides the first audio control data to the audio output communicator by providing the audio control data to a bus or transmitter, as referenced. Figure 6W As stated above.

[0372] Optionally, the first audio control data is associated with changing the volume of the audio content being played.

[0373] Optionally, the first audio control data is associated with changing the audio content being played.

[0374] Optionally, at frame 1106, in response to the detected contact being in a second position relative to the rotatable input mechanism, the device waives the provision of first audio data to the audio output communicator.

[0375] Optionally, at box 1106, in response to the detected contact position at the second position, audio control data is associated with a second audio function.

[0376] Note that the above reference method 1100 (for example, Figure 11 The details of the process described above also apply in a similar manner to the methods described above. For example, method 1000 optionally includes one or more features of the various methods described above with reference to method 1100. For example, when the display is active and device 500 is not in audio control mode, device 500 can pan the display of the document as described with respect to method 1000. When the display is inactive and device 500 has been placed in audio control mode, device 500 can provide the features described above with respect to method 1000. Figure 11 The audio control functions described above. For the sake of brevity, these details will not be repeated.

[0377] According to some implementation plans Figure 16 An exemplary functional block diagram of an electronic device 1600 configured according to the principles of various described embodiments is shown. According to some embodiments, functional blocks of the electronic device 1600 are configured to perform the techniques described above. The functional blocks of the device 1600 are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of the various examples described. Those skilled in the art will understand that... Figure 16 The functional blocks described herein may optionally be combined or separated into sub-blocks to implement the principles of the various examples described. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0378] like Figure 16As shown, the electronic device 1600 includes: a display unit 1602 configured to display a graphical user interface; optionally, a touch-sensitive surface unit 1604 configured to receive contact; a rotatable input mechanism unit 1604 configured to receive user input; a housing unit 1604 on which contact is detected; an audio output unit 1604 configured to provide audio control data to an external audio device; and a processing unit 1612 coupled to the display unit 1602, the rotatable input mechanism unit 1606, the housing unit 1608, the audio output unit 1610, and optionally the touch-sensitive surface unit 1604. In some embodiments, the processing unit 1612 includes a detection unit 1614, a providing unit 1616, a mode entry unit 1618, a determining unit 1620, and an activation unit 1622.

[0379] The processing unit 1612 is configured to: when the display unit is inactive and the device is operating in audio control mode, (e.g., using the detection unit 1614) detect a contact on the device at a position corresponding to the rotatable input mechanism unit, wherein the contact does not activate the display unit; and in response to the detected contact being at a first position relative to the rotatable input mechanism, (e.g., using the providing unit 1616) provide first audio control data to the audio output unit, wherein the first audio control data is associated with an audio function.

[0380] In some implementations, the processing unit is further configured to relinquish the provision of first audio control data to the audio output communicator in response to the detected position of the second contact being in a second position relative to the rotatable input mechanism.

[0381] In some implementations, the processing unit is further configured to: in response to the detected location at the second location, (e.g., using the providing unit 1616) provide the audio output unit with second audio control data, wherein the second audio control data is different from the first audio control data, and wherein the second audio control data is associated with a second audio function.

[0382] In some implementations, the input does not activate the mechanical actuation.

[0383] In some implementations, the processing unit is further configured to detect user input (e.g., using detection unit 1614) before contact is detected; and to enter an audio control mode (e.g., using mode entry unit 1618) in response to the detection of user input.

[0384] In some implementations, the first audio control data is associated with changing the volume of the audio content being played.

[0385] In some implementations, the first audio control data is associated with changing the audio content being played.

[0386] In some implementations, contact includes movement of the contact, and the processing unit is further configured to (e.g., using determining unit 1620) determine the direction of movement of the contact relative to the rotatable input mechanism, wherein the first audio control data is based on the determined direction of movement.

[0387] In some embodiments, the processing unit is further configured to: provide the audio output unit with first audio control data associated with reducing the volume of the playing audio content based on determining that the movement is in a first direction relative to the rotatable input mechanism (e.g., using the providing unit 1616); and provide the audio output unit with first audio control data associated with reducing the volume of the playing audio content based on determining that the movement is in a second direction different from the first direction (e.g., using the providing unit 1616).

[0388] In some embodiments, the processing unit is further configured to: provide the audio output unit with first audio control data associated with switching the audio content to be played from a first track to a second track, based on determining that the movement is in a third direction relative to the rotatable input mechanism (e.g., using the providing unit 1616); and provide the audio output unit with first audio control data associated with switching the audio content to be played from a first track to a third track, based on determining that the movement is in a fourth direction other than the third direction (e.g., using the providing unit 1616).

[0389] In some implementations, the contact is a tap, and the first audio control data is based on the position of the tap relative to the rotatable input mechanism.

[0390] In some implementations, in response to a detected tap being located in a first position relative to the rotatable input mechanism, audio control data is associated with decreasing the volume of the played audio content; and in response to a detected tap being located in a second position relative to the rotatable input mechanism, the second position being different from the first position, audio control data is associated with increasing the volume of the played audio content.

[0391] In some implementations, in response to a detected tap being located in a third position relative to the rotatable input mechanism, audio control data is associated with switching the audio content being played from a first track to a second track; and in response to a detected tap being located in a fourth position relative to the rotatable input mechanism, the fourth position being different from the third position, audio control data is associated with switching the audio content being played from a first track to a third track.

[0392] In some implementations, the processing unit is further configured to: (e.g., using the detection unit 1614) detect a second contact on the touch-sensitive display unit; and activate the display unit in response to detecting the second contact.

[0393] In some implementations, the processing unit is further configured to: in audio control mode, (e.g., using detection unit 1614) detect user input corresponding to the rotation of the rotatable input mechanism unit; and in response to detecting user input, (e.g., using providing unit 1616) provide audio control data associated with the audio function to the audio output communicator.

[0394] In some implementations, the detected contact is located outside the display unit on the device.

[0395] The above reference Figure 11 The described operation is optionally provided by Figure 1A-Figure 1B or Figure 16 The components depicted are used to implement this. For example, the detection operation 1102 and the provision operation 1104 are optionally implemented by an event classifier 170, an event recognizer 180, and an event handler 190. An event monitor 171 in the event classifier 170 detects contact at a position corresponding to the rotatable input mechanism 506, and an event distributor module 174 delivers the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with a corresponding event definition 186 and determines whether the contact corresponds to a predefined event or sub-event, such as a request to provide audio control data. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes a data updater 176 or an object updater 177 to update the application's internal state 192. In some embodiments, the event handler 190 accesses RF circuitry or a bus to provide audio control data. Similarly, those skilled in the art will readily understand how this can be achieved based on... Figure 1A-Figure 1B The components depicted in the diagram are used to perform other processes.

[0396] Exemplary methods, non-transitory computer-readable storage media, systems, and electronic devices are listed in items 1 through 130 below:

[0397] 1. A method comprising:

[0398] In devices that have a display, housing, and rotatable input mechanism:

[0399] Receive requests to display items;

[0400] In response to receiving the request, a user interface comprising multiple items is displayed on the display, wherein a focus selector is associated with one or more of the multiple items;

[0401] Detect the contact at the position on the device corresponding to the rotatable input mechanism;

[0402] In response to the detected position of the contact at a first position relative to the rotatable input mechanism, for the one or more of the focus selectors associated with the focus selector

[0403] The project execution process; and

[0404] In response to the detected position of the contact being in a second position relative to the rotatable input mechanism, the process for the one or more items associated with the focus selector is abandoned, wherein the second position is different from the first position.

[0405] 2. The method according to item 1, wherein the first position is located on the housing above the rotatable input mechanism.

[0406] 3. The method according to item 1, wherein the first position is located on the rotatable input mechanism.

[0407] 4. The method according to any one of items 1 to 3, wherein the contact does not activate the mechanically actuated input mechanism.

[0408] 5. The method according to any one of items 1 to 4, wherein the contact does not include any contact on the display.

[0409] 6. The method according to any one of items 1 to 5, wherein the one or more items are one or more text strings.

[0410] 7. The method according to any one of items 1 to 6, further comprising:

[0411] In response to the detected position of the contact at the first position relative to the rotatable input mechanism, the one or more items are sent as electronic messages to an external device, wherein performing the process includes sending the one or more items to the external device.

[0412] 8. The method according to any one of items 1 to 7, further comprising:

[0413] In response to the detected position of the contact being at the first position relative to the rotatable input mechanism, the other items not associated with the focus selector are released, wherein performing the process includes releasing the other items.

[0414] 9. The method according to any one of items 1 to 8, further comprising:

[0415] In response to the detected position of the contact at the first position relative to the rotatable input mechanism, the one or more items are placed in a text field, wherein performing the process includes placing the one or more items in the text field.

[0416] 10. The method according to any one of items 1 to 9, wherein the contact is part of a tapping gesture.

[0417] 11. The method according to any one of items 1 to 10, wherein the display is a touch-sensitive display.

[0418] 12. The method according to any one of items 1 to 11, wherein the device includes one or more sensors configured to detect contact on the housing.

[0419] 13. The method according to any one of items 1 to 12, wherein the device includes one or more sensors configured to detect contact on the rotatable input mechanism.

[0420] 14. The method according to any one of items 1 to 13, wherein the sensor is a capacitive sensor.

[0421] 15. The method according to any one of items 1 to 14, further comprising:

[0422] In response to the detected contact being at a second position, a second process is performed.

[0423] 16. The method of item 15, wherein the second process includes displaying a second user interface.

[0424] 17. The method according to any one of items 15 to 16, wherein the second process is performed on the one or more items associated with the focus selector.

[0425] 18. The method according to any one of items 1 to 17, wherein the contact is detected to be located outside the display on the device.

[0426] 19. A method comprising:

[0427] At devices with displays and rotatable input mechanisms:

[0428] A user interface comprising multiple items is displayed on the display, wherein a focus selector is associated with a first group of one or more items among the multiple items;

[0429] Detect user input corresponding to the rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism;

[0430] In response to detecting the rotation, the plurality of items are scrolled and adjusted according to the rotation.

[0431] Associate the focus selector with one or more items in the second group;

[0432] Determine whether the contact has been removed from the rotatable input mechanism when one or more items in the second group are associated with the focus selector;

[0433] Based on the determination that the contact has been removed when the second group of one or more items is associated with the focus selector, the process is performed on the second group of one or more items; and

[0434] If it is determined that the contact has not been removed while the second item is in focus, the process is abandoned for one or more items in the second group.

[0435] 20. The method according to item 19 further includes:

[0436] Based on the determination that the contact has been removed when one or more items in the second group are associated with the focus selector, the item not in the one or more items in the second group is deactivated, wherein performing the process includes deactivating the item.

[0437] 21. The method according to any one of items 19 to 20, further comprising:

[0438] Based on the determination that the contact has been removed when the second group of one or more items is associated with the focus selector, the second group of one or more items is placed in a text field, wherein performing the process includes placing the items in the text field.

[0439] 22. The method according to any one of items 19 to 21, further comprising:

[0440] Sending one or more items from the second group to an external device in an electronic message, wherein performing the process includes sending the items.

[0441] 23. The method according to any one of items 19 to 22, further comprising:

[0442] Based on the determination that the contact has not been removed when one or more items in the second group are associated with the focus selector, and while continuing to detect the user input corresponding to the rotation of the rotatable input mechanism:

[0443] Scroll through the items to reveal those that were not previously visible on the display, and

[0444] The focus selector is associated with one or more items in a third group based on the rotation.

[0445] 24. A method comprising:

[0446] At devices with displays and rotatable input mechanisms:

[0447] A first user interface screen is displayed on the monitor;

[0448] When the first user interface screen is displayed, a swipe gesture is detected starting from the position corresponding to the rotatable input mechanism;

[0449] Determine the direction of the swipe relative to the rotatable input mechanism; and based on the determined direction of the swipe relative to the rotatable input mechanism,

[0450] Replace the display of the first user interface screen with the display of the second user interface screen.

[0451] 25. The method according to item 24, wherein the swiping gesture includes contact with a horizontal movement on the rotatable input mechanism.

[0452] 26. The method of claim 24, wherein the device further includes a housing, and wherein the swiping gesture includes a vertical movement on the housing.

[0453] 27. The method of claim 24, wherein the device further includes a housing, and wherein the swipe gesture includes contacting movement on the housing along an axis perpendicular to the display.

[0454] 28. The method according to any one of items 24 to 27, wherein the swiping position is located above the rotatable input mechanism.

[0455] 29. The method according to any one of items 24 to 28, wherein the swipe gesture is a first swipe gesture, the method further comprising:

[0456] The detection starts from the position corresponding to the rotatable input mechanism and proceeds along the path of the rotatable input mechanism.

[0457] A second swipe gesture in the opposite direction to the first swipe gesture; and

[0458] Replace the second user interface screen with the first user interface screen.

[0459] 30. The method according to any one of items 24 to 29, wherein at least a portion of the swipe gesture is detected at a location other than the display on the device.

[0460] 31. A method comprising:

[0461] At devices with displays and rotatable input mechanisms:

[0462] A portion of an electronic document is displayed on the monitor;

[0463] Detects a swipe gesture on the device at a position corresponding to the rotatable input mechanism;

[0464] Determine the direction of the swipe gesture relative to the rotatable input mechanism;

[0465] as well as

[0466] The document is translated in the direction determined by the swipe gesture.

[0467] 32. The method according to item 31, wherein translating the display of the document includes displaying a second portion of the document that is different from the first portion.

[0468] 33. The method according to any one of items 31 to 32, wherein the device includes a housing, and at least a portion of the swiping gesture is on the housing.

[0469] 34. The method according to any one of items 31 to 33, wherein at least a portion of the swipe gesture is detected at a location other than the display on the device.

[0470] 35. A method comprising:

[0471] In devices that include a display, housing, rotatable input mechanism, and audio output communicator:

[0472] When the display is inactive and the device is in audio control mode, a contact is detected on the device at a position corresponding to the rotatable input mechanism, wherein the contact does not activate the display; and

[0473] In response to the detected position of the contact being in a first position relative to the rotatable input mechanism, first audio control data is provided to the audio output communicator, wherein the first audio control data is associated with an audio function.

[0474] 36. The method described according to item 35 further includes:

[0475] The contact is detected at a second position relative to the rotatable input mechanism, and in response to the detected contact being at the second position, the supply of the first audio control data to the audio output communicator is abandoned.

[0476] 37. The method described according to item 36 further includes:

[0477] Further in response to the detected location being at the second location:

[0478] Provide second audio control data to the audio output communicator, wherein

[0479] The second audio control data is different from the first audio control data, and the second audio control data is associated with a second audio function.

[0480] 38. The method according to any one of items 35 to 37, wherein the contact does not activate the mechanically actuated input.

[0481] 39. The method according to any one of items 35 to 38, further comprising:

[0482] Detect user input before detecting the contact; and

[0483] In response to the detection of the user input, the user enters the audio control mode.

[0484] 40. The method according to any one of items 35 to 39, wherein the first audio control data is associated with changing the volume of the played audio content.

[0485] 41. The method according to any one of items 35 to 39, wherein the first audio control data is associated with changing the audio content being played.

[0486] 42. The method according to any one of items 35 to 41, wherein the contact includes movement of the contact, the method further comprising:

[0487] Determine the direction of movement of the contact relative to the rotatable input mechanism, wherein the first audio control data is based on the determined direction of movement.

[0488] 43. The method described in item 42 further includes:

[0489] Based on determining that the movement is in a first direction relative to the rotatable input mechanism, first audio control data is provided that is associated with reducing the volume of the audio content being played;

[0490] as well as

[0491] Based on determining that the movement is in a second direction different from the first direction, first audio control data associated with reducing the volume of the audio content being played is provided.

[0492] 44. The method according to any one of items 42 to 43, further comprising:

[0493] Based on determining that the movement is in a third direction relative to the rotatable input mechanism, first audio control data is provided associated with switching the audio content to be played from a first track to a second track; and

[0494] Based on determining that the movement is in a fourth direction different from the third direction, first audio control data is provided that is associated with switching the audio content to be played from the first track to the third track.

[0495] 45. The method according to any one of items 35 to 41, wherein the contact is a tap, and wherein the first audio control data is based on the position of the tap relative to the rotatable input mechanism.

[0496] 46. ​​The method according to item 45, wherein in response to the detected position of the tap being at a first position relative to the rotatable input mechanism, the audio control data is associated with decreasing the volume of the audio content being played; and

[0497] The position in response to the detected tap is in a second position relative to the rotatable input mechanism, which is different from the first position, and the audio control data is associated with increasing the volume of the audio content being played.

[0498] 47. The method according to items 45 to 46, wherein in response to the detected tap being at a third position relative to the rotatable input mechanism, the audio control data is associated with switching the played audio content from a first track to a second track; and

[0499] The position in response to the detected tap is in a fourth position relative to the rotatable input mechanism, which is different from the third position, and the audio control data is associated with switching the audio content to be played from the first track to the third track.

[0500] 48. The method according to any one of items 35 to 47, wherein the display is a touch-sensitive display, the method further comprising:

[0501] Detecting a second contact on the touch-sensitive display; and

[0502] In response to detecting the second contact, the touch-sensitive display is activated.

[0503] 49. The method described according to items 35 to 48 further includes:

[0504] When in the audio control mode, detect user input corresponding to the rotation of the rotatable input mechanism; and

[0505] In response to the detection of the user input, audio control data associated with the audio function is provided to the audio output communicator.

[0506] 50. The method according to any one of items 35 to 49, wherein the contact is detected to be located outside the display on the device.

[0507] 51. An apparatus comprising:

[0508] monitor;

[0509] shell;

[0510] Rotatable input mechanism;

[0511] One or more processors;

[0512] Memory; and

[0513] One or more programs stored in memory, comprising instructions that, when executed by the one or more processors, cause the device to perform the following operations:

[0514] Receive requests to display items;

[0515] In response to receiving the request, a user interface comprising multiple items is displayed on the display, wherein a focus selector is associated with one or more of the multiple items;

[0516] Detect the contact at the position on the device corresponding to the rotatable input mechanism;

[0517] In response to the detected position of the contact being in a first position relative to the rotatable input mechanism, a process is performed on the one or more items associated with the focus selector; and

[0518] In response to the detected position of the contact being in a second position relative to the rotatable input mechanism, the process for the one or more items associated with the focus selector is abandoned, wherein the second position is different from the first position.

[0519] 52. An apparatus comprising:

[0520] monitor;

[0521] Rotatable input mechanism;

[0522] One or more processors;

[0523] Memory; and

[0524] One or more programs stored in memory, which include programs that, when spoken of by one or more of the above, can be used to...

[0525] Instructions that, when executed by multiple processors, cause the device to perform the following operations:

[0526] A user interface comprising multiple items is displayed on the display, wherein a focus selector is associated with a first group of one or more items among the multiple items;

[0527] Detect user input corresponding to the rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism;

[0528] In response to detecting the rotation, the plurality of items are scrolled and the focus selector is associated with a second group of one or more items based on the rotation;

[0529] Determine whether the contact has been removed from the rotatable input mechanism when one or more items in the second group are associated with the focus selector;

[0530] Based on the determination that the contact has been removed when the second group of one or more items is associated with the focus selector, the process is performed on the second group of one or more items; and

[0531] If it is determined that the contact has not been removed while the second item is in focus, the process is abandoned for one or more items in the second group.

[0532] 53. An apparatus comprising:

[0533] monitor;

[0534] Rotatable input mechanism;

[0535] One or more processors;

[0536] Memory; and

[0537] One or more programs stored in memory, which include programs that, when spoken of by one or more of the above, can be used to...

[0538] Instructions that, when executed by multiple processors, cause the device to perform the following operations:

[0539] A first user interface screen is displayed on the monitor;

[0540] When the first user interface screen is displayed, a swipe gesture is detected starting from the position corresponding to the rotatable input mechanism;

[0541] Determine the direction of the sweep relative to the rotatable input mechanism;

[0542] as well as

[0543] Based on the determined direction of the swipe relative to the rotatable input mechanism, the display of the first user interface screen is replaced by the display of the second user interface screen.

[0544] 54. An apparatus comprising:

[0545] monitor;

[0546] Rotatable input mechanism;

[0547] One or more processors;

[0548] Memory; and

[0549] One or more programs stored in memory, which include programs that, when spoken of by one or more of the above, can be used to...

[0550] Instructions that, when executed by multiple processors, cause the device to perform the following operations:

[0551] A portion of an electronic document is displayed on the monitor;

[0552] Detects a swipe gesture on the device at a position corresponding to the rotatable input mechanism;

[0553] Determine the direction of the swipe gesture relative to the rotatable input mechanism; and

[0554] The document is translated in the direction determined by the swipe gesture.

[0555] 55. An apparatus comprising:

[0556] monitor;

[0557] shell;

[0558] Rotatable input mechanism;

[0559] Audio output communicator;

[0560] One or more processors;

[0561] Memory; and

[0562] One or more programs stored in memory, comprising instructions that, when executed by the one or more processors, cause the device to perform the following operations:

[0563] When the display is inactive and the device is in audio control mode, a contact is detected on the device at a position corresponding to the rotatable input mechanism, wherein the contact does not activate the display; and

[0564] In response to the detected position of the contact being in a first position relative to the rotatable input mechanism, first audio control data is provided to the audio output communicator, wherein the first audio control data is associated with an audio function.

[0565] 56. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display, a housing, and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform the following operations:

[0566] Receive requests to display items;

[0567] In response to receiving the request, a user interface comprising multiple items is displayed on the display, wherein a focus selector is associated with one or more of the multiple items;

[0568] Detect the contact at the position on the device corresponding to the rotatable input mechanism;

[0569] In response to the detected position of the contact being in a first position relative to the rotatable input mechanism, a process is performed on the one or more items associated with the focus selector; and

[0570] In response to the detected position of the contact being in a second position relative to the rotatable input mechanism, the process for the one or more items associated with the focus selector is abandoned, wherein the second position is different from the first position.

[0571] 57. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform the following operations:

[0572] A user interface comprising multiple items is displayed on the display, wherein a focus selector is associated with a first group of one or more items among the multiple items;

[0573] Detect user input corresponding to the rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism;

[0574] In response to detecting the rotation, the plurality of items are scrolled and the focus selector is associated with a second group of one or more items based on the rotation;

[0575] Determine whether the contact has been removed from the rotatable input mechanism when one or more items in the second group are associated with the focus selector;

[0576] Based on the determination that the contact has been removed when the second group of one or more items is associated with the focus selector, the process is performed on the second group of one or more items; and

[0577] If it is determined that the contact has not been removed while the second item is in focus, the process is abandoned for one or more items in the second group.

[0578] 58. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform the following operations:

[0579] A first user interface screen is displayed on the monitor;

[0580] When the first user interface screen is displayed, a swipe gesture is detected starting from the position corresponding to the rotatable input mechanism;

[0581] Determine the direction of the swipe relative to the rotatable input mechanism; and based on the determined direction of the swipe relative to the rotatable input mechanism,

[0582] The display of the first user interface screen is replaced by the display of a second user interface screen. 59. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform the following operations:

[0583] A portion of an electronic document is displayed on the monitor;

[0584] Detects a swipe gesture on the device at a position corresponding to the rotatable input mechanism;

[0585] Determine the direction of the swipe gesture relative to the rotatable input mechanism;

[0586] as well as

[0587] The display of the document is translated according to the direction determined by the swipe gesture. 60. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display, a housing, a rotatable input mechanism, and an audio output communicator, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform the following operations:

[0588] When the display is inactive and the device is in audio control mode, a contact is detected on the device at a position corresponding to the rotatable input mechanism, wherein the contact does not activate the display; and

[0589] In response to the detected position of the contact being in a first position relative to the rotatable input mechanism, first audio control data is provided to the audio output communicator, wherein the first audio control data is associated with an audio function.

[0590] 61. An apparatus comprising:

[0591] monitor;

[0592] shell;

[0593] Rotatable input mechanism;

[0594] A device for receiving requests to display items;

[0595] Means for displaying a user interface comprising multiple items on the display in response to receiving the request, wherein a focus selector is associated with one of the multiple items.

[0596] Or multiple related;

[0597] A device for detecting contact at a position on the device corresponding to the rotatable input mechanism;

[0598] In response to the detected contact, the position is at a first position relative to the rotatable input mechanism, for the one associated with the focus selector.

[0599] or a device for performing multiple project processes; and

[0600] A means for relinquishing the process of one or more items associated with the focus selector in response to the detected position of the contact being in a second position relative to the rotatable input mechanism, wherein the second position is different from the first position.

[0601] 62. An apparatus comprising:

[0602] monitor;

[0603] Rotatable input mechanism;

[0604] A means for displaying a user interface comprising multiple items on the display, wherein a focus selector is associated with a first group of one or more items among the multiple items;

[0605] A device for detecting user input corresponding to rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism;

[0606] Means for scrolling the plurality of items in response to detecting the rotation and associating the focus selector with a second group of one or more items according to the rotation;

[0607] A means for determining whether the contact has been removed from the rotatable input mechanism when one or more items in the second group are associated with the focus selector;

[0608] Used to determine, based on the fact that the contact has been removed when the second group of one or more items is associated with the focus selector, for the second group of one or more items

[0609] Devices for executing processes; and

[0610] A means for abandoning the process on one or more items in the second group based on the determination that the contact has not been removed while the second item is in focus.

[0611] 63. An apparatus comprising:

[0612] monitor;

[0613] Rotatable input mechanism;

[0614] A means for displaying a first user interface screen on the display;

[0615] A means for detecting a swipe gesture starting from a position corresponding to the rotatable input mechanism when the first user interface screen is displayed;

[0616] A means for determining the direction of the sweep relative to the rotatable input mechanism; and

[0617] A means for replacing the display of the first user interface screen with the display of the second user interface screen according to a determined direction of the swipe relative to the rotatable input mechanism.

[0618] 64. An apparatus comprising:

[0619] monitor;

[0620] Rotatable input mechanism;

[0621] A means for displaying a portion of an electronic document on the display;

[0622] Device for detecting a swipe gesture on the device at a position corresponding to the rotatable input mechanism;

[0623] A means for determining the direction of the swipe gesture relative to the rotatable input mechanism; and

[0624] A means for translating the display of the document according to the direction determined by the swipe gesture.

[0625] 65. An apparatus comprising:

[0626] monitor;

[0627] shell;

[0628] Rotatable input mechanism;

[0629] Audio output communicator;

[0630] Means for detecting contact on the device at a position corresponding to the rotatable input mechanism when the display is inactive and the device is in audio control mode, wherein the contact does not activate the display; and

[0631] Means for providing first audio control data to the audio output communicator in response to a detected position of the contact relative to the rotatable input mechanism at a first position, wherein the first audio control data is associated with an audio function. 66. An apparatus comprising:

[0632] monitor;

[0633] shell;

[0634] Rotatable input mechanism;

[0635] One or more processors;

[0636] Memory; and

[0637] One or more programs stored in memory, the one or more programs including instructions for performing the method according to any one of items 1 to 18.

[0638] 67. An apparatus comprising:

[0639] monitor;

[0640] Rotatable input mechanism;

[0641] One or more processors;

[0642] Memory; and

[0643] One or more programs stored in memory, the one or more programs including instructions for performing the method according to any one of items 19 to 23.

[0644] 68. An apparatus comprising:

[0645] monitor;

[0646] Rotatable input mechanism;

[0647] One or more processors;

[0648] Memory; and

[0649] One or more programs stored in memory, the one or more programs including instructions for performing the method according to any one of items 24 to 30.

[0650] 69. An apparatus comprising:

[0651] monitor;

[0652] Rotatable input mechanism;

[0653] One or more processors;

[0654] Memory; and

[0655] One or more programs stored in memory, the one or more programs including instructions for performing the method according to any one of items 31 to 34.

[0656] 70. An apparatus comprising:

[0657] monitor;

[0658] shell;

[0659] Rotatable input mechanism;

[0660] Audio output communicator;

[0661] One or more processors;

[0662] Memory; and

[0663] One or more programs stored in memory, the one or more programs including instructions for performing the method according to any one of items 35 to 50.

[0664] 71. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display, a housing, and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform a method according to any one of items 1 to 18.

[0665] 72. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform a method according to any one of items 19 to 23.

[0666] 73. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform a method according to any one of items 24 to 30.

[0667] 74. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display and a rotatable input mechanism, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform a method according to any one of items 31 to 34.

[0668] 75. A non-transitory computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device having a display, a housing, a rotatable input mechanism, and an audio output communicator, the one or more programs comprising instructions, when executed by the one or more processors, causing the electronic device to perform the method according to any one of items 35 to 50.

[0669] 76. An apparatus comprising a display, a housing, a rotatable input mechanism, and means for performing the method according to any one of items 1 to 18.

[0670] 77. An apparatus comprising a display, a rotatable input mechanism, and means for performing the method according to any one of items 19 to 23.

[0671] 78. An apparatus comprising a display, a rotatable input mechanism, and means for performing the method according to any one of items 24 to 30.

[0672] 79. An apparatus comprising a display, a rotatable input mechanism, and means for performing the method according to any one of items 31 to 34.

[0673] 80. An apparatus comprising a display, a housing, a rotatable input mechanism, an audio output communicator, and means for performing the method according to any one of items 35 to 50.

[0674] 81. An electronic device, comprising:

[0675] Display unit;

[0676] Housing unit;

[0677] Rotatable input mechanism unit;

[0678] A processing unit coupled to the display unit, the housing unit, and the rotatable input mechanism unit, the processing unit comprising:

[0679] A receiving unit configured to receive requests for display items;

[0680] Display enable unit, configured to enable the display of a user interface comprising multiple items on the display unit in response to receiving the request, wherein a focus selector is associated with one or more of the multiple items;

[0681] A detection unit is configured to detect contact at a position on the device corresponding to the rotatable input mechanism unit;

[0682] The execution unit is configured as follows:

[0683] In response to the detected position of the contact being in a first position relative to the rotatable input mechanism unit, a process is performed on the one or more items associated with the focus selector; and in response to the detected position of the contact being in a second position relative to the rotatable input mechanism unit, the process is abandoned on the one or more items associated with the focus selector, wherein the second position is different from the first position.

[0684] 82. The electronic device according to item 81, wherein the first position is located on the housing unit above the rotatable input mechanism unit.

[0685] 83. The electronic device according to item 81, wherein the first position is located on the rotatable input mechanism unit.

[0686] 84. The electronic device according to any one of items 81 to 83, wherein the contact does not activate a mechanically actuated input mechanism.

[0687] 85. The electronic device according to any one of items 81 to 84, wherein the contact does not include any contact with the display unit.

[0688] 86. An electronic device according to any one of items 81 to 85, wherein the one or more items are one or more text strings.

[0689] 87. The electronic device according to any one of items 81 to 86, wherein the processing unit further comprises:

[0690] A sending unit is configured to send the one or more items as electronic messages to an external device in response to detecting contact at the first position relative to the rotatable input mechanism unit, wherein performing the process includes sending the one or more items to the external device.

[0691] 88. The electronic device according to any one of items 81 to 87, wherein the processing unit further comprises:

[0692] A release unit is configured to release other items not associated with the focus selector in response to detecting contact at the first position relative to the rotatable input mechanism, wherein performing the process includes releasing the other items.

[0693] 89. The electronic device according to any one of items 81 to 88, wherein the processing unit further comprises:

[0694] A placement unit is configured to place one or more items in a text field in response to detecting contact at the first position relative to the rotatable input mechanism unit, wherein performing the process includes placing one or more items in the text field.

[0695] 90. The electronic device according to any one of items 81 to 89, wherein the contact is part of a tapping gesture.

[0696] 91. The electronic device according to any one of items 81 to 90, wherein the display unit is a touch-sensitive display unit.

[0697] 92. The electronic device according to any one of items 81 to 91, wherein the device includes one or more sensors configured to detect contact on the housing unit.

[0698] 93. The electronic device according to any one of items 81 to 92, wherein the device includes one or more sensors configured to detect contact on the rotatable input mechanism unit.

[0699] 94. The electronic device according to any one of items 81 to 93, wherein the sensor is a capacitive sensor.

[0700] 95. The electronic device according to any one of items 81 to 94, wherein the execution unit is further configured to perform a second process in response to the detected position of the contact being at a second position.

[0701] 96. The electronic device according to item 95, wherein the second process includes enabling the display of a second user interface on the display unit.

[0702] 97. An electronic device according to any one of items 95 to 96, wherein the second process is performed on one or more items associated with the focus selector.

[0703] 98. An electronic device according to any one of items 81 to 97, wherein the contact is detected to be located outside the display unit on the device.

[0704] 99. An electronic device comprising:

[0705] Display unit;

[0706] Rotatable input mechanism unit;

[0707] A processing unit coupled to the display unit and the rotatable input mechanism unit, the processing unit comprising:

[0708] Display enable unit, which is configured to enable the display of a user interface comprising multiple items on the display unit, wherein a focus selector is associated with a first group of one or more items among the multiple items;

[0709] A detection unit is configured to detect user input corresponding to the rotation of the rotatable input mechanism, wherein the user input includes contact with the rotatable input mechanism;

[0710] A scrolling unit is configured to scroll the plurality of items in response to detecting the rotation and associate the focus selector with a second group of one or more items based on the rotation;

[0711] A determining unit is configured to determine whether the contact has been removed from the rotatable input mechanism unit when one or more items in the second group are associated with the focus selector;

[0712] The execution unit is configured as follows:

[0713] Based on the determination that the contact has been removed when the second group of one or more items is associated with the focus selector, the process is performed on the second group of one or more items; and

[0714] Based on the determination that the contact has not been removed while the second item is in focus, the process for the second group of one or more items is abandoned. 100. The electronic device according to item 99, wherein the processing unit further comprises:

[0715] A release unit is configured to release an item that is not in one or more items in the second group based on a determination that the contact has been removed when the second group of one or more items is associated with the focus selector, wherein performing the process includes releasing the item.

[0716] 101. The electronic device according to any one of items 99 to 100, wherein the processing unit further comprises:

[0717] A placement unit is configured to place one or more items in a text field based on the determination that the contact has been removed when the second group of one or more items is associated with the focus selector, wherein performing the process includes placing the items in the text field.

[0718] 102. The electronic device according to any one of items 99 to 101, wherein the processing unit further comprises:

[0719] A sending unit configured to send one or more items of the second group in an electronic message to an external device, wherein performing the process includes sending one or more items of the second group in an electronic message to the external device.

[0720] 103. The electronic device according to any one of items 99 to 102, wherein the scrolling single

[0721] The element is further configured to, based on the determination that the contact has not been removed when one or more items in the second group are associated with the focus selector, and to continue detecting the contact with the selected item...

[0722] When the rotation of the rotatable input mechanism unit corresponds to the user input:

[0723] Scroll the items to display items that were not previously visible on the display unit, and

[0724] The focus selector is associated with one or more items in a third group based on the rotation.

[0725] 104. An electronic device, comprising:

[0726] Display unit;

[0727] Rotatable input mechanism unit;

[0728] A processing unit coupled to the display unit and the rotatable input mechanism unit, the processing unit comprising:

[0729] Display enable unit, which is configured to enable the display of a first user interface screen on the display unit;

[0730] A detection unit is configured to detect a swipe gesture originating from a position corresponding to the rotatable input mechanism unit when the first user interface screen is displayed; and

[0731] A determining unit is configured to determine the direction of the swipe relative to the rotatable input mechanism;

[0732] The display activation unit is further configured to enable the replacement of the display of the first user interface screen on the display unit with the display of the second user interface screen on the display unit, based on the determined direction of the swipe relative to the rotatable input mechanism unit.

[0733] 105. The electronic device according to item 104, wherein the swipe gesture includes a horizontal movement of contact on the rotatable input mechanism unit.

[0734] 106. The electronic device according to item 104, wherein the device further includes a housing unit.

[0735] And the swiping gesture mentioned therein includes a vertical movement that contacts the housing unit.

[0736] 107. The electronic device according to item 104, wherein the device further includes a housing unit, and wherein the swipe gesture includes a movement of contact on the housing unit along an axis perpendicular to the display unit.

[0737] 108. The electronic device according to items 104 to 107, wherein the swipe position is located above the rotatable input mechanism unit.

[0738] 109. The electronic device according to any one of items 104 to 108, wherein the swipe gesture is a first swipe gesture, and wherein the detection unit is further configured to detect a second swipe gesture starting from a position corresponding to the rotatable input mechanism unit and in the opposite direction to the first swipe gesture.

[0739] The display enabling unit is further configured to enable the replacement of the display of the second user interface screen on the display unit with the display of the first user interface screen on the display unit.

[0740] 110. The electronic device according to any one of items 104 to 109, wherein in the device

[0741] At least a portion of the swipe gesture is detected at a location outside the aforementioned display unit. 111. An electronic device comprising:

[0742] Display unit;

[0743] Rotatable input mechanism unit;

[0744] A processing unit coupled to the display unit and the rotatable input mechanism unit, the processing unit comprising:

[0745] Display enable unit, which is configured to enable the display of a portion of an electronic document on the display unit;

[0746] A detection unit is configured to detect a swipe gesture at a position on the device corresponding to the rotatable input mechanism unit;

[0747] A determining unit, configured to determine the direction of the swipe gesture relative to the rotatable input mechanism unit; and

[0748] A translation unit is configured to translate the display of the document on the display unit according to a direction determined by the swipe gesture.

[0749] 112. The electronic device according to item 111, wherein the display of panning the document includes displaying a second portion of the document, different from the first portion, on the display unit.

[0750] 113. The electronic device according to any one of items 111 to 112, wherein the device includes a housing unit, and at least a portion of the swiping gesture is on the housing unit.

[0751] 114. The electronic device according to any one of items 111 to 113, wherein in the device

[0752] At least a portion of the swipe gesture is detected at a location outside the aforementioned display unit. 115. An electronic device comprising:

[0753] Display unit;

[0754] Housing unit;

[0755] Rotatable input mechanism unit;

[0756] Audio output communicator unit; and

[0757] A processing unit coupled to the display unit, the housing unit, the rotatable input mechanism unit, and the audio output communicator unit, the processing unit comprising:

[0758] A detection unit configured to detect contact on the device at a position corresponding to the rotatable input mechanism unit when the display unit is inactive and the device is in audio control mode, wherein the contact does not activate the display unit; and

[0759] A providing unit is configured to provide first audio control data to the audio output communicator unit in response to the detected position of the contact at a first position relative to the rotatable input mechanism, wherein the first audio control data is associated with an audio function.

[0760] 116. The electronic device according to item 115, wherein the detection unit is further configured to detect the contact at a second position relative to the rotatable input mechanism, and

[0761] The providing unit is further configured to, in response to detecting the contact at the second location, relinquish the provision of the first audio control data to the audio output communicator.

[0762] 117. The electronic device according to item 116, wherein the providing unit is further configured to provide second audio control data to the audio output communicator unit in response to the detected location being at the second location, wherein the second audio control data is different from the first audio control data, and wherein the second audio control data is associated with a second audio function.

[0763] 118. The electronic device according to any one of items 115 to 117, wherein the contact does not activate a mechanically actuated input.

[0764] 119. The electronic device according to any one of items 115 to 118, wherein the detection unit is further configured to detect user input prior to detecting the contact, and the processing unit further comprises:

[0765] A mode entry unit is configured to enter the audio control mode in response to detecting the user input.

[0766] 120. The electronic device according to any one of items 115 to 119, wherein the first audio control data is associated with changing the volume of played audio content.

[0767] 121. The electronic device according to any one of items 115 to 119, wherein the first audio control data is associated with changing the audio content being played.

[0768] 122. The electronic device according to any one of items 115 to 121, wherein the contact includes movement of the contact, and wherein the determining unit is further configured to determine the direction of movement of the contact relative to the rotatable input mechanism unit, and

[0769] The first audio control data is based on the determined direction of movement.

[0770] 123. The electronic device according to item 122, wherein the providing unit is further configured to:

[0771] Based on determining that the movement is in a first direction relative to the rotatable input mechanism unit, first audio control data associated with reducing the volume of the playing audio content is provided; and

[0772] Based on determining that the movement is in a second direction different from the first direction, first audio control data associated with reducing the volume of the audio content being played is provided.

[0773] 124. The electronic device according to any one of items 122 to 123, wherein the providing unit is further configured to:

[0774] Based on the determination that the movement is in a third direction relative to the rotatable input mechanism unit, the audio content to be played is switched from the first track to the second track.

[0775] The associated first audio control data; and

[0776] Based on determining that the movement is in a fourth direction different from the third direction, first audio control data is provided that is associated with switching the audio content to be played from the first track to the third track.

[0777] 125. The electronic device according to any one of items 115 to 121, wherein the contact is a tap, and wherein the first audio control data is based on the position of the tap relative to the rotatable input mechanism unit.

[0778] 126. The method according to item 125, wherein in response to the detected position of the tap being at a first position relative to the rotatable input mechanism, the audio control data is associated with decreasing the volume of the audio content being played; and

[0779] The position in response to the detected tap is in a second position relative to the rotatable input mechanism, which is different from the first position, and the audio control data is associated with increasing the volume of the audio content being played.

[0780] 127. The electronic device according to items 125 to 126, wherein, in response to the detected tap being at a third position relative to the rotatable input mechanism unit, the audio control data is associated with switching the played audio content from a first track to a second track; and

[0781] The position in response to the detected tap is at a fourth position relative to the rotatable input mechanism unit, which is different from the third position, and the audio control data is associated with switching the audio content to be played from the first track to the third track.

[0782] 128. The electronic device according to any one of items 115 to 127, wherein the detection unit is further configured to detect a second contact on the touch-sensitive display unit, and wherein the processing unit further includes an activation unit configured to activate the display unit in response to detecting the second contact.

[0783] 129. The electronic device according to items 115 to 128, wherein the detection unit is further configured to detect user input corresponding to rotation of the rotatable input mechanism unit when in the audio control mode; and

[0784] The providing unit is further configured to provide audio control data associated with the audio function to the audio output communicator unit in response to detecting the user input.

[0785] 130. An electronic device according to any one of items 115 to 129, wherein the contact is detected to be located outside the display unit on the device.

[0786] For purposes of explanation, the foregoing description has been given by reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible based on the teachings above. These embodiments were chosen and described in order to best explain the principles of these techniques and their practical applications. Others skilled in the art will thus be able to best utilize these techniques and the various embodiments with various modifications suitable for the particular intended use.

[0787] While this disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. It should be understood that such changes and modifications are considered to be included within the scope of this disclosure and examples as defined by the claims.

Claims

1. A method for controlling a device with a housing, comprising: On a device having a housing, one or more sensors, and an audio output communicator: When the device is in audio control mode and the display is showing relevant content, movement of contact along the housing of the device is detected via the one or more sensors, wherein the user interface associated with the audio control mode does not require input on the display; as well as In response to movement of a contact along the housing of the device, an audio function associated with the device is executed without displaying content corresponding to the movement of the contact on the display and without causing the display to display different content different from the corresponding content, wherein the audio function is based on parameters of the movement of the contact along the housing of the device.

2. The method of claim 1, wherein the movement of the contact does not activate the mechanical actuation input.

3. The method according to any one of claims 1-2, further comprising: User input is detected before the movement of the contact is detected; and In response to the detection of the user input, the audio control mode is entered.

4. The method according to any one of claims 1-2, wherein the audio function is associated with changing the volume of the audio content being played.

5. The method according to any one of claims 1-2, wherein the audio function is associated with changing the audio content being played.

6. The method according to any one of claims 1-2, further comprising: The direction of movement of the contact is determined, wherein the audio function is based on the determined direction of movement along the housing of the device.

7. The method according to claim 6, wherein: Based on the determined movement of the contact in a first direction, the audio function includes increasing the volume of the audio content being played; and Based on the determination that the movement of the contact is in a second direction different from the first direction, the audio function includes reducing the volume of the audio content being played.

8. The method according to any one of claims 6-7, wherein: Based on the determination that the movement of the contact is in the third direction, the audio function includes switching the playing audio content from a first track to a second track; as well as Based on the determination that the movement of the contact is in a fourth direction different from the third direction, the audio function includes switching the playing audio content from the first track to the third track.

9. The method according to any one of claims 1-2, wherein the audio function is based on the determined amplitude of movement along the housing of the device.

10. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a device having a housing, one or more sensors, and an audio output communicator, the one or more programs comprising instructions for: When the device is in audio control mode and when the display is showing relevant content, movement of contact along the casing of the device is detected via the one or more sensors, wherein the user interface associated with the audio control mode does not require input on the display; and In response to movement of the contact along the housing of the device, an audio function associated with the device is performed without displaying content corresponding to the movement of the contact on the display and without causing the display to display different content different from the corresponding content, wherein the audio function is based on parameters of the movement of the contact along the housing of the device.

11. The computer-readable storage medium of claim 10, wherein the movement of the contact does not activate the mechanically actuated input.

12. The computer-readable storage medium according to any one of claims 10-11, wherein the one or more programs further comprise instructions for: Before detecting movement of the contact, user input is detected; and In response to the detection of the user input, the audio control mode is entered.

13. The computer-readable storage medium according to any one of claims 10-11, wherein the audio function is associated with changing the volume of the played audio content.

14. The computer-readable storage medium according to any one of claims 10-11, wherein the audio function is associated with changing the audio content being played.

15. The computer-readable storage medium according to any one of claims 10-11, wherein the one or more programs further comprise instructions for: The direction of movement is determined, wherein the audio function is based on the determined direction of movement along the housing of the device.

16. The computer-readable storage medium according to claim 15, wherein: Based on the movement in the first direction that determines the contact, the audio function includes increasing the volume of the audio content being played; and Based on the determination that the movement of the contact is in a second direction different from the first direction, the audio function includes reducing the volume of the audio content being played.

17. The computer-readable storage medium according to any one of claims 15-16, wherein: Based on the determination that the movement of the contact is in the third direction, the audio function includes switching the playing audio content from a first track to a second track; as well as Based on the determination that the movement of the contact is in a fourth direction different from the third direction, the audio function includes switching the playing audio content from the first track to the third track.

18. The computer-readable storage medium according to any one of claims 10-11, wherein the audio function is based on the determined amplitude of movement along the housing of the device.

19. A device having a housing, one or more sensors, and an audio output communicator, comprising: One or more processors; and A memory storing one or more programs configured to be executed by one or more processors, the one or more programs including instructions for: When the device is in audio control mode and the display is showing relevant content, movement of contact along the housing of the device is detected via the one or more sensors, wherein the user interface associated with the audio control mode does not require input on the display; and In response to movement of a contact along the housing of the device, an audio function associated with the device is executed without displaying content corresponding to the movement of the contact on the display and without causing the display to display different content different from the corresponding content, wherein the audio function is based on parameters of the movement of the contact along the housing of the device.

20. The device of claim 19, wherein the movement of the contact does not activate the mechanical actuation input.

21. The device according to any one of claims 19-20, wherein the one or more programs further comprise instructions for: Before detecting movement of the contact, user input is detected; and In response to the detection of the user input, the audio control mode is entered.

22. The device according to any one of claims 19-20, wherein the audio function is associated with changing the volume of the played audio content.

23. The device according to any one of claims 19-20, wherein the audio function is associated with changing the audio content being played.

24. The device according to any one of claims 19-20, wherein the one or more programs further comprise instructions for: The direction of movement of the contact is determined, wherein the audio function is based on the determined direction of movement along the housing of the device.

25. The device according to claim 24, wherein: Based on the determined movement of the contact in a first direction, the audio function includes increasing the volume of the audio content being played; and Based on the determination that the movement of the contact is in a second direction different from the first direction, the audio function includes reducing the volume of the audio content being played.

26. The device according to any one of claims 24-25, wherein: Based on the determination that the movement of the contact is in the third direction, the audio function includes switching the playing audio content from a first track to a second track; as well as Based on the determination that the movement of the contact is in a fourth direction different from the third direction, the audio function includes switching the playing audio content from the first track to the third track.

27. The device according to any one of claims 19-20, wherein the audio function is based on the determined amplitude of movement along the housing of the device.

28. A device having a housing and an audio output communicator, comprising: Apparatus for performing the method of any one of claims 1-2.

29. A computer program product comprising one or more programs configured to be executed by one or more processors of a device having a casing and an audio output communicator, said one or more programs comprising instructions for performing the method of any one of claims 1-2.

Images