User interface to improve one-handed operation of the device
By displaying the GUI and offset keyboard in the bottom area of the display screen of a portable electronic device, the problem of one-hand operation inconvenience caused by a large display screen is solved, and the convenience and operation efficiency of user interface interaction are improved.
Patent Information
- Application Number
- CN202110609514.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2015-06-25
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2035-06-25
AI Technical Summary
The large display of portable electronic devices makes one-handed operation inconvenient because the user has difficulty reaching the user interface elements in the top area of the display, causing the user to switch to two-handed operation or reposition the device.
By displaying a graphical user interface (GUI) and offset keyboard in the bottom area of the display, it makes it easier for the user to interact with the user interface objects of the application and toggle the operation mode by detecting specific gestures and inputs.
Improves the one-handed operation experience of portable electronic devices, allowing users to more easily access and use the features of the device without frequent switching to two-handed operation or repositioning the device.
Smart Images

Figure CN113407106B_ABST
Abstract
Description
[0001] This application is a divisional application of an invention patent application with an international application date of June 25, 2015, which entered the Chinese national phase on February 28, 2017, with Chinese national application number 201580046789.7 and invention name “User interface for improving one-handed operation of a device”. Technical Field
[0002] The present invention generally relates to electronic devices with touch-sensitive surfaces, including but not limited to electronic devices with touch-sensitive surfaces that detect input for displaying a user interface that improves one-handed operation of the electronic device. Background Art
[0003] As technology advances, portable electronic devices have become more compact than their predecessors. The compactness of portable electronic devices allows for easy loading of the devices. However, as the types of content that can be displayed on portable electronic devices increase, the small display screens of the devices make it increasingly difficult for users to easily consume the content on the devices. Therefore, portable electronic devices are becoming larger to allow users to more easily consume content through larger display screens. Summary of the invention
[0004] However, the larger display screens of portable electronic devices make it more difficult for users to operate the device with one hand. For example, in order to reach a user interface element of an application executed on a portable electronic device with a large display screen, the user must switch to two-handed operation of the device or the user must reposition the device in their hands. Thus, the large display screens of current portable electronic devices make one-handed operation of the device inconvenient and uncomfortable for users.
[0005] Embodiments herein describe modes of application on portable electronic devices that improve one-handed operation of the device. For example, embodiments herein describe an ergonomic mode of application that displays a graphical user interface (GUI) of the application in a bottom area of an electronic device display screen. By displaying the GUI in the bottom area of the display screen, a user can more easily interact with any user interface object of the application that could not be reached without switching to two-handed operation of the device or repositioning the electronic device in the user's hand.
[0006] In some embodiments, a computer-implemented method includes displaying an application in a first mode at an electronic device having a touch screen display. When in the first mode, the application is displayed as having a first height. When the application is displayed in the first mode, a first input corresponding to a request to display the application in the second mode is detected. In response to detecting the first input, the application is displayed in the second mode, wherein in the second mode, the application is displayed in the bottom area of the touch screen display. The application is displayed in the second mode as having a second height less than the first height. When the application is displayed in the second mode, a second input is detected at a position corresponding to the application on the touch screen display. In response to detecting the second input, it is determined whether the second input is a navigation input or an interactive input. Based on the determination that the second input is a navigation input, the application is navigated without switching back to the first mode. Based on the determination that the second input is an interactive input, a response to the interactive input is generated and the first mode is switched back from the second mode.
[0007] In addition, embodiments herein describe an ergonomic mode for a keyboard displayed on a display screen of a portable electronic device. During the ergonomic mode of the keyboard, the keyboard is offset toward the vertical edges of the display screen to allow a user to more easily reach keys on the keyboard that were previously inaccessible without the user switching to two-handed operation of the device or repositioning the electronic device in the user's hands.
[0008] A computer-implemented method includes, at an electronic device having a display capable of receiving input via the display, displaying a view of a first application and displaying a keyboard including a first set of keys while displaying the first application. The first set of keys is displayed at a first location on the display. The method also includes detecting a gesture at the display in a direction toward an edge of the display. In response to the detection, the first set of keys is resized in the direction of the gesture and the first set of keys is shifted to a second location on the display without shifting the view of the application.
[0009] It should be noted that the various embodiments of the calendar application described above can be combined with any other embodiments described herein. The features and advantages described in this specification are not all inclusive, and in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, description, and claims. In addition, it should be noted that the language used in this specification has been primarily selected for readability and instructional purposes, and may not have been selected to describe or define the subject matter of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1A is a block diagram showing a portable multifunction device with a touch-sensitive display according to some embodiments.
[0011] Figure 1Bis a block diagram illustrating example components for event processing according to some embodiments.
[0012] Figure 2 A portable multifunction device with a touch screen according to some embodiments is shown.
[0013] Figure 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.
[0014] Figure 4A An exemplary user interface for a menu of applications on a portable multifunction device is shown according to some embodiments.
[0015] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.
[0016] Figure 5A An exemplary user interface for a clock application according to some embodiments is shown.
[0017] Figure 5B , Figure 5C and Figure 5D Various gestures are shown on a user interface of a clock application for activating an ergonomic mode of a clock application according to some embodiments.
[0018] Figure 5E , Fig. 5F , Figure 5G , Figure 5H , Fig.5I , Figure 5J , Figure 5K and Figure 5L An exemplary user interface for an ergonomic mode for a clock application according to some embodiments is shown.
[0019] Fig. 6A An exemplary user interface for a calendar application according to some embodiments is shown.
[0020] Figure 6B An exemplary user interface for an ergonomic mode for a calendar application according to some embodiments is shown.
[0021] Fig. 7A An exemplary user interface for a home screen of a multifunction device according to some embodiments is shown.
[0022] Figure 7B , Figure 7C and Fig.7D An exemplary user interface for an ergonomic mode for a home screen of a multifunction device according to some embodiments is shown.
[0023] Fig. 8A An exemplary user interface for a folder according to some embodiments is shown.
[0024] Figure 8B An exemplary user interface for an ergonomic mode of a folder according to some embodiments is shown.
[0025] Fig.9A An exemplary search user interface is shown according to some embodiments.
[0026] Fig. 9B An exemplary user interface for search results according to some embodiments is shown.
[0027] Fig. 9C An exemplary user interface for an ergonomic mode of search results according to some embodiments is shown.
[0028] Fig. 10A An exemplary user interface for a multitasking view according to some embodiments is shown.
[0029] Fig. 10B An exemplary user interface for an ergonomic mode of a multitasking view is shown according to some embodiments.
[0030] Fig. 10C and Fig. 10D An exemplary user interface for an ergonomic mode of a multitasking view displayed on exemplary multifunction devices of different sizes is shown according to some embodiments.
[0031] Fig.11A An exemplary user interface of an email application and keyboard is shown according to some embodiments.
[0032] Fig. 11B and Fig.11F Gestures are shown for invoking an ergonomic mode for a keyboard according to some embodiments.
[0033] Fig. 11C , Fig.11G and Fig.11H An exemplary user interface for an ergonomic mode of a keyboard according to some embodiments is shown.
[0034] Fig.11D Gestures for scrolling through auxiliary keys of a keyboard are shown on a user interface of an ergonomic mode of a keyboard according to some embodiments.
[0035] Fig.11E Exemplary auxiliary keys for a keyboard according to some embodiments are shown.
[0036] Fig. 12AGestures are shown for resizing the keyboard while in ergonomic mode according to some embodiments.
[0037] Fig. 12B An exemplary user interface of a resized keyboard is shown according to some embodiments.
[0038] Fig.13A , Fig. 13B and Fig. 13C An exemplary user interface of a keyboard in landscape view is shown according to some embodiments.
[0039] Fig.13D and Fig.13E An exemplary user interface of a keyboard in landscape view displayed on exemplary multifunction devices of different sizes is shown according to some embodiments.
[0040] Fig.14A , Fig. 14B and Fig. 14C An exemplary user interface for a keyboard for improving key selection accuracy is shown according to some embodiments.
[0041] Fig.15 is a flow chart of a method for activating an ergonomic mode of an application according to some embodiments.
[0042] Fig.16 is a flow chart of a method for activating an ergonomic mode of a keyboard according to some embodiments.
[0043] Figure 17-Figure 18 is a functional block diagram of an electronic device according to some embodiments. DETAILED DESCRIPTION
[0044] Many portable electronic devices have a graphical user interface (GUI) for applications that can be executed on the device. Portable electronic devices with large display screens (e.g., diagonal dimensions greater than 5 inches) make it more difficult for a user to interact with the GUI of an application during one-handed operation of the device. For example, portable electronic devices typically display applications in full-screen mode. In the full-screen mode of the application (e.g., a first mode), the GUI of the application is displayed as having a height (e.g., a first height), the GUI corresponding to a GUI that occupies the entire display screen of the portable electronic device (e.g., all or substantially all), except for any area of the status bar for the device and / or the control area of the device. When the application is displayed in full-screen mode, the GUI includes one or more UI objects that cannot be touched by the user's fingers due to the size of the display screen during the one-handed operation of the device. That is, in full-screen mode, the application includes a UI object outside the touchable area of the touch screen 112. The user must switch from the one-handed operation of the device to the two-handed operation of the device or reposition the device in the user's hand in order to touch the UI object.
[0045] The embodiments herein describe a mode of application on a portable electronic device, which improves the one-handed operation of the device. For example, the embodiments herein describe an ergonomic mode (e.g., a second mode) of application, which displays the GUI of the application in the bottom area of the electronic device display screen. By displaying the GUI in the bottom area of the display screen, the user can more easily interact with any UI object of the application, which could not be touched before if the user did not switch to two-handed operation of the device or did not reposition the electronic device in the user's hand. Therefore, in the ergonomic mode of application, the UI object is located within the touchable area of the touch screen 112. For example, any UI object displayed at the top of the display screen in full screen mode is now closer to the bottom of the display screen because the GUI is displayed in the bottom area of the display screen during the second mode.
[0046] In addition, embodiments herein describe an ergonomic mode for a keyboard displayed on a display screen of a portable electronic device. During the ergonomic mode of the keyboard, the keyboard is offset toward the vertical edge of the display screen to enable a user to more easily reach keys on the keyboard that were previously inaccessible without the user switching to two-handed operation of the device or repositioning the electronic device in the user's hands. Thus, one-handed operation of a portable electronic device is improved through various embodiments of the ergonomic mode, which will be further described below.
[0047] Exemplary Devices
[0048] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be put into practice without these specific details. In other cases, well-known methods, processes, parts, circuits, and networks are not described in detail, thereby not unnecessarily making the various aspects of the embodiments difficult to understand.
[0049] It will also be understood that, although the terms "first", "second" etc. are used to describe various elements in some instances herein, these elements should not be limited by these terms. These terms are only used to distinguish an element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.
[0050] The terms used in the description of the various described embodiments herein are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms "a", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the items listed in association. It will also be understood that when the term "includes", "including", "comprises" and / or "comprising" is used in this specification, it specifies the presence of the features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groupings.
[0051] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined that" or "if [the condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [the condition or event]" or "in response to detecting [the condition or event]", depending on the context.
[0052] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple Watch from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices are optionally used, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or trackpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or trackpads).
[0053] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.
[0054] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, gaming applications, telephony applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.
[0055] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied from one application to the next and / or within a respective application. Thus, a common physical architecture of the device (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and transparent to the user.
[0056] Attention is now directed to embodiments of portable devices having touch-sensitive displays. Figure 1Ais a block diagram showing a portable multifunction device 100 with a touch-sensitive display 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a "touch screen" for convenience, and may sometimes be referred to as or referred to as a "touch-sensitive display system." The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.
[0057] As used in this specification and in the claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values that includes at least four different values and more typically includes hundreds of different values (e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., a weighted average) to determine an estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface adjacent to the contact and / or its change, and / or the resistance of the touch-sensitive surface adjacent to the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in pressure units). Using the intensity of the contact as an attribute of the user input allows the user to access additional device functionality that would not otherwise be accessible to the user on a smaller device with limited real estate, which is used to display an enable indication (e.g., on a touch-sensitive display) and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).
[0058] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device that will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of a device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or a touchpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuator button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. While such interpretation of touch by the user will be limited by the user's individualized sensory perceptions, there are many sensory perceptions of touch that are common to most users. Thus, when tactile outputs are described as corresponding to a particular sensory perception of a user (e.g., "up click," "down click," "roughness"), unless otherwise stated, the tactile outputs generated correspond to physical displacements of the device or a component thereof that would generate the described sensory perceptions for a typical (or average) user.
[0059] It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the EMBODIMENTS 2000 are implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.
[0060] The memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices. Access to the memory 102 by other components of the device 100 (such as the CPU 120 and the peripheral device interface 118) is optionally controlled by a memory controller 122.
[0061] Peripherals interface 118 may be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.
[0062] In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.
[0063] RF (radio frequency) circuit 108 receives and transmits RF signals, also called electromagnetic signals. RF circuit 108 converts electrical signals into / from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec (CODEC) chipsets, subscriber identity module (SIM) cards, memory, etc. RF circuit 108 optionally communicates with networks and other devices through wireless communications, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs). Wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Evolution-Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n).
[0064] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset having both output (e.g., a single-ear headset or a dual-ear headset) and input (e.g., a microphone).
[0065] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch screen 112 and other input control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, and the like. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. One or more buttons (e.g., Figure 2 208) optionally includes an up / down button for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2 206).
[0066] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch screen 112 and / or sends electrical signals to the touch screen 112. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.
[0067] The touch screen 112 has a touch-sensitive surface, sensor, or set of sensors that accept input from a user based on tactile and / or haptic contact. The touch screen 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to a finger of the user.
[0068] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch screen 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as the . iPod and Technology found in.
[0069] The touch screen 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen has a video resolution of about 160 dpi. The user optionally uses any suitable object or appendage such as a stylus, finger, etc. to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily through finger-based contacts and gestures, which may not be as precise as stylus-based input due to the larger contact area of the finger on the touch screen. In some embodiments, the device converts the rough finger-based input into a precise pointer / cursor position or command for performing the action desired by the user.
[0070] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch screen 112, or an extension of the touch-sensitive surface formed by the touch screen.
[0071] Device 100 also includes a power system 162 for powering the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power fault detection circuits, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in a portable device.
[0072] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the optical sensor 164 optionally captures a static image or video. In some embodiments, the optical sensor is located on the back of the device 100, opposite to the touch screen display 112 on the front of the device, so that the touch screen display can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device so that the user can optionally obtain the image of the user for video conferencing while viewing other video conference participants on the touch screen display.
[0073] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1AA contact force sensor is shown coupled to a force sensor controller 159 in I / O subsystem 106. Contact force sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). Contact force sensor 165 receives contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the back of device 100, opposite to touch screen display 112 located on the front of device 100.
[0074] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, the proximity sensor turns off and disables the touch screen 112 when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call).
[0075] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1A A tactile output generator is shown coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices such as a speaker or other audio component; and / or an electromechanical device that converts energy into linear motion such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component for converting an electrical signal into a tactile output on the device). The contact force sensor 165 receives tactile feedback generation instructions from the tactile feedback module 133 and generates a tactile output on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is arranged in parallel or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., forward and backward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on the back of the device 100, opposite the touch screen display 112 located on the front of the device 100.
[0076] Device 100 optionally also includes one or more accelerometers 168 . Figure 1AAn accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on an analysis of data received from one or more accelerometers. The device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to the one or more accelerometers 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100 as described in U.S. patent application No. 11 / 969,800 filed on January 4, 2008, which is incorporated herein by reference in its entirety.
[0077] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, such as Figure 1A and Figure 3 . The device / global internal state 157 includes one or more of the following: an active application state indicating which applications, if any, are currently active; a display state indicating what applications, views, or other information occupy various areas of the touch screen display 112; a sensor state including information obtained from the device's various sensors and input control devices 116; and position information regarding the device's position and / or posture.
[0078] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.
[0079] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FIREWIRE, etc.) is suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, similar to, and / or compatible with the 30-pin connector used on iPod (trademark of Apple Inc.) devices.
[0080] The contact / motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the contact strength (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single contact (e.g., one finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and display controller 156 detect contact on a touch pad.
[0081] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of the touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).
[0082] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact intensity increases, and / or before or after contact intensity decreases). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of contact intensity, the mean value of contact intensity, the average value of contact intensity, the value at the top 10% of contact intensity, the half-maximum value of contact intensity, the 90% maximum value of contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this embodiment, a contact whose feature strength does not exceed the first threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the third threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.
[0083] In some embodiments, a portion of a gesture is identified for determining a characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an ending position where the strength of the contact increases. In this embodiment, the characteristic strength of the contact at the ending position may be based on only a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only a portion of the swipe contact at the ending position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe contact before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the strength of the swipe contact to achieve the purpose of determining the characteristic strength.
[0084] The contact / motion module 130 optionally detects gesture input from a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another embodiment, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event.
[0085] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch screen 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.
[0086] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphics to be displayed from an application or the like, and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data for output to the display controller 156.
[0087] Haptic feedback module 133 includes various software components for generating instructions used by one or more tactile output generators 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .
[0088] Text input module 134, optionally as part of graphics module 132, provides a soft keyboard for entering text in various applications (eg, contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).
[0089] The GPS module 135 determines the location of the device and provides such information for use in various applications (e.g., to the phone 138 for location-based dialing, to the camera 143 as picture / video metadata, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and maps / navigation widgets).
[0090] Application 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:
[0091] Contacts module 137 (sometimes called an address book or contact list);
[0092] Telephone module 138;
[0093] Video conferencing module 139;
[0094] Email client module 140;
[0095] Instant messaging (IM) module 141;
[0096] Fitness support module 142;
[0097] A camera module 143 for still images and / or video images;
[0098] Image management module 144;
[0099] Browser module 147;
[0100] Calendar module 148;
[0101] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, and a user-created widget 149-6;
[0102] A widget creator module 150 for forming a user-created widget 149-6;
[0103] Search module 151;
[0104] Video and music player module 152, which optionally consists of a video player module and a music player module;
[0105] Notepad module 153;
[0106] Map module 154; and / or
[0107] Online video module 155.
[0108] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.
[0109] In combination with the touch screen 112, display controller 156, contact module 130, graphics module 132 and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (for example, in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting one or more names from the address book; associating one or more phone numbers, one or more email addresses, one or more physical addresses or other information with a name; associating images with names; categorizing and sorting names; providing a phone number or email address to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140 or IM 141, and the like.
[0110] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 is optionally used to enter a sequence of characters corresponding to a phone number, access one or more phone numbers in address book 137, modify an already entered phone number, dial a corresponding phone number, conduct a conversation, and disconnect or hang up when the conversation is complete. As described above, wireless communications optionally use any of a variety of communication standards, protocols, and technologies.
[0111] In combination with RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137 and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting and terminating a video conference between a user and one or more other participants in accordance with user instructions.
[0112] In conjunction with RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.
[0113] In conjunction with RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132 and text input module 134, instant messaging module 141 includes executable instructions for inputting a character sequence corresponding to an instant message, for modifying previously input characters, for transmitting a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), for receiving instant messages, and for viewing received instant messages. In some embodiments, the instant messages transmitted and / or received optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).
[0114] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the music player module 146, the fitness support module 142 includes executable instructions for the following operations: creating a fitness (e.g., with time, distance, and / or calorie burn goals); communicating with fitness sensors (sports equipment); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting fitness music and playing it; and displaying, storing, and transmitting fitness data.
[0115] In conjunction with touch screen 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132 and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, or deleting still images or videos from memory 102.
[0116] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.
[0117] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving and displaying web pages or parts thereof and attachments and other files linked to web pages) according to user instructions.
[0118] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.
[0119] In conjunction with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5) or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).
[0120] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134 and the browser module 147, the desktop widget creator module 150 is optionally used by a user to create a desktop widget (for example, converting a user-specified portion of a web page into a desktop widget).
[0121] In combination with the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos and / or other files that match one or more search conditions (e.g., one or more user-specified search terms) in accordance with user instructions.
[0122] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple Inc.).
[0123] In conjunction with the touch screen 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do lists, etc. according to user instructions.
[0124] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135 and the browser module 147, the map module 154 is optionally used to receive, display, modify and store maps and data associated with the maps (e.g., driving directions; data relating to stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.
[0125] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on the touch screen or on an external display connected via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, the link to a particular online video is sent using instant messaging module 141 instead of email client module 140.
[0126] Each of the above modules and applications corresponds to an executable instruction set for performing one or more of the above functions and the methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores additional modules and data structures not described above.
[0127] In some embodiments, device 100 is a device on which operation of a predefined set of functions is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operation of device 100, the number of physical input control devices (such as push buttons, dials, etc.) on device 100 is optionally reduced.
[0128] The predefined set of functions that are performed exclusively through the touch screen and / or trackpad optionally includes navigation between user interfaces. In some embodiments, the trackpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, a home screen menu, or a root menu. In such embodiments, a "menu button" is implemented using a trackpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a trackpad.
[0129] Figure 1B is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in the operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 137-13, 155, 380-390).
[0130] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event distributor module 174. In some embodiments, the application 136-1 includes an application internal state 192 that indicates one or more current application views displayed on the touch-sensitive display 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.
[0131] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when application 136-1 resumes execution, user interface state information indicating information being displayed by application 136-1 or information ready to be displayed by application 136-1, a state queue for enabling a user to return to a previous state or view of application 136-1, and a repeat / undo queue of previous actions taken by the user.
[0132] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 through audio circuit 110. The information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.
[0133] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).
[0134] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.
[0135] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can view on the display.
[0136] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to programmatic levels within the application's programming or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are considered to be correct input is optionally determined based at least in part on the hit view of the initial touch that started the touch-based gesture.
[0137] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.
[0138] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if the touch sub-event is completely confined to the area associated with one particular view, higher views in the hierarchy will still remain as actively participating views.
[0139] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.
[0140] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In other embodiments, event classifier 170 is a standalone module or is part of another module stored in memory 102, such as contact / motion module 130.
[0141] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring in a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event identifiers 180. Typically, the corresponding application view 191 includes multiple event identifiers 180. In other embodiments, one or more of the one or more event identifiers 180 are part of an independent module, such as a user interface toolkit (not shown) or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in the corresponding application view 191.
[0142] A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).
[0143] Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events, such as touches or touch movements. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).
[0144] Event comparator 184 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one embodiment, the definition of event 1 (187-1) is a double-click on a display object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a display object, a first lift-off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift-off (touch end) of a predetermined duration. In another embodiment, the definition of event 2 (187-2) is a drag on a display object. For example, the dragging includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch-sensitive display 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.
[0145] In some embodiments, event definition 187 includes a definition of an event for a corresponding user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with the sub-event and the object that triggered the hit test.
[0146] In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays the delivery of the event information until it has been determined whether the sub-event sequence does or does not correspond to the event type of the event identifier.
[0147] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definition 186, the corresponding event recognizer 180 enters an event not possible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.
[0148] In some embodiments, the corresponding event identifier 180 includes metadata 183 having configurable properties, tags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event identifiers. In some embodiments, metadata 183 includes configurable properties, tags, and / or lists that indicate how event identifiers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, tags, and / or lists that indicate whether sub-events are delivered to varying levels in a view or programming hierarchy.
[0149] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with the tag obtains the tag and performs a predefined process.
[0150] In some embodiments, the event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with a sub-event sequence or to an actively participating view. An event handler associated with a sub-event sequence or with an actively participating view receives the event information and executes a predetermined process.
[0151] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module 145. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 176 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.
[0152] In some embodiments, one or more event handlers 190 include or have access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.
[0153] It should be understood that the above discussion on event processing of user touches on a touch-sensitive display also applies to other forms of user input using input devices to operate the multifunction device 100, and not all user inputs are initiated on the touch screen. For example, mouse movement and mouse button presses, optionally combined with single or multiple keyboard presses or maintenance; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as inputs corresponding to sub-events defining the event to be recognized.
[0154] Figure 2A portable multifunction device 100 with a touch screen 112 according to some embodiments is shown. The touch screen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and in other embodiments described below, a user can select one or more of these graphics by, for example, making gestures on the graphics using one or more fingers 202 (not drawn to scale in the accompanying drawings) or using one or more styluses 203 (not drawn to scale in the accompanying drawings). In some embodiments, selection of one or more graphics will occur when the user interrupts contact with the one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of fingers that have been in contact with the device 100 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with a graphic will not select the graphic. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.
[0155] The device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. In some embodiments, the menu button 204 includes a fingerprint sensor that recognizes a fingerprint on the menu button 204. The fingerprint sensor is optionally used to determine whether the finger on the menu button 204 has a fingerprint that matches the fingerprint used to unlock the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112.
[0156] In one embodiment, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for powering on / off and locking the device, one or more volume adjustment buttons 208, a user identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The push button 206 is optionally used to: power on / off the device by pressing the button and keeping the button in a pressed state for a predetermined time interval; lock the device by pressing the button and releasing the button before a predetermined time interval passes; and / or unlock the device or initiate an unlocking process. In an alternative embodiment, the device 100 also accepts voice input for activating or deactivating some functions through a microphone 113. The device 100 also optionally includes one or more contact strength sensors 165 for detecting the strength of contact on the touch screen 112, and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.
[0157] Figure 3 300 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (e.g., a home or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuits (sometimes called chipsets) that interconnect system components and control communications between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touch screen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 167 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from one or more CPUs 310. In some embodiments, memory 370 stores information related to portable multifunction device 100 ( Figure 1A ) or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100 ( Figure 1A )'s memory 102 optionally does not store these modules.
[0158] Figure 3Each element in the above-mentioned identified element is optionally stored in one or more previously mentioned memory devices.Each module in the above-mentioned identified module corresponds to the instruction set for performing the above-mentioned functions.The above-mentioned identified module or program (that is, instruction set) need not be implemented as a separate software program, process or module, and therefore each subset of these modules is optionally combined or otherwise rearranged in various embodiments.In some embodiments, memory 370 optionally stores the above identified module and the subset of data structure.In addition, memory 370 optionally stores additional modules and data structures not described above.
[0159] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .
[0160] Figure 4A An exemplary user interface for a menu of applications on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:
[0161] a signal strength indicator 402 for one or more wireless communications, such as a cellular signal and a Wi-Fi signal;
[0162] Time 404;
[0163] Bluetooth indicator 405;
[0164] Battery status indicator 406;
[0165] A tray 408 with common application icons, such as:
[0166] an icon 416 labeled "Phone" of the phone module 138, the icon optionally including an indicator 414 of the number of missed calls or voice messages;
[0167] an icon 418 labeled “Mail” of the email client module 140 , the icon optionally including an indicator 410 of the number of unread emails;
[0168] An icon 420 labeled "Browser" of the browser module 147; and
[0169] An icon 422 labeled "iPod" of the video and music player module 152 (also called the iPod (trademark of Apple Inc.) module 152); and
[0170] Icons for other apps, such as:
[0171] An icon 424 labeled "Text" of the IM module 141;
[0172] An icon 426 labeled “Calendar” of the calendar module 148;
[0173] An icon 428 labeled “Photos” of the image management module 144;
[0174] An icon 430 labeled “Camera” of the camera module 143;
[0175] An icon 432 labeled “Online Video” of the online video module 155;
[0176] An icon 434 labeled "Stock Market" of the stock market desktop applet 149-2;
[0177] An icon 436 labeled “MAP” of the map module 154;
[0178] An icon 438 labeled “Weather” of the weather desktop applet 149-1;
[0179] An icon 440 labeled “clock” of the alarm clock desktop applet 149-4;
[0180] an icon 442 labeled “Fitness Support” of the fitness support module 142 ;
[0181] An icon 444 labeled "Notepad" of the notepad module 153; and
[0182] An icon 446 of a settings application or module that provides access to settings for the device 100 and its various applications 136;
[0183] It should be noted that Figure 4A The icon labels shown in are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.
[0184] Figure 4B A touch-sensitive surface 451 (eg, touch screen display 112) is shown having a touch-sensitive surface 451 (eg, Figure 3 A device (e.g., a tablet computer or touch pad 355) Figure 3Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357) for detecting intensity of contact on touch-sensitive surface 451, and / or one or more tactile output generators 359 for generating tactile output for a user of device 300.
[0185] Although some of the embodiments that follow will be given with reference to input on a touch screen display 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with the touch-sensitive surface 451 at a location corresponding to a corresponding location on the display (e.g., Figure 4B 460 and 462) (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g. Figure 4B When the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is separated, the device is used to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.
[0186] In addition, although the following embodiments are primarily given with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detection of contact, followed by termination of detecting contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and finger contacts are optionally used simultaneously.
[0187] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface that a user is interacting with. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 3 Touchpad 355 or Figure 4B In the event that an input (e.g., a press input) is detected on the touch-sensitive surface 451 in the display, the particular user interface element is adjusted according to the detected input. In the case that a touch screen display (e.g., Figure 1A A touch-sensitive display system 112 or Figure 4A In some implementations of the touch screen 112 in FIG. 1 , a detected contact on the touch screen acts as a “focus selector” such that when an input (e.g., a press input by the contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted in accordance with the detected input. In some implementations, the focus moves from one area of the user interface to another area of the user interface without corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user to convey the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a trackpad or touch screen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).
[0188] User interface and associated processes
[0189] Attention is now directed to embodiments of a user interface ("UI") and associated processes that may be implemented on an electronic device having a display and a touch-sensitive surface, such as device 300 or portable multifunction device 100. Although the following description is with respect to device 100, embodiments herein may be incorporated in device 300.
[0190] Figure 5A An exemplary user interface of a clock application is shown. Specifically, Figure 5AAn application view of a full screen mode (e.g., a first mode) of a clock application is shown. The full screen mode is optionally referred to as an extended mode or a two-handed operation mode. The full screen mode of the clock application includes a toolbar 501 and a content area 503. Although the following description is relative to a clock application, the embodiments herein are applicable to any application.
[0191] Toolbar 501 optionally includes selectable UI elements. Each UI element included in toolbar 501 is associated with a different function of the clock application. In some embodiments, toolbar 501 includes a world clock UI element 505 associated with a world clock function, an alarm UI element 507 associated with an alarm function, a stopwatch UI element 509 associated with a stopwatch function, and a timer UI element 511 associated with a timer function.
[0192] The content area 503 displays an application view of the application. Specifically, the device 100 displays the application view corresponding to the selected UI element from the toolbar 501 in the content area 503. Figure 5A In the embodiment shown in , since the world clock UI element 505 is selected, the world clock view is displayed in the content area 503 .
[0193] Typically, the world clock view displays the current time of one or more cities in the content area 503. Figure 5A As shown in , the content area 503 of the world clock view includes one or more rows, each of which is associated with the current time of a specific city. For example, row 513 shows the current time in Cupertino, California, row 515 shows the current time in Tokyo, Japan, and row 517 shows the current time in Manila, Philippines.
[0194] The content area 503 of the world clock application also includes one or more user UI elements for interacting with the clock application. For example, the content area 503 includes an add UI element 519 and an edit UI element 521. The add UI element 519 is used to add the current time of another city to the world clock view. For example, the user optionally selects the add UI element 519 to add the current time of Bologna, Italy to the world clock view. The edit UI element 521 is optionally used to modify the city represented in the world clock view. For example, the user selects the edit UI element 521 to delete one or more cities from the world clock view, or to modify the order in which the cities are arranged in the world clock view.
[0195] During one-handed operation of device 100 (e.g., when a user is holding the device in the palm of their hand), the user may not be able to reach one or more UI elements when an application is displayed in full-screen mode on touch screen 112, for example, because the UI elements are located at a location that is larger than a predefined reachability metric. In some embodiments, the reachability metric is based on an area of touch screen 112 that can be reached by a thumb of a typical user of device 100 when device 100 is held in the palm of the user. For example, the reachability metric is the highest point from the bottom of touch screen 112 that a typical user can reach from a first edge of touch screen 112 when extending the user's thumb from a bottom corner of device 100 adjacent to a second edge of touch screen 112. Thus, in some embodiments, the reachability metric is different for different devices (e.g., a wider device may have a smaller reachability metric than a narrower device because a typical user cannot reach as high on the far edge of the wider device as a user can reach on the far edge of the narrower device). In some embodiments, the reachability metric is determined based on the physical dimensions of the device 100 in addition to the size of the touch screen 112 , as wide bezels of the device 100 can reduce a user's ability to reach across the touch screen 112 .
[0196] If a UI element is located at a position that is larger than the reachability metric, the user's finger is generally unable to reach the UI element unless the electronic device is repositioned in the user's hand. Figure 5A In the embodiment shown in , when the clock application is displayed in full screen mode and when the user operates device 100 with one hand, the user cannot select at least one of edit UI element 521 and add UI element 519 because edit UI element 521 and add UI element 519 are located at a position larger than the reachability metric. Rather than the user shifting device 100 in the user's hand or switching to two-handed operation of device 100, the user is provided with an option to activate an ergonomic mode of the application to improve one-handed operation of device 100.
[0197] In some embodiments, the user provides input to device 100, causing device 100 to activate the ergonomic mode of the application. The user optionally provides multiple sequential inputs (e.g., double clicks) of menu button 204, such as by Figure 5A , to activate the ergonomic mode of the clock application. In some embodiments, device 100 detects multiple sequential inputs using a touch sensor included in menu button 204, which detects finger contact with menu button 204. The touch sensor is optionally a capacitive touch-sensitive surface or a fingerprint sensor integrated into a mechanical button.
[0198] In some embodiments, the menu button 204 has multiple states, such as a "half-pressed" state and a "full-pressed" state. The user optionally provides a half-press of the menu button 204 to activate the ergonomic mode of the clock application. When the menu button 204 is pressed less than a threshold amount, the device 100 detects the receipt of a half-press of the menu button 204. In contrast to the half-press of the menu button 204, when the menu button 204 is pressed greater than a threshold distance, a full press of the menu button 204 occurs, and causes the device 100 to perform different functions, such as switching from a full-screen mode or ergonomic mode of displaying an application to a home screen view of the device 100, as further described below. In some embodiments, the press of the menu button 204 is detected by a pressure sensor that detects a continuous range of pressure values, and the press is defined as an increase in the characteristic intensity (or pressure) of the contact on the button that is higher than a predetermined threshold. In some embodiments, the pressure sensor performs different operations when different pressure thresholds are met (e.g., a first pressure threshold corresponds to a half-press of the menu button 204, and a second pressure threshold higher than the first pressure threshold corresponds to a full press of the button).
[0199] Rather than providing a half-press or multiple sequential inputs of the menu button 204 to activate the ergonomic mode, the user optionally inputs a gesture on the application to activate the ergonomic mode of the application. Figure 5B A corner edge swipe gesture is shown at the lower right corner of the touch screen 112. In some embodiments, the corner edge swipe gesture is used by a user to activate an ergonomic mode of an application.
[0200] In some embodiments, device 100 determines input of a corner edge swipe gesture by detecting contact 523 on touch screen 112 at a first location adjacent to right vertical edge 525 of touch screen 112. Device 100 then detects contact 523 at a first location adjacent to right vertical edge 525 of touch screen 112. Figure 5B The corner swipe gesture is moved continuously in the direction of the arrow shown to a second position adjacent to a bottom horizontal edge 527 on the touch screen 112 that intersects with the right vertical edge 525, without contact 523 breaking contact with the touch screen 112. Finally, the device 100 detects completion of the corner edge swipe gesture in response to detecting finger lift at the second position, such that contact 523 breaks contact with the touch screen 112. In some embodiments, detecting the corner swipe gesture includes detecting that the contact has moved from a bezel of the device onto the touch screen 112 (e.g., by detecting that the contact gradually increases in size as it moves onto the touch screen).
[0201] Similarly, Figure 5C100 detects contact 523 on touch screen 112 at a first location adjacent to left vertical edge 529 of touch screen 112. Device 100 then detects contact 523 at a first location adjacent to left vertical edge 529 of touch screen 112. Figure 5C 112, and contact 523 continues to move in the direction of the arrow shown to a second position adjacent to bottom horizontal edge 527 on touch screen 112 that intersects left vertical edge 529, without contact 523 breaking contact with touch screen 112. Finally, device 100 detects completion of the corner edge swipe gesture in response to detecting finger lift-up at the second position, such that contact 523 breaks contact with touch screen 112.
[0202] In some embodiments, the user enters a gesture on the menu button 204 to activate the ergonomic mode of the application. Figure 5D A flick gesture on menu button 204 is shown to activate the ergonomic mode of the clock application. Device 100 determines that a flick gesture is input by detecting contact 523 of a user's finger with home button 204 via a sensor, such as a fingerprint sensor or a touch sensor. The flick gesture also includes a quick vertical movement of a finger placed on home button 204. For example, the flick gesture includes contact 523 at Figure 5D Rapid vertical movement in the direction of the arrow shown. The rapid vertical movement of contact 523 causes contact 523 to accelerate in the vertical direction and break contact with touch screen 112, thereby indicating completion of the gesture.
[0203] According to the device 100 detecting any of the above inputs for calling the ergonomic mode of the application, the device 100 switches from the full screen mode of the display application to the ergonomic mode of the application. When in ergonomic mode, the UI element previously positioned at a distance greater than the reachability metric is now located at a position less than the reachability metric. Therefore, there is no need to reposition the device 100 in the user's hand, and the user's finger is able to touch the UI element. In some embodiments, the ergonomic mode of the application can only be displayed in a longitudinal view. Alternatively, the ergonomic mode of the application can be displayed in a longitudinal view or a transverse view.
[0204] Figure 5EAn exemplary user interface of an ergonomic mode for a clock application is shown. In some embodiments, the ergonomic mode is also referred to as a compact mode or a one-handed operation mode. Typically, in the ergonomic mode of an application, the device 100 displays the application in the bottom area of the touch screen 112, and the height of the application (e.g., the second height) is less than the height of the application when displayed in full screen mode (e.g., the first height). For example, the height of the application displayed in the ergonomic mode corresponds to half the height of the touch screen 112. However, the height of the application in the ergonomic mode can be any height that is less than the height of the touch screen 112.
[0205] like Figure 5E As shown in , during the ergonomic mode, the content area 503' of the world clock view of the calendar application is displayed in the bottom area 531 of the touch screen 112, and a blank screen 533 (blank area) is displayed in the top area 535 of the touch screen 112 located above the content area 503'. In order to display the ergonomic mode of the application, the device 100 determines an application view of the application in full screen mode including the content area 503 and the toolbar 501 of the application. For example, the device 100 captures the world clock view of the clock application displayed in full screen, as shown in FIG. Figure 5A . The device 100 shifts the captured application view in the vertical direction until the content area 503' occupies the bottom area 531 of the touch screen 112 having a second height, the second height corresponding to the ergonomic mode, so that the top of the content area 531 is displayed in the bottom area 531. For example, the device 100 shifts the full screen view of the world clock view downward until the bottom area 531 of the touch screen 112 is occupied by the content area 503' of the world clock view, so that the top portion of the content area 503 including the add UI element 519 and the edit UI element 521 is displayed in the bottom area 531. In some embodiments, rather than capturing a static image of the user interface, the device 100 shifts the canvas on which the application is drawn so that the application does not need to be notified that the device 100 is operating in ergonomic mode (for example, the application is able to keep updating the user interface and respond normally to input from the user, but the drawing canvas and the touch canvas are offset on the display so that the top portion of the drawing canvas and the top portion of the touch canvas are offset toward the bottom of the touch screen).
[0206] Thus, the ergonomic mode of the application includes only the content area 503' of the application, and in some embodiments does not include the toolbar 501 of the application. Figure 5EAs shown, the ergonomic mode of the world clock view of the clock application includes only the content area 503' displayed in the bottom area 531 of the touch screen 112 without the toolbar 501. In addition, the ergonomic mode of the world clock view of the clock application is displayed with a height less than the height of the clock application displayed in full screen mode.
[0207] Fig. 5F Another exemplary user interface for the ergonomic mode of the clock application is shown. Rather than displaying a blank screen 533 in the top area 535 during the ergonomic mode of the application, the device automatically displays a notification view 537 in the top area 535 of the touch screen 112 in response to a user request to display the application in the ergonomic mode, as shown in FIG. Fig. 5F As shown. Notification view 537 includes one or more notifications for the user. Examples of notifications included in notification view 537 are the current date, weather and / or traffic information for the user's current location. Other types of notifications may be included in notification view 537. In some embodiments, the notification view can also be displayed in normal operating mode (e.g., in response to a swipe gesture from the top edge of the touch screen), in which case the notification view is dragged above the application user interface (e.g., at the top).
[0208] Figure 5G Another exemplary user interface for the ergonomic mode of the clock application is shown. Figure 5E , when in ergonomic mode, the device 100 automatically displays a blank screen 533 in the top area 535 of the touch screen 112. However, Fig. 5F Embodiments include displaying relative to Fig. 5F The handle UI element 539 associated with the notification view 537. The device 100 detects Figure 5G 523 on the handle UI element 539, and the continued movement of the contact 523 causes the handle UI element 539 to move in a horizontal direction. Figure 5G Move in the direction of the arrow shown (e.g., drag downward).
[0209] like Figure 5H As shown in , when the user drags handle UI element 539 downward in the vertical direction, device 100 displays a portion of notification view 537 in top area 535, the height of which is proportional to the amount of movement of handle UI element 539 in the vertical direction. Device 100 displays a blank screen 533 in the remaining portion of top area 535 not occupied by notification view 537. If the user drags handle UI element 539 to the bottom of top area 535, device 100 displays the entire notification view 537, so that notification view 537 occupies the entire top area 535 of touch screen 112, as shown in FIG. Fig.5IThe bottom of the top area is the boundary between the top area 535 and the bottom area 531. In some embodiments, the notification view 537 is displayed in full screen mode, rather than occupying only the top area 535.
[0210] In some embodiments, if the handle UI element 539 is dragged downward less than a threshold distance, the device 100 displays a portion of the notification view that is proportional to the amount of movement of the handle UI element 539 in the vertical direction in the top area 535 as described above. The threshold distance is optionally ¾ of the height of the top area 535 relative to the top of the touch screen 112. Alternatively, the threshold distance is 1 / 3 of the height of the touch screen 112 relative to the top of the touch screen 112. In some embodiments, if the handle UI element 539 is dragged downward from the top of the touch screen 112 more than the threshold distance, the device 100 automatically displays the entire notification view 537 in the top area 535, so that the notification area 537 occupies the entire top area 535 of the touch screen 112.
[0211] To display notification view 537 based on a threshold distance, device 100 detects contact 523 on handle UI element 539, such as Figure 5G Device 100 detects a continuous movement of contact 523 that causes handle UI element 539 to move in a direction of continuous movement having a vertical component. If device 100 detects that the vertical component is less than a threshold distance, device 100 determines the magnitude of the vertical component of the continuous movement and displays a portion of notification view 537 in top area 535 that is the same height as the vertical component of the continuous movement. Figure 5H 112. The device 100 may display the notification view 537 in the top area, thereby occupying the entire top area 535 of the touch screen 112. Alternatively, the device 100 displays the notification view 537 in full screen mode. In some embodiments, in response to detecting a downward gesture from the top edge of the touch screen 112, the handle UI element 539 is not displayed and the notification view 537 is displayed.
[0212] Figure 5J Another exemplary embodiment of the ergonomic mode of application is shown. Figure 5E As described, during the ergonomic mode, the device 100 displays a content area 503' of a world clock view of a calendar application without a toolbar 501 in a bottom area 531 of the touch screen display 112, and displays a blank screen 533 in a top area 535 located above the content area 503. Figure 5EIn the embodiment, the world clock view of the clock application displayed in full screen mode is only offset vertically downward to the bottom area 531. In contrast, in some embodiments, during ergonomic mode, the device 100 resizes the application view displayed in full screen mode to fit into the bottom area 531 of the touch screen 112, such as Figure 5J As shown. Figure 5J As shown, the world clock view of the clock application is resized so that the content area 503 ″ and the toolbar 501 of the world clock view are displayed in the bottom portion 531 of the touch screen 112 during the ergonomic mode.
[0213] To display the resized application, the device 100 first determines an application view of the application that is in full screen mode and includes both the content area 503 and the toolbar 501. That is, the device 100 creates a view of the application that includes both the content area 503 and the toolbar 501. The device 100 creates the view so that it can fit in the bottom portion 531 of the touch screen 112. The content (e.g., text, images, etc.) and toolbar of the application are resized by the device 100 according to formatting rules that ensure that the content and toolbar can still be read by the user.
[0214] Figure 5K Another exemplary embodiment of an ergonomic mode of an application is shown. As previously described, during ergonomic mode, the device 100 displays a content area 503' of a world clock view of a calendar application with or without a toolbar 501 in the bottom area 531 of the touch screen display 112. Figure 5E , Fig. 5F , Figure 5G and Figure 5J , device 100 optionally displays a background image 540 of device 100 in top area 535 of touch screen 112. As will be further described below, background image 540 (also referred to as "wallpaper") corresponds to an image displayed on device 100 when a home screen view of device 100 is displayed on touch screen 112.
[0215] exist Figure 5K In the embodiment shown in , the background image 540 includes a picture of the sun. By displaying the background image 540 in the top area 535 instead of the blank screen 533, the screen occupancy area is maximized and the user's visual experience is improved. In some embodiments, when the ergonomic mode of the application is activated, the icons and / or desktop applets displayed on the main screen are offset downward, while the background image 540 of the main screen view remains in a fixed or relatively fixed position on the touch screen 112.
[0216] When an application is displayed in ergonomic mode, the application may interact with the user (e.g., respond to user input) based on the type of input received by device 100. In some embodiments, device 100 is optionally enabled to receive navigation input while in ergonomic mode. Navigation input includes a request to scroll through the content of an application, scroll through a list, or scroll through different views of an application. Based on device 100 determining that navigation input was received, device 100 navigates within the application without switching back to full screen mode for the application. For example, in Figure 5E to Figure 5K In any of the figures, device 100 optionally responds to navigation input and Figure 5E and Figure 5K In some embodiments, device 100 detects a request to scroll through the content of an application by detecting a contact at a first location on touch screen 112, and detects continuous movement of the contact to a second location on touch screen 112. The continuous movement has a lateral (e.g., horizontal) component or a horizontal component of the movement without the contact breaking contact with touch screen 112. Based on detecting the continuous movement, device 100 scrolls to a different view of the application.
[0217] Although application can receive navigation input when being displayed in ergonomic mode, in some embodiments, this application can not receive object interaction input when being displayed in ergonomic mode.According to some embodiments, contrary to navigation input, object interaction input is the user input of the user interface object interaction with application.The embodiment of object interaction input is that user selects to add UI element 519 or edit UI element 521.If device 100 determines to receive object interaction input when being displayed in ergonomic mode, then device 100 generates the response to object interaction input and switches from displaying application in ergonomic mode to displaying application in full screen mode.For example, if device 100 detects the user selection to adding UI element 519, then device 100 generates the view of clock application for adding another city to world clock view, and switches from the ergonomic mode of displaying world clock view to displaying generated view in full screen mode.
[0218] In some embodiments, the device 100 automatically switches from displaying the application in ergonomic mode to full screen mode based on time. The device 100 determines the amount of time after the user input to the application was last received when the application was displayed in ergonomic mode. In order to determine the amount of time, the device 100 optionally executes a timer that counts the amount of time after the input to the application was last received during the ergonomic mode. If the device 100 determines that the amount of time exceeds a threshold, the device 100 automatically switches from displaying the application in ergonomic mode to full screen mode. However, if the user input is received before the amount of time exceeds the threshold, the device 100 resets the timer.
[0219] In some embodiments, after entering the ergonomic mode and before the input is detected while in the ergonomic mode, the device 100 first uses the first time threshold to determine whether to automatically switch back to the full screen mode of the application. Before receiving the input while in the ergonomic mode, the device uses the first time threshold to automatically determine whether to return to the full screen mode, so that if the input is received within the first time threshold, the device 100 switches back to the full screen mode. Before the first time threshold expires, after receiving the input while in the ergonomic mode, instead of resetting the first time threshold in response to receiving the previous user input, the device 100 reduces the threshold to a second time threshold that is shorter than the first time threshold. According to the device 100 receiving another input to the application when the amount of time since the last input to the application is less than the second time threshold, the device 100 increases the threshold. By receiving additional input from the user before the second time threshold expires, the device 100 determines that the user intends to interact with the application when the application is displayed in the ergonomic mode, thereby increasing the second time threshold. However, if the amount of time since the last input to the application was received exceeds the second time threshold, the device 100 automatically switches from displaying the application in the ergonomic mode back to the full screen mode.
[0220] In some embodiments, the device 100 switches back to full-screen mode from displaying the application in ergonomic mode upon user request. Specifically, if the device 100 detects a gesture that is repeated for activating the ergonomic mode of the application when the application is displayed in ergonomic mode, the device 100 switches back to full-screen mode from displaying the application in ergonomic mode. For example, an edge swipe gesture detected by the device 100 causes the device 100 to activate the ergonomic mode of the application, and the device 100 detects a repetition of the edge swipe gesture when the ergonomic mode of the application is displayed on the touch screen 112, and the device 100 switches from displaying the application in ergonomic mode to full-screen mode. Alternatively, if the device 100 detects any type of input that can be used to activate ergonomic mode (e.g., a corner edge swipe, a tap gesture on the menu button 204) when the application is already displayed in ergonomic mode, the device 100 switches from displaying the application in ergonomic mode to full-screen mode. In some embodiments, while the application is displayed in ergonomic mode, input on the blank screen 533 or background image 540 (e.g., a tap gesture, a double-tap gesture, a swipe gesture) also causes the device 100 to switch from displaying the application in ergonomic mode to full-screen mode. In some embodiments, if a reversal of the gesture used to activate the ergonomic mode of the application is received, the device 100 switches from displaying the application in ergonomic mode back to full-screen mode.
[0221] In some embodiments, the ergonomic mode of the application is automatically locked by the device 100 until the device 100 detects that the menu button 204 is fully pressed. If the ergonomic mode of the application is locked, the device 100 prevents (or abandons) switching back to the full screen mode of the application from the ergonomic mode. Therefore, the device 100 does not switch back to the full screen mode of the application from the ergonomic mode of the application in response to the interactive input. The device 100 optionally ignores any object interaction input received from the user when the application is displayed in the ergonomic mode. Similarly, the device 100 does not switch back to the full screen mode of the application from the ergonomic mode of the application based on time. Therefore, in some embodiments, the ergonomic mode is exited only in response to the device 100 detecting the full press of the menu button 204.
[0222] The ergonomic mode of the application is optionally locked by the device 100 upon user request. When in ergonomic mode, the device 100 optionally displays a user interface element associated with preventing switching from ergonomic mode to full screen mode. For example, Figure 5LA lock user interface element 541 is shown included in an ergonomic mode of a world clock view of a clock application. If device 100 detects a selection of lock user interface element 541, device 100 locks the application in ergonomic mode. In some embodiments, if ergonomic mode is locked, device 100 does not automatically switch from the ergonomic mode of the application to the full screen mode of the application after a threshold amount of time since the last input was received.
[0223] In some embodiments, even if the application is released, the application is also locked in the ergonomic mode. For example, the device 100 receives input from the user to exempt the application from being displayed on the touch screen 112 when the application is locked in the ergonomic mode. Later, the device 100 receives input from the user to open the application. In response, the device 100 automatically displays the application in the ergonomic mode because the ergonomic mode of the application is locked. Although the application is locked in the ergonomic mode, the user can unlock the ergonomic mode by reselecting the lock user interface element 541.
[0224] In some embodiments, the ergonomic mode of the application persists across all applications of the device 100. While displaying a first application in ergonomic mode, the device 100 receives an input for closing the first application, and the device 100 closes the first application based on the input. The device 100 then receives an input for opening a second application, and in response to receiving the input for opening the second application, automatically displays the second application in ergonomic mode rather than full screen mode. Therefore, the device 100 automatically displays the second application in ergonomic mode because the ergonomic mode was activated in the first application before the first application was closed.
[0225] Alternatively, if the application is closed when the application is displayed in the ergonomic mode, the device 100 automatically deactivates the ergonomic mode of the application. For example, when the application is displayed in the ergonomic mode, the device 100 receives an input for closing the application and closes the application according to the input. The device 100 then receives an input for opening the application and automatically displays the application in full screen mode in response to the input. Therefore, the ergonomic mode of the application is deactivated when the application is closed.
[0226] In some embodiments, if the application is closed while being displayed in ergonomic mode, the ergonomic mode of the application persists in the application. For example, when the application is displayed in ergonomic mode, the device 100 receives an input for closing the application and closes the application according to the input. The device 100 then receives an input for opening the application and automatically displays the application in ergonomic mode in response to the input. Therefore, even if the application is closed, the ergonomic mode of the application persists in the application.
[0227] Fig. 6A is an exemplary user interface of a calendar application. Specifically, Fig. 6A The application view of the full screen mode of the calendar application is shown. Similar to the full screen mode of the clock application described above, the full screen mode of the calendar application includes a toolbar 601 and a content area 603.
[0228] Toolbar 601 includes selectable UI elements that are each associated with a different function of the calendar application. In some embodiments, toolbar 601 includes today's events UI element 605 associated with events of the current day, calendar UI element 607 associated with calendar settings, and inbox UI element 609 associated with received calendar invitations. Device 100 displays the application view corresponding to the selected UI element in content area 603. Fig. 6A In the embodiment shown in , due to selection of today UI element 605 , the today view is displayed in content area 603 .
[0229] In some embodiments, the today view includes several different calendar UI views. The calendar UI view includes a year view, a month view, and a date view. Specifically, Fig. 6A Displays the calendar app's year view in full screen mode. Fig. 6A As shown, content area 603 displays the twelve months of a calendar year, and toolbar 601 includes the above-mentioned selectable UI elements.
[0230] In response to device 100 detecting an input for activating the calendar application in the ergonomic mode, device 100 displays the ergonomic mode of the calendar application, such as Figure 6B As previously described, in the ergonomic mode of applications, the device 100 displays applications in the bottom area 531 of the touch screen 112, and the height of the applications is less than the height of the applications when displayed in the full screen mode. Figure 6B As shown in , during ergonomic mode, a content area 603' of a year view of a calendar application is displayed in the bottom area 531 of the touch screen 112, and a blank screen 533 is displayed in the top area 535 located above the content area 603'. Alternatively, during ergonomic mode of the calendar application, a background image 540 or a notification view is displayed in the top area 535 instead of the blank screen 533. During ergonomic mode, the device 100 displays six months of the calendar year in the content area 603 instead of the twelve months of the calendar year. In some embodiments, the toolbar 601 of the calendar application is not displayed during ergonomic mode, while in other embodiments, the toolbar 601 is displayed during ergonomic mode.
[0231] Fig. 7AAn exemplary user interface of the device 100 home screen view 700 is shown. Specifically, Fig. 7A The main screen view 700 of the device 100 displayed in full screen mode is shown. The main screen view 700 includes multiple application user interface (UI) elements (e.g., message icon 424, calendar icon 426, photo icon 428) and tray 408, which are called icons or application icons in the foreground. In some embodiments, tray 408 is also called a taskbar (dock). In some embodiments, each icon is associated with a corresponding application. The main screen view 700 optionally also includes one or more software desktop applets, each associated with its own function. The user can select an icon from the main screen view 700 to execute the corresponding application on the device 100. In some embodiments, the main screen view 700 is the home page of the device 100 and is displayed in the content area of the touch screen 112 in response to the user selection of the menu button 204. The main screen view 700 optionally also displays a background image (e.g., an image of the sun) in the background of the view.
[0232] like Fig. 7A As shown in , the home screen view 700 includes a page indicator 701, which indicates the number of pages of icons that can be displayed on the touch screen 112. If the home screen view includes multiple pages, the page indicator 701 visually indicates the specific page number of the home screen view 700 currently displayed on the touch screen 112. In some embodiments, the page indicator 701 includes multiple shapes (e.g., circles), each shape is associated with a specific page of the home screen view 700. Fig. 7A In the example, one of the shapes is filled to visually indicate a page currently displayed on the touch screen 112.
[0233] For example, in Fig. 7A , the first circle is filled to visually indicate that the first page of the home screen view is currently displayed on the touch screen 112. Fig. 7A As shown in , the first page of the home screen view includes four rows of different icons. In response to the user input to display the home screen 700 in ergonomic mode, the device 100 displays the ergonomic mode of the home screen 700 ', such as Figure 7B As shown. Figure 7BIn the illustrated ergonomic mode of home screen 700', device 100 displays home screen view 700' in bottom area 531 of touch screen 112, the height of the home screen view being less than the height of the home screen view when displayed in full screen mode. Specifically, during the ergonomic mode of home screen view 700, the content area of home screen view 700' is displayed in bottom area 531 of touch screen 112 and background image 540 is displayed. Device 100 shifts home screen view 700' downward in the vertical direction until a home screen view having a height less than the height of the home screen view in full screen mode is displayed in the bottom area of touch screen 100. Optionally, a blank screen 533 is displayed in the top area instead of background image 540.
[0234] In some embodiments, the ergonomic mode of the home screen view includes only a portion of the row of icons and the tray 408. For example, Fig. 7A Unlike the four rows of icons included in the full screen mode shown in FIG, the ergonomic mode of the main screen view 700′ includes the following Figure 7B The three rows of icons shown. During Ergonomic Mode, the last row of icons included in Full Screen Mode is not displayed.
[0235] During Ergonomic Mode, the user can Figure 7B The swipe gesture 703 shown navigates through icons on different pages. Figure 7B The device 100 displays a swipe gesture 703 in the direction of the arrow shown. Figure 7C Another page of icons (e.g., a second home screen view) is shown. Page indicator 701 is also updated to visually indicate that the second page of icons is currently being displayed by filling the second circle and removing the fill of the first circle. In some embodiments, if device 100 returns to full screen mode when displaying the second page of icons, device 100 displays a full screen view of the second page of icons on touch screen 112.
[0236] Fig.7D An alternative embodiment of an ergonomic mode of the home screen view of device 100 is shown. Fig.7D , the home screen view 700″ includes a row of icons and a page indicator 701. However, Fig.7D The home screen view 700″ shown in FIG. Figure 7B Tray 408 shown in .
[0237] In some embodiments, device 100 allows reducing clutter on device home screen view 700 by grouping icons together in folders. Device 100 creates a folder in response to a user selecting a first icon from home screen view 700 and dragging the first icon from home screen view onto a second icon. Device 100 creates a folder in home screen view 700 that includes the first icon and the second icon and creates an icon to represent the created folder.
[0238] Fig. 8A An exemplary user interface of a folder 800 according to some embodiments is shown. In response to selection of an icon representing a folder in the home screen view 700 shown in FIG. 7 , the folder 800 is displayed on the touch screen 112. Fig. 8A , folder 800 is displayed in full screen mode. Folder 800 includes an icon for the application "Reminder" and an icon for the application "Notes". Folder 800 can optionally be displayed in ergonomic mode. In response to device 100 receiving an input for activating the ergonomic mode of folder 800, device 100 switches from the full screen mode of displaying folder 800 to the ergonomic mode of folder 800. The input for activating ergonomic mode is multiple sequential presses of menu button 204 represented by contact 523. However, as previously described, other inputs for activating ergonomic mode can be received.
[0239] Figure 8B An exemplary user interface of a folder 800' in ergonomic mode is shown. When in ergonomic mode, the device 100 displays the folder 800 in the bottom area 531 of the touch screen 112. Thus, the device 100 vertically offsets the folder 800' downward so that the icons included in the folder 800' are more easily accessible to the user's fingers without the user having to reposition the device 100 in the user's hand. In ergonomic mode, the device 100 continues to display the background image 540 in the background of the touch screen 112. In an alternative embodiment, a blank screen 533 or notification view is displayed in the top area above the folder 800'.
[0240] In some embodiments, device 100 allows searching for content on device 100. To activate the search function, the user optionally enters a gesture associated with the search function (e.g., a downward swipe) on home screen view 700. Upon detecting a gesture for activating the search function, device 100 displays a search UI. Fig.9A An exemplary user interface of a search function according to some embodiments is shown. Note that although the search function is activated in the home screen view 700, the search function can be activated in other applications, such as a mail application, a messaging application, a calendar application, etc.
[0241] like Fig.9A As shown in , the search function UI includes a search field 900 and a keyboard 901 displayed together with the main screen view 700. The keyboard 901 is optionally displayed below the area where the main screen view 700 is displayed, and the search field 900 is optionally displayed above the area where the main screen view 700 is displayed. The keyboard 901 and the search field 900 can be displayed on Fig.9A other locations than those shown.
[0242] In some embodiments, the keyboard 901 is a soft keyboard or a virtual keyboard. The device 100 automatically displays the keyboard 901 and the search field 900 when the search function is activated. The search field 900 is updated to include any search query entered into the device 100 by the user using the keyboard 901. Fig. 9B 900 and the search results are shown according to some embodiments. Fig. 9B As shown in FIG, search field 900 includes the query "calendar", and device 100 replaces home screen view 700 with search results 903. The search results include content on device 100 that matches the search query, and optionally also include one or more remote search results (e.g., Internet search results or results retrieved from a remote server, such as movie shows or current events). Fig. 9B In the embodiment shown in , a "calendar" query results in the device 100 identifying search results 903 that include a calendar icon 426 that, when selected, activates a calendar application.
[0243] In some embodiments, search results 903 may be displayed in an ergonomic mode. Upon detecting input for displaying the search results in an ergonomic mode, device 100 deactivates keyboard 901 and shifts search results 903 vertically toward the bottom area of touch screen 112, such as Fig. 9C In some embodiments, the device 100 displays an animation of the search result 903 shifting downward, and when the search result 903 shifts downward, the keyboard 901 moves away from the touch screen 112. Fig. 9C As shown in , when displaying search results 903' in ergonomic mode, the search field 900 is still displayed on the touch screen 112. When in ergonomic mode, the search results 903' are now more easily accessible to the user's fingers without the user having to reposition the device 100 in the user's hand.
[0244] In some embodiments, device 100 provides a multitasking view that displays applications running in the background of device 100. A user of device 100 can switch between different applications via the multitasking view. Fig. 10AAn exemplary embodiment of a multitasking view 1000 is shown. The multitasking view 1000 is activated via a gesture on the touch screen and / or multiple sequential presses of the menu button 204. Fig. 10A As shown in , multitasking view 1000 includes a graphical representation 1001 of one or more applications executing in the background of device 100 and an icon area 1003 including icons representing the applications. Fig. 10A In FIG. 1 , each graphical representation of an application is displayed in a portion of the touch screen 112. For example, Fig. 10A Graphical representations 1001 of a calendar application and an email application are included, with their respective icons displayed below the graphical representations. The calendar application is displayed in a first portion of the touch screen 112, and the email application is displayed in a second portion of the touch screen 112. The device 100 displays one of the applications in full screen mode in response to a user selection of the graphical representations 1001 of the applications or the icons representing the applications.
[0245] In some embodiments, the multitasking view 1000 can be displayed in an ergonomic mode. Upon detecting an input for displaying the ergonomic mode of the multitasking view 1000, the device 100 shifts the multitasking view 1000 downward toward the bottom area of the touch screen 112. During the ergonomic mode of the multitasking view 1000, the device 100 displays an animation of the graphical representation 1001 of the application shifting downward, and as the graphical representation 1001 of the application shifts downward, the icon area 1003 is moved away from the touch screen 112. In some embodiments, the multitasking view 1000 includes a plurality of contact icons 1005 corresponding to recent contacts or favorite contacts displayed above the representation 1001 of the application, and the ergonomic mode enables the user to reach the contact icons without shifting the user's grip (e.g., the contact icons are moved into the reachable area).
[0246] Fig. 10B An exemplary embodiment of an ergonomic mode of the multitasking view 1000' is shown. Fig. 10B As shown, multitasking view 1000' is shown in the bottom area of touch screen 112 and includes only a graphical representation of application 1001. Fig. 10B , icon area 1003 is no longer displayed on touch screen 112. In addition, background image 540 of home screen view 700 is displayed in an area above multitasking view 1000'. When in ergonomic mode, graphical representations 1001 of applications are now more easily accessible to the user's fingers without the user having to reposition device 100 in the user's hand, such as Fig. 10B shown.
[0247] In some embodiments, the view of the ergonomic mode of the application displayed on the device 100 depends on the size of the touch screen 112. Therefore, on devices of different sizes, the ergonomic mode of the application can be visually different. For example, Fig. 10C and Fig. 10D The ergonomic mode of the multitasking view when displayed on devices of different sizes according to some embodiments is shown. Specifically, Fig. 10C The device shown is 100% Fig. 10D The device 100 shown in FIG. 1 is larger.
[0248] The reachability area for each device is given by Fig. 10C The dashed line 1007 and Fig. 10D According to some embodiments, the dashed lines 1007 and 1009 are respectively based on holding in one hand Fig. 10C equipment and Fig. 10D The device is the physiological range of motion of the user's thumb. Fig. 10C and Fig. 10D The reachable areas of the devices represented in are based on the predefined reachability metrics for each device as described above. Fig. 10C The touch screen 112 of the device 100 is larger, and during one-handed operation Fig. 10C The reachable area of the device is less than Fig. 10D The reachable area of smaller devices during one-handed operation. Fig. 10C The accessible area of the touch screen 112 is small. Fig. 10D Compared to the embodiment shown in , the device displays the multitasking view 1000 ″ in an ergonomic mode further down in the bottom area of the touch screen 112 to allow the user to comfortably interact with the ergonomic mode of the multitasking view 1000 ″ using one-handed operation.
[0249] exist Fig. 10D In the example, the reachable area defined by the dashed line 1009 is greater than Fig. 10C The reachable area in Fig. 10C Compared to the ergonomic mode of the multitasking view 1000″ shown in FIG. 1 , the device displays the ergonomic mode of the multitasking view 1000″ higher in the bottom area of the touch screen 112. Therefore, due to Fig. 10D Because the accessible area of the touch screen 112 is larger, the device does not need to offset the ergonomic mode of the multitasking view further downward in the touch screen 112, while still allowing the user to more comfortably interact with the ergonomic mode of the multitasking view 1000"' using one-handed operation.
[0250] As previously mentioned, embodiments herein also include an ergonomic mode of a keyboard displayed on a display screen of the device 100. The ergonomic mode of the keyboard improves one-handed operation of the device 100. Figures 11A-11H Used to describe the ergonomics of a keyboard. Specifically, Fig.11A An exemplary keyboard for an email application and the keyboard in full screen mode when the device is oriented in portrait mode 901 are shown. While the description herein is described with respect to using a keyboard for an email application, embodiments herein may be applied to any application that utilizes a keyboard to input information into the device 100. In some embodiments, the full screen mode of the application includes the keyboard superimposed on the application.
[0251] like Fig.11A As shown in , the content area 1100 includes various information fields for an email. For example, the information fields include a "To" field in which the recipients of the email are entered, a Cc / Bcc field indicating other recipients of the email, a "From" field indicating the sender of the email, and a "Subject" field indicating the subject of the email.
[0252] The keyboard 901 is optionally displayed below the content area 1100 of the email application. When the application is opened, the device 100 automatically displays the keyboard 901 together with the content area 1100 of the application, or the device 100 displays the keyboard upon user request. When the keyboard 901 is displayed in full screen mode, the keyboard 901 includes a set of primary keys 1103, which are substantially centered along the width of the touch screen 112 (e.g., the first or initial position). The device 100 displays the primary keys by default during full screen mode. In some embodiments, the primary keys 1103 include the letters A to Z, a space, and a delete key.
[0253] During one-handed operation of the device 100, if the user does not switch from one-handed to two-handed operation of the device 100 or reposition the device 100 in the user's hand, it may be difficult for the user to reach certain primary keys. For example, if the user's right hand is being used to operate the device 100, the user may have difficulty reaching keys near the left edge of the touch screen 112, such as the "Q" or "W" keys. In another embodiment, if the user's left hand is being used to operate the device, the user may have difficulty reaching keys near the right edge of the touch screen 112, such as the "P" or "L" keys. Therefore, during the full-screen mode of the keyboard, it includes keys outside the reachable area of the display. In other words, the keys are displayed at a position greater than a predefined reachable metric. The predefined reachable metric can be the same as the predefined reachable metric described above, or it can be a different reachable metric. The predefined reachability metric for the keyboard describes the distance at which one-handed typing is comfortable and is based on the width of the touch screen 112. Rather than the user shifting the device 100 in the user's hand or switching to two-handed operation of the device 100, the user can activate the ergonomic mode of the keyboard 901. During the ergonomic mode, the keys of the keyboard 901 are offset to a second position so that the keys are within a reachable area of the touch screen 112, as further described below.
[0254] In some embodiments, the user provides an edge swipe gesture on keyboard 901 to activate an ergonomic mode of keyboard 901. Fig. 11B An edge swipe gesture using the user's left hand to activate the ergonomic mode of keyboard 901 is shown. To perform an edge swipe operation, the user may need to reposition the user's hand on device 100 to complete the gesture. This prevents the user from inadvertently requesting that keyboard 901 be displayed in ergonomic mode.
[0255] The device 100 determines input of an edge swipe gesture by detecting whether the gesture is over a portion of an application or over a portion of the keyboard 901. Based on determining that the edge swipe gesture is over the application, the device 100 performs an operation related to the content of the application, such as by navigating or scrolling the application, navigating back or forward (e.g., between web pages in a web browser history), or any other operation defined by the application as triggered in response to a swipe gesture initiated from the edge of the touch screen 112. The operation is optionally performed without adjusting the appearance of the keyboard. If the device 100 determines that the swipe gesture is over the keyboard 901, the device 100 displays an ergonomic mode of the keyboard 901.
[0256] To detect a swipe gesture over the keyboard, the device 100 detects contact 1105 at a first location over a portion of the keyboard 901 adjacent to the right vertical edge 525 of the touch screen 112. In some embodiments, the device 100 detects that the swipe gesture moves from the edge of the touch screen 112 onto the keyboard. The device 100 then detects continued movement of the contact 1105 in a direction toward a second location that is closer to the left vertical edge 529 of the touch screen 112 than the initial contact with the touch screen 112. The continued movement of the contact 1105 includes a lateral component of the movement without the contact 1105 breaking contact with the touch screen 112. Finally, the device 100 detects completion of the edge swipe gesture in response to detecting the finger lift at the second location, and the device 100 displays an ergonomic mode of the keyboard 901, as will be further described below.
[0257] During an edge swipe gesture, the device 100 determines the direction of the gesture. That is, the device 100 determines whether the lateral component of the edge swipe gesture is to the left or to the right. The direction of the edge swipe gesture indicates the direction in which the device 100 resizes and offsets the primary keys 1103 of the keyboard in ergonomic mode. Fig. 11B A left-direction edge swipe gesture is shown, indicated by the arrow direction of contact 1105 in the left direction. In some embodiments, an edge-to-edge swipe gesture, rather than an edge swipe gesture, is used to activate the ergonomic mode. The device 100 determines input of an edge-to-edge swipe gesture by detecting a contact on the keyboard 901 at a first position adjacent to the right vertical edge 525 of the touch screen 112 and continuous movement of the contact in a direction toward a second position adjacent to the left vertical edge 529 of the touch screen 112.
[0258] Fig. 11C An exemplary user interface of the ergonomic mode of the keyboard 901 in response to a left-directed edge swipe gesture is shown. Fig. 11C As shown in , according to the device 100 determining that the edge swipe gesture is to the left, the device 100 resets the size of the main key 1103' and shifts the key from its initial position to a second position in the horizontal direction. In some embodiments, during the resizing and shifting of the keyboard 901, the application does not move (for example, the application does not move in response to the edge swipe gesture). In ergonomic mode, the device 100 resets the size of the main key 1103' in the horizontal direction while maintaining the same height as when displayed in full screen mode. In some embodiments, the device 100 resets the size of the keyboard in the horizontal direction while maintaining the same height as when displayed in full screen mode. By shifting the main key 1103' to the left, the keys that the user cannot reach with his left hand during the one-handed operation of the device 100 are now more accessible to the user's left hand.
[0259] In addition, because the primary keys 1103' are resized and offset, a portion of the portion of the touch screen 112 that previously displayed the primary keys 1103' is no longer occupied by the portion of the primary keys 1103. During the ergonomic mode of the keyboard 901, the device 100 displays a set of auxiliary keys 1107 in the portion of the touch screen 112 that previously displayed the primary keys 1103'. The set of auxiliary keys 1107 supplements the primary keys 1103' displayed in the keyboard 901 and prevents space on the touch screen 112 from being wasted. Fig. 11C , the auxiliary keys 1107 are displayed to the right of the primary keys 1103' in a single column. In some embodiments, the device 100 displays the auxiliary keys 1107 in a manner that is visually distinguishable from the primary keys 1103'. For example, the device 100 displays the auxiliary keys 1107 in at least one of a color, brightness, hue, contrast, saturation, line width, etc. that is different from the primary keys 1103'. In some embodiments, the auxiliary keys 1107 are visually distinguished from some of the primary keys 1103' and have a similar visual appearance to one or more secondary keys in the primary keys 1103', such as a delete key and a keyboard switch key.
[0260] Fig. 11C The auxiliary keys 1107 shown by default in the figure include a cut key for cutting text, a copy key for copying text, a paste key for pasting text, and an undo key for undoing input on the device 100. Other auxiliary keys 1107, such as punctuation keys or foreign language keys, may be shown by default. Fig. 11C As shown in , the function of each key is visually represented by a graphic image on the key. For example, the Cut key includes an image of a pair of scissors, the Copy key includes an image of an object with a copy of the object behind it, the Paste key includes an image of a paste bottle adding a paste to the document, and the Undo key has an image of an arrow pointing backward. Alternatively, one or more of the auxiliary keys optionally include text of the function instead of a graphic image representing the function of the key.
[0261] In some embodiments, device 100 switches from the ergonomic mode displaying keyboard 901 back to the full screen mode of keyboard 901 in response to receiving a gesture on keyboard 901. The gesture is optionally a swipe gesture in the opposite direction of the gesture used to activate the ergonomic mode of keyboard 901. For example, a swipe gesture to the left is used to activate Fig. 11B Ergonomic mode for the middle keyboard 901. Fig. 11C, a rightward swipe gesture on keyboard 901 represented by contact 1105 is detected by device 100 and causes device 100 to switch from the ergonomic mode of keyboard 901 back to the full screen mode of keyboard 901. Thus, a second gesture for deactivating the ergonomic mode of keyboard 901 is detected starting from a second edge of touch screen 112 that is different from (e.g., relative to) the first edge of touch screen 112, and a gesture for activating the ergonomic mode is received from the first edge of touch screen 112.
[0262] In some embodiments, the auxiliary buttons 1103 can be scrolled to display additional auxiliary buttons. Fig. 11C , the last two default auxiliary keys are the paste key and the undo key. In order to scroll in the auxiliary keys 1107, the user enters a gesture on the auxiliary keys 1107. The gesture is optionally an up swipe or a down swipe gesture. Fig.11D A vertical swipe gesture for scrolling through the auxiliary keys is shown on auxiliary keys 1107. The vertical swipe gesture includes the device 100 detecting contact 1109 at the portion of the touch screen 112 where auxiliary keys 1107 are displayed and vertical movement of contact 1109 in an upward or downward direction. Fig.11D A vertical swipe gesture in an upward direction is shown. When the device 100 detects vertical movement of the contact 1109, the device 100 removes one or more of the default auxiliary keys and replaces the removed keys with one or more additional auxiliary keys 1107. For example, the device 100 optionally displays an animation of the auxiliary keys 1107 sliding upward to expose the additional auxiliary keys and remove some of the default auxiliary keys. In some embodiments, the device 100 optionally displays an indication that there are additional auxiliary keys to be displayed. The number of additional auxiliary buttons 1107 displayed by the device 100 is based on the size of the vertical component of the vertical swipe gesture. Fig.11D , the additional auxiliary keys 1107 displayed in response to the swipe gesture include a "Kana" key. The Kana key is associated with a Kana foreign keyboard displayed in response to the device 100 detecting a selection of the Kana key. Specifically, the device 100 replaces the primary keys 1103 with the Kana key in response to the selection of the Kana key, which is a Japanese syllable symbol.
[0263] In some embodiments, auxiliary key 1107 includes multiple foreign language keyboard keys by default. Each foreign language keyboard key is associated with displaying the corresponding foreign language keyboard. The foreign language keyboard key is selected based on the currently displayed keyboard. For example, when the current keyboard is a Chinese stroke keyboard, auxiliary keys for Chinese handwriting and pinyin keyboards are displayed, or when a Japanese romaji keyboard is the current keyboard, a Japanese kana keyboard and an emoticon keyboard are displayed.
[0264] Fig.11EAnother embodiment of auxiliary keys 1107 is shown. Rather than displaying a single column of auxiliary keys 1107 in response to a request to display the ergonomic mode of keyboard 901, device 100 displays auxiliary keys 1107 in multiple columns to the right of primary keys 1103, such as Fig.11E As shown. Fig.11E The auxiliary keys are arranged in two columns to allow Fig. 11C The illustrated embodiment displays more modifier keys than the default ones.
[0265] Fig.11F An edge swipe gesture is shown for activating the ergonomic mode of the keyboard 901 using the user's right hand. The device 100 determines input of the edge swipe gesture by detecting contact 1105 on the keyboard 901 at a first location adjacent to the left vertical edge 529 of the touch screen 112. The device 100 then detects continued movement of the contact 1105 in a direction toward a second location that is closer to the vertical edge 525 of the touch screen 112. The continued movement of the contact 1105 includes a lateral component of the movement without the contact 1105 breaking contact with the touch screen 112. Finally, the device 100 detects completion of the edge swipe gesture in response to detecting the finger lift-up at the second location, and the device 100 displays the ergonomic mode of the keyboard 901, as shown. Fig.11G shown.
[0266] like Fig.11G As shown in , based on the device 100 determining that the edge swipe gesture is to the right, the device 100 resizes the primary button 1103' and shifts the button in the right direction without shifting the application's content area 1100. By shifting the primary button to the right, the button that was out of reach of the user with their right hand during one-handed operation of the device 100 is now more accessible to the user's right hand.
[0267] Additionally, because primary key 1103' is resized and shifted to the right, a portion of the portion of touch screen 112 that previously displayed primary key 1103' is no longer occupied by that portion of primary key 1103. In some embodiments, device 100 displays a set of secondary keys 1107 in the portion of touch screen 112 that previously displayed primary key 1103, such as to the left of primary key 1103' in a single column, e.g. Fig.11G Device 100 also displays auxiliary keys 1107 in a manner that is visually distinguishable from primary keys 1103', such as in a different color. In some embodiments, device 100 displays the same auxiliary keys 1107 regardless of the direction of the edge swipe gesture. For example, Fig.11G The auxiliary key 1107 shown in FIG. Fig. 11CAlternatively, different auxiliary keys 1107 are displayed depending on the direction of the edge swipe gesture. For example, if the keyboard 901 is shifted to the right, the auxiliary keys 1107 include a cut button, a copy button, and a paste button, and if the keyboard 901 is shifted to the left, the auxiliary keys 1107 include a bold key, an underline key, and an italic key.
[0268] Fig.11H Another embodiment of auxiliary keys 1007 is shown. Rather than displaying a single column of auxiliary keys 1107 in response to a request to display the keyboard in ergonomic mode, device 100 displays auxiliary keys in multiple columns, such as Fig.11H As shown. Fig.11H In the example, the auxiliary keys are arranged in two columns to the left of the main keys 1103 to allow Fig.11G The illustrated embodiment shows more auxiliary keys than before.
[0269] In some embodiments, the device 100 automatically switches back to full screen mode from displaying the keyboard 901 in ergonomic mode based on time. The device 100 determines the amount of time since the last time the user input of the keyboard 901 was received when the keyboard 901 was displayed in ergonomic mode. In order to determine this amount of time, the device 100 optionally executes a timer, which counts the amount of time since the input of the keyboard 901 was received. If the device 100 determines that the amount of time exceeds a threshold, the device 100 automatically switches back to full screen mode from displaying the keyboard 901 in ergonomic mode. In order to switch back to the full screen mode of the keyboard 901, the device 100 offsets the main keys 1103 back to the center position and resets the size of the main keys 1103 back to its initial size before the main keys 1103 are reset in ergonomic mode and offset. However, if the user input is received before this amount of time exceeds the threshold, the device 100 resets the timer.
[0270] In some embodiments, the ergonomic mode of the keyboard 901 persists across all applications of the device 100. When the keyboard 901 is displayed in ergonomic mode in a first application, the device 100 receives an input for closing the first application and closes the first application according to the input. The device 100 receives an input for opening a second application including the keyboard 901, and automatically displays the ergonomic mode of the keyboard 901 in the second application instead of the full screen mode of the keyboard 901 in response to receiving an input for opening the second application. Therefore, before closing the first application, the size of the main keys 1103 is automatically reset and offset as previously displayed. Therefore, the device 100 automatically displays the ergonomic mode of the keyboard 901 in the second application because the ergonomic mode of the keyboard 901 has been activated in the first application before closing the first application. Alternatively, the ergonomic mode of the keyboard is not automatically displayed in the second application. On the contrary, when displaying the second application, the device 100 receives a request for displaying the keyboard 901, such as detecting a selection of a text input area in the second application. In response to receiving the request to display a keyboard, device 100 displays the keyboard with resized and offset primary keys 1103. Primary keys 1103 are optionally displayed based on a default size or based on a custom size resize and offset specified by a user.
[0271] Alternatively, if the application displayed with the keyboard 901 is closed when the keyboard 901 is displayed in the ergonomic mode, the device 100 automatically deactivates the ergonomic mode of the keyboard 901. For example, when the keyboard 901 is displayed in the ergonomic mode, the device 100 receives an input for closing the application displayed with the keyboard 901 and closes the application according to the input. The device 100 receives an input for opening an application and automatically displays the application and the keyboard in full screen mode in response to the input. Alternatively, the device 100 receives an input for opening a second application and automatically displays the second application and the keyboard in full screen mode in response to the input. Therefore, when the application is closed, the ergonomic mode of the keyboard 901 is deactivated.
[0272] In some embodiments, the ergonomic mode of the keyboard 901 persists even if the application displayed with the ergonomic mode of the keyboard 901 is closed. For example, when the ergonomic mode of the keyboard 901 is displayed with the first application, the device 100 receives an input for closing the first application and closes the first application according to the input. The device 100 receives an input for opening a second application and displays the second application and the keyboard 901 in full screen mode, wherein the primary keys 903 are displayed in a central position along the width of the touch screen 112. If the device detects an input for closing the second application, the device 100 closes the second application. If the device 100 detects an input for opening the first application, the device 100 automatically displays the keyboard 100 in the first application in ergonomic mode in response to the input. Therefore, in some embodiments, the ergonomic mode of the keyboard 901 persists in the first application even if the first application is closed.
[0273] In some embodiments, the width of a portion of the touch screen 112 displaying the auxiliary keys 1107 and the width of a portion of the touch screen 112 displaying the primary keys 1103 during the ergonomic mode of the keyboard 901 are resizable. That is, the keyboard 901 is resizable. Therefore, the user can resize the width of the auxiliary keys 1107 and the primary keys 1103 displayed in the keyboard 901. The device 100 detects a gesture provided by the user indicating a user request to resize, for example, to change the width of the primary keys 1103 and the auxiliary keys 1107. Fig. 12A A resize gesture is shown for increasing the width of the secondary key 1107 and decreasing the width of the primary key 1103 .
[0274] The resize gesture includes: the user initially makes two fingers contact on the touch screen 112 at the same time, wherein the first finger contacts a portion of the touch screen 112 displaying the primary button 1103, and the second finger contacts a portion of the touch screen 112 displaying the auxiliary button 1107, which is also called a two-finger touch. Fig. 12A As shown, contact 1201 on primary key 1103 indicates that a first finger of a user has contacted touch screen 112, and contact 1203 on auxiliary key 1107 indicates that a second finger of a user has contacted touch screen 112. Specifically, contact 1201 is to the left of dividing line 1205, and contact 1203 is to the right of dividing line 1205. Dividing line 1205 indicates the boundary between primary key 1103 and auxiliary key 1107 displayed in the ergonomic mode of keyboard 801.
[0275] exist Fig. 12AIn the illustrated embodiment, the resize gesture indicates a request to increase the width of a portion of the touch screen 112 that displays the auxiliary key 1107 and decrease the width of a portion of the touch screen 112 that displays the primary key 1103, as indicated by the leftward gesture direction (e.g., a first direction). The resize gesture also includes the user continuously moving two fingers that are in contact with the touch screen 112 to the left. As the user's fingers move, the computing device 100 detects the lateral component of the movement of contact 1201 and contact 1203 to determine the direction and size of movement to resize the primary key 1103 and the auxiliary key 1107. As contact 1201 and contact 1203 both move to the left on the touch screen 112, the dividing line 1203 moves in accordance with the movement of contact 1201 and contact 1203, and in accordance with the size of the lateral component of the movement, the width of a portion of the touch screen 112 that displays the auxiliary key 1107' increases and the width of a portion of the touch screen 112 that displays the primary key 1103" decreases, as shown in FIG. Fig. 12B shown.
[0276] exist Fig. 12B , primary key 1103″ is resized in the horizontal direction to have a smaller width than before device 100 received the resize gesture, and primary key 1103″ is offset to the left. Similarly, auxiliary key 1107' is resized in the horizontal direction to have a larger width than before device 100 received the resize gesture, and auxiliary key 1107' is also offset to the left. Once the user lifts two fingers to no longer contact touch screen 112, the resize gesture is completed. In order to reduce the width of a portion of touch screen 112 displaying auxiliary key 1107 and increase the width of a portion of touch screen 112 displaying primary key 1103, the user provides a resize gesture to the device 100 to the right (e.g., a second direction) instead of the above-mentioned left gesture.
[0277] In some embodiments, when performing a gesture that causes the keyboard to shift and resize so as to be displayed in an ergonomic mode, the size of the keyboard can be adjusted based on the size of the lateral component of the movement of the contact of the resize gesture. Thus, the keyboard 901 is resized and shifted in response to detecting the first part of the resize gesture, and then the user can further reduce the size and shift the keyboard in the same direction by continuing to move in the same direction, or can increase the size of the keyboard and shift the keyboard in another direction by beginning to move the contact back in the opposite direction.
[0278] In some embodiments, even if the application is closed, the custom width of the keyboard displayed in ergonomic mode also continues in the application. When the main key 1103 and auxiliary key 1107 are displayed with the reset width, the device 100 optionally closes the application. If the application is reopened, the keyboard is displayed in full screen mode at the first position. In response to detecting the edge swipe gesture for activating ergonomic mode on the keyboard, the device 100 determines the reset width of a group of main keys before closing the application, and resets and offsets a group of main keys 1103 in the direction of the edge swipe gesture without offsetting the view of the application. Therefore, in some embodiments, when the custom size and position of the offset key are specified by the user, the subsequent request to offset the keyboard and reset the size will be offset and reset the size of the keyboard based on the custom size and position specified by the user when the user requests to offset and reset the keyboard size next, without the need for the user to specify the custom size and position again.
[0279] In some embodiments, the custom width of the keyboard displayed in ergonomic mode does not persist in the application when the application is closed. For example, the device 100 closes the application when the primary key 1103 and the auxiliary key 1107 are displayed in the custom width, and then reopens the application. The keyboard 901 is displayed in full screen mode at a first position. In response to detecting an edge swipe gesture for activating ergonomic mode on the keyboard, the device 100 resets the reset width of a group of primary keys and offsets the reset size of the primary keys to a second position in the direction of the gesture without offsetting the view of the first application. Therefore, in some embodiments, when the custom size and position of the offset key are specified by the user, the subsequent request to offset the keyboard and resize it will be offset and resize the keyboard based on the default size and position when the user next requests to offset and resize the keyboard, without reference to the custom size and position.
[0280] In some embodiments, the customized size and position for the offset key lasts for a predetermined period of time. In some embodiments, the customized size and position for the offset key lasts for one application, but is not applicable to other applications (e.g., other applications can use different customized sizes or default customized sizes).
[0281] Fig.13AAn exemplary view of an email application displayed in a landscape view of device 100 is shown. In some embodiments, device 100 automatically displays auxiliary keys based on detecting that device 100 is oriented in a landscape view, also referred to as landscape mode. That is, in some embodiments, the landscape mode of the keyboard includes a second portion of a second group of keys displayed on a second side of a first group of keys. Therefore, auxiliary keys that are optionally displayed in the portrait mode of the keyboard (e.g., in response to a user swipe gesture that causes the keyboard to shift and resize) are displayed in the landscape mode of the keyboard by default because the landscape mode of the keyboard has more room to display additional keys. Therefore, the device automatically displays an ergonomic mode of the keyboard including auxiliary keys in response to device 100 being oriented from a portrait view to a landscape view. Before being oriented to a landscape view, the full screen mode of the keyboard is displayed in the portrait mode of device 100.
[0282] like Fig.13A As shown, when the device 100 is oriented in the landscape position, the primary keys 1103 are displayed substantially centered along the width of the keyboard. The device 100 displays a first group of auxiliary keys 1301A on a first side (e.g., the left side) of the primary keys 1103, and displays a second group of auxiliary keys 1301B on a second side (e.g., the right side) of the primary keys 1103.
[0283] like Fig.13A As shown in FIG. 1 , a first group of auxiliary keys 1300A are displayed in a single column to the left of the primary keys, and a second group of auxiliary keys 1300B are displayed in a single column to the right of the primary keys. Fig. 13B As shown, a first group of auxiliary keys 1300A are displayed in multiple columns to the left of the primary keys, and a second group of auxiliary keys 1300B are displayed in multiple columns to the right of the primary keys.
[0284] In some embodiments, if a foreign language keyboard is activated in device 100 while in the keyboard's landscape view, device 100 automatically includes foreign language keys in auxiliary keys 1301. Fig. 13B As shown, the kana key is included in the first group of auxiliary keys 1301A. According to the device 100 detecting the selection of the kana key, the device 100 uses Fig. 13C The Kana keyboard shown replaces the English keyboard. Fig. 13C As shown, the kana keyboard includes primary kana keys 1300A and 1300B, an auxiliary key 1301, and suggested candidates 1303. In response to a user selection of one or more primary kana keys 1300A, the suggested candidates 1303 display one or more candidate characters. In some embodiments, the auxiliary key 1301 is positioned between the primary kana keys 1300A and the suggested candidates 1303.
[0285] In some embodiments, the number of auxiliary buttons displayed when the device 100 is oriented in landscape mode depends on the size of the device touch screen 112. That is, more auxiliary buttons are displayed on a device with a larger touch screen than on a device with a smaller touch screen. Fig.13D and Fig.13E An exemplary view of an email application with a keyboard displayed in landscape view on devices of different sizes according to some embodiments is shown. Specifically, Fig.13D The device 100 shown in FIG. Fig.13E The touch screen 112 of the device 100 shown in FIG. 1 is larger than the touch screen 112 .
[0286] exist Fig.13D In FIG. 1 , when the device 100 is in the landscape orientation, the device 100 automatically displays the first group of auxiliary keys 1300A in multiple columns to the left of the primary key 1103, and also displays the second group of auxiliary keys 1300B in multiple columns to the right of the primary key 1103. Fig.13D In the embodiment shown, a total of 15 auxiliary keys are shown. Fig.13E In FIG. 1 , when the device 100 is in the landscape orientation, the device 100 displays the first group of auxiliary keys 1300A in a single column to the left of the primary keys 1103 and displays the second group of auxiliary keys 1300B in a single column to the right of the primary keys. Fig.13E In the embodiment shown, a total of 8 auxiliary keys are shown. Fig.13E The touch screen 112 of the device 100 is shown to be relatively small in size and thus Fig.13D Compared to the device 100 shown having a larger touch screen 112 , the device 100 displays fewer auxiliary buttons.
[0287] In some embodiments, the device 100 automatically activates an ergonomic mode of the keyboard 901 to improve the accuracy of primary key selections. The device 100 monitors the accuracy of primary key selections by detecting multiple incorrect selections of one or more of the primary keys. In some embodiments, the device 100 determines a ratio of correct primary key selections to the total number of primary key selections. If the ratio is below a threshold, the device 100 determines whether the incorrect selection of primary keys is isolated to a particular portion of the keyboard 801. For example, Fig.14A The body 1405 of the email is shown, which includes several misspelled words due to incorrect selection of multiple primary keys by the user. The device 100 determines that the entered text "Wr", "ataying", "st", "ahould", and "comr" should be "We", "staying", "at", "should", and "come", respectively. The device 100 determines that the incorrectly selected primary keys are all located in the portion 1403 of the keyboard 901.
[0288] In some embodiments, device 100 then determines the vertical edge of keyboard 901 that is farthest in the horizontal direction from portion 1403 that includes the incorrectly selected primary key, and automatically offsets and resizes primary key 1103 in the direction of the vertical edge. Fig. 14B , device 100 determines that vertical edge 525 is farthest in the horizontal direction from the incorrectly selected primary key in portion 1403, and resizes and offsets primary key 1103 in a direction toward vertical edge 525. Thus, when the user's error is biased toward input that is too far in the first direction, the device offsets the key in the first direction, and shifting the key in the first direction thereby causes the primary keys in portion 1403 to now be closer to the user's hand and will improve the accuracy of primary key selection.
[0289] In some embodiments, rather than automatically resizing and offsetting the keyboard, device 100 displays the Fig. 14C Prompt 1407 is shown requesting the user's permission to shift and resize the keyboard in response to determining that the incorrectly selected primary key is isolated from a particular location on the keyboard. In response to device 100 receiving permission to shift and resize the keyboard (e.g., selecting "yes" from prompt 1407), device 100 resizes and shifts the keyboard as described above.
[0290] Fig.15 is an exemplary method flow chart of a method 1500 for displaying an ergonomic mode of an application. Note that in other embodiments, other methods may be performed except Fig.15 Different steps than those shown in .
[0291] The device 100 displays 1501 an application in full screen mode on the touch screen 112 of the device 100, such as Figure 5A . In full screen mode, the GUI of the application is displayed with a height that occupies the entire touch screen 112 of the device 100, except for any area for the status bar of the device and / or the control area of the device. In addition, the application is displayed with a width that corresponds to the width of the touch screen 112 when in full screen mode. During the full screen mode of the application, the application includes a toolbar (e.g., toolbar 51) displayed in the bottom area of the touch screen 112. The toolbar includes multiple user interface elements (e.g., world clock 505, alarm 507, stopwatch 509, and timer 511), each of which is associated with a function of the application.
[0292] When in full screen mode, the application includes user interface objects, such as edit UI element 521 and / or add UI element 519, that are located outside the reachable area of the touch screen display. That is, the user interface object is located at a position greater than a predefined reachable metric, which indicates that the user interface object cannot be reached by the user's finger unless the electronic device is repositioned in the user's hand. When the application is displayed in full screen mode, the device 100 detects 1503 an input for displaying the application in an ergonomic mode. The input for displaying the ergonomic mode of the application includes multiple sequential presses of the menu button 204 (e.g., such as Figure 5A ), a half-press of the menu button 204, a corner edge swipe gesture at the bottom corner of the touch screen 112 (e.g., as shown in Figure 5B and Figure 5C ) or a flick gesture on the menu button 204 (e.g., Figure 5D ).
[0293] In response to detecting the input, the device 100 displays 1505 the application in an ergonomic mode. In the ergonomic mode, the user interface object is within the reachable area of the touch screen display. That is, the user interface object is located at a position that is less than a predefined reachable metric, which indicates that the user's finger can reach the user interface object without repositioning the device 100 in the user's hand. During the ergonomic mode of the application, the device 100 displays the application in the bottom area of the touch screen 112, and the height of the application is less than the height of the application when displayed in full screen mode. By displaying the application in the bottom area of the touch screen 112, the user can more easily interact with any UI object of the application, which was previously inaccessible without switching to two-handed operation of the device or repositioning the electronic device in the user's hand.
[0294] In order to display the application in ergonomic mode, the device 100 optionally determines a view of the application displayed in full screen mode. The determined view includes a first portion that displays the content of the application and a second portion that displays the toolbar of the application. The device 100 shifts the view of the application in a vertical direction until the view is displayed in the bottom area of the touch screen 112 with a height that is less than the height of the application when displayed in full screen mode. The shifted view includes the first portion that displays the content of the application and does not include the second portion that displays the toolbar.
[0295] In some embodiments, the ergonomic mode of the application includes a toolbar. In order to display the application in the ergonomic mode with the toolbar, the device 100 determines a first view of the application displayed in full screen mode. The first view includes a first portion that displays the content of the application and a second portion that displays the toolbar of the application. The device 100 determines a second view of the application associated with the application in the second mode. The second view includes a first portion that displays the content of the application and is resized according to the second height, and a second portion that displays the toolbar of the application and is resized according to the second height.
[0296] In some embodiments, the device 100 switches from displaying an ergonomic mode to a full-screen mode of an application based on time. When displaying an application in ergonomic mode, the device 100 determines the amount of time since the last input to the application was received. Based on determining that the amount of time exceeds a threshold, the device 100 automatically switches from the ergonomic mode back to the full-screen mode.
[0297] In some embodiments, after entering ergonomic mode and before detecting an input while in ergonomic mode, the device 100 uses a first time threshold to determine whether to automatically switch back to full-screen mode. After detecting a second input, the device 100 uses a second time threshold different from the first time threshold to determine whether to automatically switch back to full-screen mode. That is, when displaying an application in ergonomic mode, the device 100 determines the amount of time since the last input to the application was received. Based on receiving an input to the application when the amount of time is less than the threshold, the device 100 reduces the threshold. Based on determining that the amount of time since the last input to the application was received exceeds the reduced threshold, the device 100 automatically switches back to full-screen mode from ergonomic mode. Based on receiving another input to the application when the amount of time since the last input to the application was received is less than the reduced threshold, the device 100 increases the threshold.
[0298] In some embodiments, when displaying applications in ergonomic mode, device 100 automatically displays a notification view in a top area of the touch screen that is above a bottom area where applications are displayed, such as Fig. 5F The notification view includes one or more notifications for the user. Alternatively, an image of the home screen view of the device 100 is automatically displayed in the top area of the touch screen 112 above the bottom area where the application is displayed, such as Figure 5K In some other embodiments, a blank area is automatically displayed in the top area of the touch screen 112 above the bottom area where the application is displayed, for example, Figure 5E If the device detects input in the blank area, the device switches from ergonomic mode back to full-screen mode, such as Figure 5A as shown in .
[0299] In some embodiments, the blank area includes a user interface element associated with displaying a notification view (e.g., handle UI element 539), such as Figure 5G As shown in . Device 100 detects a contact on a user interface element and a continuous movement of the contact that moves the user interface element in a direction of continuous motion having a vertical component. Based on detecting that the vertical component is less than a threshold distance, device 100 determines a magnitude of the vertical component of the continuous movement and displays a portion of a notification view in a top area of the touch screen above a bottom area that displays applications in ergonomic mode. The height of the portion of the notification view is proportional to the determined magnitude of the vertical component. Based on device 100 detecting that the vertical component is greater than a threshold distance, device 100 displays the entire notification view.
[0300] When the application is displayed in ergonomic mode, the device 100 detects 1507 an input to the application at a position corresponding to the application on the touch screen. In response to detecting the input, the device 100 determines whether the input is a navigation input or an object interaction input. The navigation input includes a request to scroll in the content of the application, scroll in a list, or scroll in different views of the application. On the contrary, according to some embodiments, the object interaction input is a user input that interacts with the user interface object of the application or a request to switch between different views of the application. In response to 1509 that the input is an object interaction input, the device 100 generates 1513 a response to the input and switches 1515 from the ergonomic mode to the full screen mode. On the contrary, in response to the input being a navigation input, the device 100 navigates within the application without switching back to the full screen mode.
[0301] In some embodiments, the ergonomic mode of the application can be locked. The device 100 optionally receives a request to prevent switching back from the ergonomic mode to the first mode and locks the application in the ergonomic mode. When the application is locked in the second mode, the device 100 receives an input for revoking the application and revokes the application according to the input for revoking the application. The device 100 receives an input for opening the application and displays the application in the ergonomic mode.
[0302] In some embodiments, once the application is closed, the ergonomic mode of the application is revoked. When the application is displayed in the ergonomic mode, the device 100 receives an input for closing the application, and the device 100 closes the application according to the input. The device 100 receives an input for reopening the application and automatically displays the application in full screen mode according to the input for reopening the application.
[0303] In some embodiments, the ergonomic mode of the application is maintained even if the application is closed. When the application is displayed in the ergonomic mode, the device 100 receives an input for closing the application and closes the application according to the input. After closing the application, the device 100 receives an input for opening the application. In response to receiving the input for opening the application, the device 100 displays the application in the ergonomic mode according to the input for opening the application.
[0304] While displaying the application in full screen mode or ergonomic mode, device 100 detects a single press of the device's home button 204, indicating a request to display a home screen view. In response to the detection, device 100 switches from displaying the application in full screen mode or ergonomic mode to displaying the home screen view of device 100 in full screen mode, e.g., Fig. 7A The Home screen view consists of a row of icons and favorite icons below the row of icons.
[0305] In some embodiments, the home screen view can be displayed in an ergonomic mode. While displaying the home screen view, device 100 detects an input for displaying the home screen view in an ergonomic mode. Based on the input for displaying the home screen view in an ergonomic mode, device 100 displays the home screen view in an ergonomic mode, for example Figure 7B An ergonomic mode of the home screen view is displayed in a bottom area of the touch screen, the height of the ergonomic mode being less than the height of the full screen mode of the home screen view. The home screen view displayed in the ergonomic mode includes a portion of a row of icons included in the home screen view when displayed in the full screen mode and a plurality of favorite icons below the portion of the row.
[0306] While the home screen view is displayed in the ergonomic mode, the user can optionally navigate among the pages of the home screen view. While the home screen view is displayed in the ergonomic mode, device 100 receives an input for viewing a second view of the home screen view. Based on the input for viewing the second home screen view, device 100 switches from displaying the home screen view in the ergonomic mode to displaying the second home screen view in the ergonomic mode, e.g. Figure 7C The second home screen view includes a portion of the second plurality of rows of icons included in the second home screen view when the second home screen view is displayed in the full screen mode and a plurality of favorite icons located below the portion of the second plurality of rows.
[0307] Although the main screen view is displayed in ergonomic mode, device 100 receives a selection of an icon in the icon row, which indicates a request input for opening a second application. In some embodiments, device 100 automatically displays the second application in ergonomic mode based on the selection.
[0308] While displaying an application in full screen mode or ergonomic mode, device 100 detects multiple sequential presses of a button of device 100. In response to detecting the sequential presses, device 100 simultaneously displays the application in a first portion of touch screen 112 and another application in a second portion of touch screen 112.
[0309] It should be understood that Fig.15 The specific order in which the operations have been described in the description is merely exemplary and is not intended to indicate that the described order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize a variety of ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., method 1600) also apply in a similar manner to the details of the above processes with respect to Fig.15 Method 1500 is described. For example, the inputs, applications, gestures, and user interface objects described above with reference to method 1500 optionally have one or more of the characteristics of the inputs, applications, gestures, and user interface objects described herein with reference to other methods described herein (e.g., method 1600). For the sake of brevity, these details are not repeated herein.
[0310] References Fig.15 The operation is optionally performed by Figure 1A-1B and Fig.17 , and the components shown in . For example, detection operations 1503, 1507 and display operations 1501, 1505 are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 delivers event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186, and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as the selection of an object on a user interface, or the rotation of the device from one orientation to another orientation. When a corresponding predefined event or sub-event is detected, event identifier 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A-1B and Fig.17 How the components shown in FIG. 1 can implement other processes.
[0311] As mentioned above, reference Fig.15 The operation is optionally performed by Fig.17 According to some embodiments, Fig.17 1 shows a functional block diagram of a device 100 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of various described embodiments. It should be understood by those skilled in the art that Fig.17 The functional blocks described in the invention are optionally combined or divided into sub-blocks to implement the principles of various embodiments. Therefore, the description herein optionally supports any possible combination or separation or additional definition of the functional blocks described herein.
[0312] like Fig.17 As shown in , the device 100 includes a display unit 1701 configured to display an application, a touch-sensitive surface unit 1703 configured to receive user contact, and a processing unit 1705 coupled to the display unit 1701 and the touch-sensitive surface unit 1703. In some embodiments, the processing unit 1705 includes a display enabling unit 1707, a detection unit 1709, a determination unit 1711, a navigation unit 1713, a generation unit 1715, a switching unit 1717, an offset unit 1719, and a closing unit 1721.
[0313] Processing unit 1705 is configured to display an application in a first mode (e.g., using display enabling unit 1707), the application being displayed as having a first height when in the first mode. While displaying the application in the first mode, processing unit 1705 is further configured to detect (e.g., using detection unit 1709) a first input corresponding to a request to display the application in a second mode. In response to detecting the first input, processing unit 1705 is further configured to display the application in a second mode (e.g., using display enabling unit 1707), wherein in the second mode, the application is displayed in a bottom area of display unit 1701, and the application displayed in the second mode has a second height that is less than the first height. While displaying the application in the second mode, processing unit 1705 is further configured to detect a second input at a location on touch-sensitive surface unit 1703 corresponding to the application (e.g., using detection unit 1709). In response to detecting the second input, the processing unit 1705 is further configured to determine (e.g., using the determination unit 1711) whether the second input is a navigation input or an interactive input, and based on determining that the second input is a navigation input, the processing unit 1705 is configured to navigate within the application without switching back to the first mode (e.g., using the navigation unit 1713). Based on determining that the second input is an interactive input, the processing unit 1705 is further configured to generate a response to the interactive input (e.g., using the generation unit 1715) and switch from the second mode back to the first mode (e.g., using the switching unit 1717).
[0314] In some embodiments, in a first mode, the application includes a user interface object that is located outside a reachable area of display unit 1701 , and in a second mode, the user interface object is located within a reachable area of display unit 1701 .
[0315] In some embodiments, wherein in a first mode, the application includes a user interface object located at a position greater than a predefined reachable measure, indicating that the user interface object cannot be reached by a user's finger unless the device 100 is repositioned in the user's hand, and wherein in a second mode, the user interface object is located at a position less than a predefined reachable measure, indicating that the user interface object can be reached by a user's finger without repositioning the device 100 in the user's hand.
[0316] In some embodiments, the first mode includes a full screen mode in which the application occupies the entire display unit 1701, except for an area for a status bar of the electronic device, and the application is displayed when in full screen mode as having a first height corresponding to the height of the display unit 1701 and a first width corresponding to the width of the display unit 1701.
[0317] In some embodiments, where in the second mode, the application is displayed while in the second mode (e.g., utilizing display enabling unit 1707) as having a first width and a second height that is less than the height of display unit 1701.
[0318] In some embodiments, the navigation input includes a request to scroll a displayed screen of an application.
[0319] In some embodiments, the interactive input includes at least one of: a selection of a user interface object of the application or a request to switch between different views of the application.
[0320] In some embodiments, processing unit 1705 is configured to detect a first input by detecting (e.g., using detection unit 1709) multiple sequential inputs of a button of device 100, wherein a single press of the button is associated with displaying (e.g., using display enabling unit 1707) a first home screen view that includes a first plurality of user interface elements associated with a corresponding application.
[0321] In some embodiments, processing unit 1705 is configured to detect a first input by detecting (e.g., using detection unit 1709) a press of a button of device 100 that is less than a threshold amount, where a press of the button that exceeds the threshold amount is associated with performing a different function.
[0322] In some embodiments, the processing unit 1705 is configured to detect a first input by detecting (e.g., using detection unit 1709) contact at a first position on the touch-sensitive surface unit 1703 adjacent to a vertical edge of the display unit 1701, and detecting (e.g., using detection unit 1709) the contact moving continuously to a second position adjacent to a horizontal edge on the touch-sensitive surface unit 1703 that intersects the vertical edge without interrupting contact with the display unit 1701.
[0323] In some embodiments, processing unit 1705 is configured to detect first input by detecting (e.g., using detection unit 1709) a gesture on a button of device 100 corresponding to a request to display an application in a second mode, wherein a single press of the button is associated with performing a different function.
[0324] In some embodiments, when displaying an application in the first mode or the second mode, processing unit 1705 is also configured to detect (for example, using detection unit 1709) a single press of the device's home screen button, which indicates a request to display a first home screen view, and in response to the detection, processing unit 1705 is further configured to switch (for example, using switching unit 1717) from displaying the first mode or the second mode of the application to displaying a first home screen view of device 100 in the first mode, the first home screen view including first multiple rows of icons and multiple favorites icons located below the multiple rows of icons.
[0325] In some embodiments, when displaying the first home screen view, the processing unit 1705 is also configured to detect (for example, using the detection unit 1709) an input for displaying the first home screen view in the second mode, and based on the input for displaying the first home screen view in the second mode, the processing unit 1705 is also configured to display the first home screen view in the second mode (for example, using the display enabling unit 1707), wherein in the second mode, the first home screen view having a second height is displayed in the bottom area of the display unit 1701, and the first home screen view includes a portion of the first multiple rows of icons included in the first home screen view when displayed in the first mode and a plurality of favorite icons located below the portion of the multiple rows.
[0326] In some embodiments, processing unit 1705 is configured to detect the first input by detecting (e.g., using detection unit 1709) a gesture on a button of the device corresponding to a request to display an application in a second mode, wherein a single press of the button is associated with performing a different function.
[0327] In some embodiments, when displaying an application in the first mode or the second mode, the processing unit 1705 is also configured to detect (for example, using the detection unit 1709) a single press of a home screen button of the electronic device, which indicates a request to display a first home screen view, and in response to the detection, the processing unit 1705 is also configured to switch (for example, using the switching unit 1717) from the first mode or the second mode of displaying the application to a first home screen view of the device displayed in the first mode, the first home screen view including first multiple rows of icons and multiple favorites icons located below the multiple rows of icons.
[0328] In some embodiments, when displaying the first home screen view, the processing unit 1705 is also configured to detect (for example, using the detection unit 1709) an input for displaying the first home screen view in the second mode, and based on the input for displaying the first home screen view in the second mode, the processing unit 1705 is also configured to display the first home screen view in the second mode (for example, using the display enabling unit 1707), wherein in the second mode, the first home screen view having a second height is displayed in the bottom area of the display unit 1701, and the first home screen view includes a portion of the first multiple rows of icons included in the first home screen view when displayed in the first mode and a plurality of favorite icons located below the portion of the multiple rows.
[0329] In some embodiments, processing unit 1705 is configured to display the first home screen view in a second mode by offsetting the first home screen view in a vertical direction (e.g., using offset unit 1719) until the first home screen view is displayed at a second height in a bottom area of display unit 1701, the offset view including the portion of the first multiple rows of icons and the multiple favorites icons located below the portion of the first multiple rows.
[0330] In some embodiments, when the first home screen view is displayed in the second mode, the processing unit 1705 is also configured to receive input for viewing the second home screen view (for example, using the touch-sensitive surface unit 1703), and based on the input for viewing the second home screen view, the processing unit 1705 is configured to switch (for example, using the switching unit 1717) from displaying the first home screen view in the second mode to displaying the second home screen view in the second mode, the second home screen view including a portion of the second plurality of rows of icons included in the second home screen view when the second home screen view is displayed in the first mode and a plurality of favorite icons located below the portion of the second plurality of rows.
[0331] In some embodiments, when the application is displayed in the first mode or the second mode, the processing unit 1705 is also configured to detect multiple sequential presses of the device button (for example, using the detection unit 1709), and in response to detecting the multiple sequential presses, the processing unit 1705 is also configured to simultaneously display the application in the first part of the display unit 1701 and display another application in the second part of the display unit 1701 (for example, using the display enabling unit 1707).
[0332] In some embodiments, when in the second mode, processing unit 1705 is also configured to detect (e.g., using detection unit 1709) a repetition of the first input corresponding to a request to display the application in the second mode, and processing unit 1705 is configured to switch from the second mode back to the first mode (e.g., using switching unit 1717) based on detecting the repetition of the first input when in the second mode.
[0333] In some embodiments, the second input is a navigation input, and the processing unit 1705 is further configured to detect the second input by detecting (e.g., using the detection unit 1709) a contact at a first location on the display unit 1701, and detecting (e.g., using the detection unit 1709) continuous movement of the contact to a second location on the display unit 1701, the continuous movement having a lateral component of the movement without the contact breaking contact with the display unit 1701. The processing unit 1705 is further configured to scroll to a different view of the application based on detecting the continuous movement (e.g., using the navigation unit 1713).
[0334] In some embodiments, the processing unit 1705 is also configured (e.g., using the display enabling unit 1707) to automatically display a notification view in a top area above a bottom area of the display unit 1701 that displays an application, wherein the notification view includes one or more notifications for the user.
[0335] In some embodiments, the processing unit 1705 is also configured (e.g., using the display enabling unit 1707) to automatically display a blank area in the top area above the bottom area of the display unit 1701 that displays the application.
[0336] In some embodiments, processing unit 1705 is also configured to detect input in the blank area (e.g., using detection unit 1709), and in response to detecting input in the blank area, processing unit 1705 is also configured to switch from the second mode back to the first mode (e.g., using switching unit 1717).
[0337] In some embodiments, the blank area includes a user interface element associated with displaying a notification view that includes one or more notifications for the user, and the processing unit 105 is further configured to detect contact on the user interface element (e.g., using the detection unit 1709) and detect continuous movement of the contact (e.g., using the detection unit 1709), the continuous movement moving the user interface element in the direction of the continuous movement, the continuous movement having a vertical component. Based on detecting that the vertical component is less than a threshold distance, the processing unit 1705 is further configured to determine (e.g., using the detection unit 1709) the size of the vertical component of the continuous movement, and (e.g., using the display enabling unit 1707) display a portion of the notification view in a top area of the display unit 1701 located above a bottom area where the application is displayed in the second mode, wherein the height of the portion of the notification view is proportional to the determined size of the vertical component. Based on detecting that the vertical component is greater than the threshold distance, the processing unit 1705 is further configured to display the entire notification view (e.g., using the display enabling unit 1707).
[0338] In some embodiments, when the application is displayed in the second mode, the processing unit 1705 is also configured (e.g., using the display enabling unit 1707) to automatically display an image of the device's home screen view in the top area above the bottom area of the display unit 1701 that displays the application.
[0339] In some embodiments, the first mode of the application includes a toolbar for the application displayed in a bottom area of the touch screen display, the toolbar including multiple user interface elements, each associated with a functionality of the application.
[0340] In some embodiments, the processing unit 1705 is further configured to determine (e.g., using the determination unit 1711) a view of the application displayed in a first mode and a first height, the view including a first portion displaying the content of the application and a second portion displaying a toolbar of the application. The processing unit 1705 is further configured to shift the view of the application in a vertical direction until (e.g., using the shift unit 1719) the view is displayed at a second height in the bottom area of the display unit 1701, the shifted view including the first portion displaying the content of the application and not including the second portion displaying the toolbar.
[0341] In some embodiments, processing unit 1705 is further configured to determine (e.g., using determination unit 1711) a first view of an application associated with the application in the first mode, the first view comprising a first portion displaying content of the application and a second portion displaying a toolbar of the application. Processing unit 1705 is further configured to determine (e.g., using determination unit 1711) a second view of an application associated with the application in the second mode, the second view comprising a first portion displaying content of the application resized according to the second height and a second portion displaying a toolbar of the application resized according to the second height.
[0342] In some embodiments, when the application is displayed in the second mode, processing unit 1705 is also configured to determine (e.g., using determination unit 1711) an amount of time since the last input to the application was received, and based on determining that the amount of time exceeds a threshold, processing unit 1705 is also configured to automatically switch from the second mode back to the first mode (e.g., using switching unit 1717).
[0343] In some embodiments, after entering the second mode and before detecting an input while in the second mode, the device 100 uses a first time threshold to determine whether to automatically switch back to the first mode, and after detecting the second input, the processing unit 1705 is also configured to use a second time threshold that is different from the first time threshold (for example, using determination unit 1711) to determine whether to automatically switch back to the first mode.
[0344] In some embodiments, when displaying the application in the second mode, the processing unit 1705 is further configured to determine (e.g., using the determination unit 1711) the amount of time since the last input to the application was received, and based on receiving the input to the application when the amount of time is less than the threshold, the processing unit 1705 is further configured to reduce the threshold (e.g., using the threshold unit 1723). Based on determining that the amount of time since the last input to the application was received exceeds the reduced threshold, the processing unit 1705 is further configured to automatically switch back to the first mode from the second mode (e.g., using the switching unit 1717), and based on receiving another input to the application when the amount of time since the last input to the application was received is less than the reduced threshold, the processing unit 1705 is further configured to increase the threshold (e.g., using the threshold unit 1723).
[0345] In some embodiments, the processing unit 1705 is further configured to receive a request to prevent switching from the second mode back to the first mode (e.g., using the switching unit 1717), and is further configured to lock the application in the second mode (e.g., using the locking unit 1725). When the application is locked in the second mode, the processing unit 1705 is further configured to receive an input for revoking the application (e.g., using the detection unit 1709), and is configured to revoke the application (e.g., using the switching unit 1717) according to the input for revoking the application. The processing unit 1705 is further configured to receive an input for opening the application (e.g., using the detection unit 1709), and display the application in the second mode (e.g., using the display enabling unit 1707) according to the input for opening the application.
[0346] In some embodiments, processing unit 1705 is further configured to display a first home screen view of the device including a first plurality of rows of icons (e.g., using display enabling unit 1707), and receive a selection of an icon in the first plurality of rows of icons (e.g., using detection unit 1709), the selection indicating a request input for opening a second application. Processing unit 1705 is further configured to automatically display the second application in a second mode based on the selection (e.g., using display enabling unit 1707).
[0347] In some embodiments, when the application is displayed in the second mode, the processing unit 1705 is further configured to receive an input for closing the application (e.g., using the detection unit 1709), and close the application according to the input (e.g., using the closing unit 1721). The processing unit 1705 is also configured to receive an input for reopening the application (e.g., using the detection unit 1709), and automatically display the application in the first mode according to the input for reopening the application (e.g., using the display enabling unit 1707).
[0348] In some embodiments, when displaying the application in the second mode, the processing unit 1705 is further configured to receive input for closing the application (e.g., using the detection unit 1709), and close the application according to the input (e.g., using the closing unit 1721). After closing the application, the processing unit 1705 is further configured to receive input for opening the application (e.g., using the detection unit 1709), and in response to receiving the input for opening the application, the processing unit 1705 is further configured to display the application in the second mode according to the input for opening the application (e.g., using the display enabling unit 1707).
[0349] In some embodiments, the application is a calendar application, and displaying the application in a first mode and displaying the application in a second mode includes processing unit 1705 (e.g., utilizing display enabling unit 1707) displaying twelve months of the calendar year when in the first mode and displaying six months of the calendar year when in the second mode.
[0350] Fig.16 is an exemplary method flow chart of method 1600 for displaying an ergonomic mode of a keyboard. Note that in other embodiments, the method may be performed except Fig.16 Different steps than those shown in .
[0351] The device 100 displays 1601 a view of a first application on the touch screen 112. The view includes a full screen view of the application. While displaying the view of the first application, the device also displays 1603 a keyboard including a set of primary keys at a first position on the touch screen 112, such as Fig.11A As shown. The keyboard is optionally overlaid above the first application. In some embodiments, the set of primary keys is substantially centered along the width of the touch screen 112. At a first position, the keyboard includes keys outside the reachable area of the display. That is, a key from the set of primary keys is displayed at a first position greater than a predefined reachable measure, the first position indicating that the key cannot be reached by a user's finger unless the device 100 is repositioned in the user's hand.
[0352] The device 100 detects 1605 a gesture in a direction toward an edge of the touch screen 112. For example, the device 100 detects an edge swipe gesture on the keyboard indicating a request to display an ergonomic mode of the keyboard, such as Fig. 11B and Fig.11F as shown in .
[0353] Based on the gesture, the device 100 resizes the primary key in the direction of the gesture and shifts 1607 the primary key to a second location on the display without shifting the application, such as Fig. 11C and Fig.11G. When in the second position, the key is now within the reachable area of the display. That is, the key is now located at a position that is less than the predefined reachable metric, which indicates that after resizing and shifting a set of primary keys in the direction of the gesture to the second position, the user's finger can reach the key without repositioning the device in the user's hand.
[0354] In some embodiments, in response to a gesture for displaying an ergonomic mode of the keyboard, device 100 displays the auxiliary keys in a portion of the display that previously displayed a portion of the set of primary keys, such as Fig. 11C and Fig.11G Optionally, the auxiliary keys are displayed in a single column (e.g., Fig. 11C and Fig.11G ) or multiple columns (for example, Fig.11E and Fig.11H ) and displayed in a manner visually distinguishable from the set of primary keys. The secondary keys optionally include a bold key, an underline key, an italic key, an undo key, a foreign language key, or a punctuation key.
[0355] In some embodiments, the user can scroll through the auxiliary buttons to view additional auxiliary buttons, such as Fig.11D The device 100 detects an input on the auxiliary key for displaying an additional key. The device 100 removes the key from the auxiliary key and displays the additional key based on the detected input, and the additional key is displayed while continuing to display the group of primary keys.
[0356] The device 100 determines the direction in which to resize and offset the group of main keys based on the direction of the gesture for activating the ergonomic mode of the keyboard. The device 100 determines whether the direction of the gesture is along the first direction or the second direction. Based on determining that the direction of the gesture is along the first direction, the device resizes the group of main keys in the first direction and offsets them to the second position. Based on determining that the direction of the gesture is along the second direction, the device 100 resizes the group of main keys in the second direction and offsets them to the second position. In some embodiments, whether the main keys are offset in the first direction or the second direction, the displayed auxiliary keys are the same. In some embodiments, the device 100 detects a second gesture in the opposite direction to the detected gesture for activating the ergonomic mode of the keyboard. In response to detecting the second gesture, the device cancels the resizing of the first group of keys and displays the first group of keys at the first position instead of the second position without offsetting the view of the first application.
[0357] In some embodiments, the set of primary keys and the set of secondary keys are resizable. Device 100 detects contact at a first location on a second set of keys and the contact continues to move to a second location on the keyboard, such as Fig. 12A The continuous movement has a lateral component of the movement, and the contact does not break contact with the display. The device 100 determines the size of the lateral component and resets the width of the group of primary keys and the width of the group of auxiliary keys based on the determined size of the lateral component. Fig. 12B As shown in , the width of the group of primary keys is reduced to a smaller width, and the width of the group of auxiliary keys is increased to a larger width. On the contrary, according to the horizontal component in the second direction, the width of the group of primary keys is increased to a larger width, and the width of the group of auxiliary keys is reduced to a smaller width.
[0358] In some embodiments, the device 100 switches from displaying the customized widths of the set of primary keys and the set of auxiliary keys based on time. The device 100 determines the time that has passed since the width of the set of primary keys and the width of the set of auxiliary keys were reset. Based on determining that the time that has passed exceeds a threshold, the device 100 resets the width of the set of primary keys back to a default width and resets the width of the set of auxiliary keys back to a default width.
[0359] In some embodiments, the customized width of the group of main keys and auxiliary keys is persistent in the application. The device 100 closes the first application when the group of main keys is displayed with the reset width based on the size of the determined horizontal component. According to the input for opening the first application, the device displays a view of the first application and a keyboard including a first group of keys displayed at a first position on the display. In response to detecting a gesture in the direction toward the edge of the display, the device determines the reset width of the group of main keys before closing the first application, and offsets the first group of keys to a second position in the direction of the gesture without offsetting the view of the first application. In an alternative embodiment, the customized width of the group of main keys is not persistent. In response to detecting the gesture, the device resets the reset width of the group of main keys and offsets the group of reset-sized main keys to a second position in the direction of the gesture without offsetting the view of the first application in the portion of the display that previously displayed the first group of keys.
[0360] In some embodiments, the ergonomic mode of the keyboard persists across all applications. For example, the device 100 switches from displaying a view of a first application to displaying a view of a second application. The device then displays a keyboard including a first set of keys while displaying a view of the second application, wherein the set of primary keys is automatically resized and displayed at a second position on the display.
[0361] In some embodiments, the ergonomic mode of the keyboard is persistent in the first application, but not between other applications. The device 100 switches from a view displaying the first application to a view displaying the second application. When displaying the view of the second application, the device 100 displays a keyboard including the group of primary keys. The device 100 automatically displays the group of primary keys at a first position on the display without resizing and offsetting as previously displayed in the first application. The device 100 then switches from the view displaying the second application back to the view displaying the first application. The device 100 then displays the keyboard including the group of primary keys when displaying the view of the first application, wherein before closing the first application, the size of the group of primary keys is automatically reset as previously displayed and the group of primary keys is displayed at a second position on the display.
[0362] In some embodiments, the device 100 uses an accuracy heuristic to automatically display an ergonomic mode of the keyboard. The device 100 detects an incorrect selection of one or more keys in the set of primary keys and resizes and offsets the set of primary keys in a direction based at least in part on the directional bias of the incorrect selection, such as Fig.14A and Fig. 14B Device 100 optionally displays a prompt to the user, requesting permission to resize and offset the first set of keys to improve the accuracy of key selection, such as Fig. 14C In response to receiving the permission, device 100 resizes and offsets the set of primary keys in a direction based at least in part on the incorrectly selected directional bias.
[0363] In some embodiments, the device detects that it is in an orientation associated with landscape mode. In response to detecting that the device is in an orientation associated with landscape mode, device 100 displays the keyboard in landscape mode. The landscape mode of the keyboard includes the set of primary keys substantially centered on the display and a first portion of the set of secondary keys displayed on a first side of the first set of keys, such as Fig.13A as shown in .
[0364] It should be understood that Fig.16 The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., method 1500) also apply in a similar manner to the details of the above processes with respect to Fig.16The method 1600 described above. For example, the inputs, applications, gestures, and user interface objects described above with reference to the method 1600 optionally have one or more of the characteristics of the inputs, applications, gestures, and user interface objects described herein with reference to other methods described herein (e.g., method 1500). For the sake of brevity, these details are not repeated herein. Fig.16 The operation is optionally performed by Figure 1A-1B and Fig.18 . For example, the detection operation 1605 and the display operations 1601, 1603 are optionally implemented by the event classifier 170, the event identifier 180 and the event handler 190. The event monitor 171 in the event classifier 170 detects the contact on the touch-sensitive display 112, and the event distributor module 174 delivers the event information to the application 136-1. The corresponding event identifier 180 of the application 136-1 compares the event information with the corresponding event definition 186, and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as the selection of an object on the user interface, or the rotation of the device from one orientation to another orientation. When the corresponding predefined event or sub-event is detected, the event identifier 180 activates the event handler 190 associated with the detection of the event or sub-event. The event handler 190 optionally uses or calls the data updater 176 or the object updater 177 to update the application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A-1B and Fig.18 How the components shown in FIG. 1 can implement other processes.
[0365] As mentioned above, reference Fig.16 The operation is optionally performed by Fig.18 According to some embodiments, Fig.18 1 shows a functional block diagram of a device 100 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software to perform the principles of various described embodiments. It should be understood by those skilled in the art that Fig.18 The functional blocks described in the invention are optionally combined or divided into sub-blocks to implement the principles of various embodiments. Therefore, the description herein optionally supports any possible combination or separation or additional definition of the functional blocks described herein.
[0366] like Fig.18As shown in , the device 100 includes a display unit 1801 configured to display an application, a touch-sensitive surface unit 1803 configured to receive user contact, an orientation sensing unit 1823 configured to sense the orientation of the device 100, and a processing unit 1805 coupled to the display unit 1801 and the touch-sensitive surface unit 1803. In some embodiments, the processing unit 1805 includes a display enabling unit 1807, a detecting unit 1809, a resizing and offsetting unit 1811, a switching unit 1813, a determining unit 1815, a removing unit 1817, a scrolling unit 1819, and a closing unit 1821.
[0367] Processing unit 1805 is configured to display a view of a first application (e.g., using display enabling unit 1807), and while displaying the first application, display a keyboard including a first group of keys (e.g., using display enabling unit 1807), the first group of keys being displayed at a first position on display unit 1801. Processing unit 1805 is also configured to detect a gesture at display unit 1801 in a direction toward an edge of display unit 1801 (e.g., using detection unit 1809), and in response to the detection, processing unit 1805 is also configured to resize the first group of keys in the direction of the gesture and shift the first group of keys to a second position on display unit 1801 without shifting the view of the application (e.g., using resizing and shifting unit 1811).
[0368] In some embodiments, wherein in a first position, the keyboard includes keys located outside a reachable area of the display unit 1801 , and wherein in a second position, the keys are within a reachable area of the display unit 1801 .
[0369] In some embodiments, a key in a first group of keys on a keyboard is displayed at a first position that is larger than a predefined reachable measure, indicating that the key cannot be reached by a user's finger unless the device is repositioned in the user's hand, and wherein the key is located at a position that is smaller than the predefined reachable measure, indicating that the key can be reached by the user's finger without repositioning the device in the user's hand after the first group of keys is resized in the direction of a gesture and offset to a second position.
[0370] In some embodiments, processing unit 1805 is also configured to switch (e.g., using switching unit 1813) from a view displaying a first application to a view displaying a second application, and to display a keyboard including a first group of keys when displaying the view of the second application (e.g., using display enabling unit 1807), wherein the first group of keys are displayed at a first position on display unit 1801 without being resized and offset as previously displayed in the first application. Processing unit 1805 is also configured to switch (e.g., using switching unit 1813) from a view displaying the second application back to a view displaying the first application, and to display a keyboard including a first group of keys when displaying the first application (e.g., using display enabling unit 1807), wherein the first group of keys are automatically resized and displayed at a second position on display unit 1801, as previously displayed before closing the first application.
[0371] In some embodiments, the processing unit 1805 is also configured to switch from displaying a view of a first application to displaying a view of a second application (e.g., using the switching unit 1813), and to display a keyboard including a first group of keys when displaying the view of the second application (e.g., using the display enabling unit 1807), wherein the first group of keys are automatically resized and displayed at a second position on the display.
[0372] In some embodiments, when the first group of keys is displayed at the second position, the processing unit 1805 is also configured to determine (e.g., using the determination unit 1815) an amount of time since the last input from the first application was received, and based on determining that the elapsed time exceeds a threshold, the processing unit 1805 is also configured to offset the first group of keys back to the first position and resize them to the initial size of the first group of keys before resizing and offsetting them to the second position (e.g., using the resizing and offsetting unit 1811).
[0373] In some embodiments, processing unit 1805 is also configured to display a second set of keys in a portion of display unit 1801 that previously displayed a portion of the first set of keys in response to detecting the gesture.
[0374] In some embodiments, the processing unit 1805 is also configured to display a second group of keys arranged in a single column or multiple columns.
[0375] In some embodiments, the processing unit 1805 is also configured to detect input on the second group of keys for displaying additional keys (for example, using the detection unit 1809), remove keys from the second group of keys (for example, using the removal unit 1817), and display additional keys based on detecting the input (for example, using the display enabling unit 1807), and the additional keys are displayed while continuing to display the first group of keys.
[0376] In some embodiments, processing unit 1805 is also configured to detect a gesture on the view of the first application (e.g., using detection unit 1809), and in response to detecting a gesture on the view of the first application, processing unit 1805 is also configured to scroll in the view of the first application (e.g., using scrolling unit 1819).
[0377] In some embodiments, the processing unit 1805 is also configured (for example, using the detection unit 1809) to detect contact at a first position on the keyboard adjacent to a first vertical edge of the display unit 1801, and detect continuous movement of the contact to a second position on the keyboard, the continuous movement having a lateral component of the movement without the contact interrupting contact with the display unit 1801.
[0378] In some embodiments, the processing unit 1805 is further configured to determine (e.g., using the determination unit 1815) whether the direction of the gesture is in the first direction or the second direction, and based on determining that the direction of the gesture is in the first direction, the processing unit 1805 is further configured to (e.g., using the resizing and offsetting unit 1811) resize the first group of keys in the first direction and offset the first group of keys to the second position. Based on determining that the direction of the gesture is in the second direction, the processing unit 1805 is further configured to (e.g., using the resizing and offsetting unit 1811) resize the first group of keys in the second direction and offset the first group of keys to the second position.
[0379] In some embodiments, the processing unit 1805 is also configured to detect a second gesture in an opposite direction to the detected gesture (e.g., using the detection unit 1809), and in response to detecting the second gesture, the processing unit 1805 is also configured to undo the resizing of the first group of keys (e.g., using the resizing and offset unit 1811) and display the first group of keys at the first position rather than the second position without offsetting the view of the first application (e.g., using the display enabling unit 1807).
[0380] In some embodiments, the same second group of keys is displayed regardless of whether the keys are offset in the first direction or the second direction.
[0381] In some embodiments, the processing unit 1805 is also configured to display (e.g., using the display enabling unit 1807) a first group of keys substantially centered on the display unit 1801.
[0382] In some embodiments, the first group of keys deviates from the center of the display unit 1801 in the direction of the gesture.
[0383] In some embodiments, the processing unit 1805 is further configured to detect contact at a first position on the second group of keys (e.g., using the detection unit 1809), and detect that the contact moves continuously to a second position on the keyboard (e.g., using the detection unit 1809), the continuous movement having a lateral component of the movement without the contact breaking contact with the display unit 1801. The processing unit 1805 is further configured to determine the size of the lateral component (e.g., using the determination unit 1815), and reset the width of the first group of keys and the width of the second group of keys based on the determined size of the lateral component (e.g., using the resize and offset unit 1811).
[0384] In some embodiments, the width of the first group of keys is reduced to a smaller width and the width of the second group of keys is increased to a larger width in the first direction according to the horizontal component, and wherein the width of the first group of keys is increased to a larger width and the width of the second group of keys is reduced to a smaller width in the second direction according to the horizontal component.
[0385] In some embodiments, the processing unit 1805 is further configured to (e.g., using the closing unit 1821) close the first application when the first group of keys is displayed with the reset width based on the determined size of the horizontal component, and detect (e.g., using the detection unit 1809) an input for opening the first application. Based on the input for opening the first application, the processing unit 1805 is further configured to (e.g., using the display enabling unit 1807) display a view of the first application, and when displaying the first application, display (e.g., using the display enabling unit 1807) a keyboard including the first group of keys, which is displayed at a first position on the display unit 1801. The processing unit 1805 is further configured to (e.g., using the detection unit 1809) detect a gesture in a direction toward an edge of the display unit 1801 at the display unit 1801, and in response to the detection, the processing unit 1805 determines (e.g., using the determination unit 1815) the reset width of the first group of keys before the first application is closed. The processing unit 1805 also resizes the first group of keys in the direction of the gesture based on the determined resized width (e.g., using the resizing and offset unit 1811) and offsets the first group of keys to a second position without offsetting the view of the first application.
[0386] In some embodiments, processing unit 1805 is further configured to close the first application when the first group of keys is displayed with the reset width based on the determined size of the horizontal component (e.g., using closing unit 1821). Processing unit 1805 is further configured to detect input for opening the first application (e.g., using detection unit 1809), and based on the input for opening the first application, processing unit 1805 is configured to display a view of the first application (e.g., using display enabling unit 1807). Processing unit 1805 is further configured to display a keyboard including a first group of keys when displaying the first application, the first group of keys being displayed at a first position on display unit 1801 (e.g., using display enabling unit 1807), and to detect a gesture at display unit 1801 in a direction toward an edge of display unit 1801 (e.g., using detection unit 1809). In response to the detection, the processing unit 1805 is also configured to (e.g., using the resize and offset unit 1811) reset the resized width of the first group of keys and offset the first group of resized keys to a second position in the direction of the gesture in the portion of the display that previously displayed the first group of keys without offsetting the view of the first application.
[0387] In some embodiments, the processing unit 1805 is also configured to determine (e.g., using the determination unit 1815) the time that has passed since the first group of keys and the second group of keys were reset, and based on determining that the elapsed time exceeds a threshold, the processing unit 1805 is also configured to reset the width of the first group of keys back to the default width and reset the width of the second group of keys back to the default width (e.g., using the resize and offset unit 1811).
[0388] In some embodiments, after resizing and offsetting the first group of keys when displaying the first application, the processing unit 1805 is further configured to switch from the first application to the second application (e.g., using the switching unit 1813). When displaying the second application, the processing unit 1805 is further configured to receive a request to display a keyboard (e.g., using the detection unit 1809), and in response to receiving the request to display the keyboard, the processing unit 1805 is configured to display the keyboard with the resized and offset first group of keys (e.g., using the display enabling unit 1807).
[0389] In some embodiments, the second group of keys includes multiple foreign keyboard keys, each foreign keyboard key is associated with a corresponding foreign keyboard displayed.
[0390] In some embodiments, the processing unit 1805 is also configured (e.g., using the display enabling unit 1807) to display the second group of keys in a manner that is visually distinguishable from the first group of keys.
[0391] In some embodiments, while displaying a first keyboard having a first set of auxiliary keys associated with the first keyboard, processing unit 1805 is configured to receive a request to switch to a second keyboard (e.g., using detection unit 1809). In response to receiving the request to switch to the second keyboard, processing unit 1805 is further configured to replace display of the second keyboard with the first keyboard (e.g., using removal unit 1817), and replace display of the first set of auxiliary keys associated with the first keyboard with display of the second set of auxiliary keys associated with the second keyboard (e.g., using removal unit 1817), wherein the second set of auxiliary keys is different from the first set of auxiliary keys.
[0392] In some embodiments, the processing unit 1805 is also configured to detect (e.g., using the detection unit 1809) multiple incorrect selections of one or more keys in the first group of keys, and automatically resize and offset the first group of keys based at least in part on the directional bias of the multiple incorrect selections (e.g., using the resizing and offset unit 1811).
[0393] In some embodiments, the processing unit 1805 is further configured to detect (e.g., using the detection unit 1809) multiple incorrect selections of one or more keys in the first group of keys, and display a prompt to the user (e.g., using the display enabling unit 1807) requesting permission to resize and offset the first group of keys to improve key selection accuracy. In response to receiving permission, the processing unit 1805 is further configured to automatically resize and offset the first group of keys in a direction that is biased at least in part based on the direction of the multiple incorrect selections (e.g., using the resizing and offsetting unit 1811).
[0394] In some embodiments, processing unit 1805 is further configured to detect (e.g., using orientation sensing unit 1823) that device 100 is in an orientation associated with landscape mode, and in response to detecting that device 100 is in an orientation associated with landscape mode, processing unit 1805 is further configured to display a keyboard in landscape mode (e.g., using display enabling unit 1807), the landscape mode of the keyboard comprising a first group of keys substantially centered on display unit 1801 and a first portion of a second group of keys displayed on a first side of the first group of keys.
[0395] For the purpose of explanation, the foregoing description is described by reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the disclosed precise form. According to the above teachings, many modifications and variations are possible. The embodiments are selected and described in order to best illustrate the principles of the invention and its practical application, so that other persons skilled in the art can best use the invention and the various described embodiments with various modifications suitable for the specific purposes envisioned.
[0396] Any step, operation or process described herein may be performed or implemented using one or more hardware or software modules alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product including a computer-readable medium containing computer program code, which may be executed by a computer processor to perform any or all of the steps, operations or processes.
[0397] Embodiments of the present invention also relate to apparatus for performing the operations herein. The apparatus may be specially constructed for the desired purpose, and / or may include a general-purpose computing device selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored in a tangible computer-readable storage medium or any type of medium suitable for storing electronic instructions and coupled to a computer system bus. In addition, any computing system mentioned in the specification may include a single processor or may be an architecture that uses multiple processor designs to enhance computing power.
[0398] The embodiments of the present invention may also be directed to a computer data signal embodied in a carrier wave, wherein the computer data signal includes any embodiment of the computer program product or other data combination described herein. The computer data signal is a product presented in a tangible medium or carrier wave and modulated or encoded in a tangible carrier wave by other means, and transmitted according to any appropriate transmission method.
[0399] Finally, the language used in this specification has been selected in principle for readability and instructional purposes, and may not be so selected to depict or define the subject matter of the present invention. It is therefore intended that the scope of the present invention is not limited by this detailed description, but by any claims issued based on the application herein. Therefore, the disclosure of the embodiments of the present invention is intended to illustrate rather than limit the scope of the present invention, which is set forth in the claims below.
Claims
1. A method comprising: At electronic devices with displays and touch-sensitive surfaces: Displaying the application in a first mode, wherein the application is displayed to have a first height when in the first mode; detecting a first input while the application is displayed in the first mode; In response to detecting the first input: If it is determined that the first input corresponds to a request to display the application in a second mode: displaying the application in a second mode by offsetting display of the application, wherein in the second mode the application is displayed having a second height that is less than the first height; while the application is displayed in the second mode and subsequently detecting an end of the first input, detecting a second input on the touch-sensitive surface; In response to detecting the second input: if it is determined that the second input is a navigation input detected on the touch-sensitive surface that corresponds to a location on the display of the application, scrolling within the application without switching back to the first mode; as well as switching from the second mode back to the first mode if it is determined that the second input is an input at a location on the touch-sensitive surface that corresponds to an external location of the application on the display; detecting a repetition of the first input while the application is displayed in the second mode; as well as In response to detecting a repetition of the first input, switching from the second mode back to the first mode.
2. A method according to claim 1, wherein in the first mode, the application includes a graphical object located at the following position on the display: the position corresponds to a distance across the touch-sensitive surface that is greater than a predefined reachable measure, and the position indicates that the graphical object is not reachable by the user's fingers of the user's hand unless the electronic device is repositioned in the user's hand; and wherein in the second mode, the graphical object is located at the following position on the display: the position corresponds to a distance across the touch-sensitive surface that is less than the predefined reachable measure, and the position indicates that the graphical object is reachable by the user's fingers of the user's hand without the electronic device being repositioned in the user's hand.
3. The method of claim 1 , wherein the first mode comprises a full screen mode, wherein when in the full screen mode, the application occupies the entirety of the display excluding an area for a status bar for the electronic device, and the application is displayed having the first height and a first width, wherein the first height corresponds to a height of the display and the first width corresponds to a width of the display.
4. The method of claim 1, wherein detecting the first input comprises: detecting a contact on the touch-sensitive surface corresponding to a location on the display at a first location adjacent a vertical edge of the display; as well as Continued movement of the contact on the touch-sensitive surface corresponding to a second location on the display adjacent a horizontal edge of the display that intersects the vertical edge is detected without the contact breaking contact with the touch-sensitive surface.
5. The method of claim 1 , wherein detecting the first input comprises: A gesture is detected on a button of the electronic device corresponding to a request to display the application in the second mode, wherein a single press of the button is associated with performing a different function.
6. The method of claim 1, wherein displaying the application in the second mode comprises: A blank area is displayed in an area of the display adjacent to the application.
7. The method of claim 1, wherein displaying the application in the second mode comprises: displaying a notification area of the display adjacent to an area displaying the application, the notification area including a user interface element associated with displaying a notification view including one or more notifications for the user; detecting a contact on the touch-sensitive surface corresponding to the user interface element; detecting a continuous movement of the contact, the continuous movement of the contact moving the user interface element in a direction of the continuous movement, the continuous movement having a vertical component; If it is detected that the vertical component is less than the threshold distance: determining a magnitude of the vertical component of the continuous movement; as well as displaying a portion of the notification view in the notification area of the display adjacent to the area where the application is displayed in the second mode, wherein a height of the portion of the notification view is proportional to the determined size of the vertical component; as well as If it is detected that the vertical component is greater than the threshold distance, the notification view is displayed in its entirety.
8. The method according to any one of claims 1 to 7, wherein displaying the application in the second mode comprises: While the application is displayed in the second mode, an image of a home screen view of the electronic device is displayed in an area of the display adjacent to the application.
9. The method of any one of claims 1 to 7, wherein the first mode of the application comprises a toolbar of the application displayed in an area of the display, the toolbar comprising a plurality of user interface elements, each user interface element being associated with a function of the application, and wherein displaying the application in the second mode comprises: Determining a view of the application displayed in the first mode and having the first height, the view comprising a first portion displaying content of the application and a second portion displaying the toolbar of the application; as well as The view of the application is shifted in a vertical direction until the view is displayed to have the second height, the shifted view including the first portion displaying the content of the application and excluding the second portion displaying the toolbar.
10. The method of any one of claims 1 to 7, wherein the first mode of the application comprises a toolbar of the application displayed in an area of the display, the toolbar comprising a plurality of user interface elements, each user interface element being associated with a function of the application, and wherein displaying the application in the second mode comprises: determining a first view of the application, the first view of the application being associated with the application in the first mode, the first view comprising a first portion displaying content of the application and a second portion displaying the toolbar of the application; as well as Determine a second view of the application, the second view of the application being associated with the application in the second mode, the second view comprising the first portion displaying content of the application and resized according to the second height and the second portion displaying the toolbar of the application and resized according to the second height.
11. The method according to claim 1, further comprising: while the application is displayed in the second mode, determining an amount of time since input to the application was last received; as well as If it is determined that the amount of time exceeds a threshold, automatically switching from the second mode back to the first mode.
12. The method of claim 1, wherein after entering the second mode and before detecting the second input while in the second mode, the electronic device uses a first time threshold to determine whether to automatically switch back to the first mode, and the method further comprises: After detecting the second input, a second time threshold different from the first time threshold is used to determine whether to automatically switch back to the first mode.
13. The method according to claim 1, further comprising: receiving a request to prevent switching from the second mode back to the first mode; locking the application in the second mode; receiving an input for deactivating the application while the application is locked in the second mode; canceling the application according to the input for canceling the application; receiving input for opening the application; as well as The application is displayed in the second mode according to the input for opening the application.
14. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display and a touch-sensitive surface, the one or more programs comprising instructions for: Displaying the application in a first mode, wherein the application is displayed to have a first height when in the first mode; detecting a first input while the application is displayed in the first mode; In response to detecting the first input: If it is determined that the first input corresponds to a request to display the application in a second mode: displaying the application in a second mode by offsetting display of the application, wherein in the second mode the application is displayed having a second height that is less than the first height; while the application is displayed in the second mode and subsequently detecting an end of the first input, detecting a second input on the touch-sensitive surface; In response to detecting the second input: if it is determined that the second input is a navigation input detected on the touch-sensitive surface that corresponds to a location on the display of the application, scrolling within the application without switching back to the first mode; as well as switching from the second mode back to the first mode if it is determined that the second input is an input at a location on the touch-sensitive surface that corresponds to an external location of the application on the display; detecting a repetition of the first input while the application is displayed in the second mode; as well as In response to detecting a repetition of the first input, switching from the second mode back to the first mode.
15. A computer-readable storage medium according to claim 14, wherein in the first mode, the application includes a graphical object located at the following position on the display: the position corresponds to a distance across the touch-sensitive surface that is greater than a predefined reachable measure, and the position indicates that the graphical object is not reachable by a user's finger of the user's hand unless the electronic device is repositioned in the user's hand; and wherein in the second mode, the graphical object is located at the following position on the display: the position corresponds to a distance across the touch-sensitive surface that is less than the predefined reachable measure, and the position indicates that the graphical object is reachable by a user's finger of the user's hand without the electronic device being repositioned in the user's hand.
16. A computer-readable storage medium according to claim 14, wherein the first mode comprises a full-screen mode, wherein when in the full-screen mode, the application occupies the entirety of the display except for an area of a status bar for the electronic device, and the application is displayed as having the first height and a first width, wherein the first height corresponds to a height of the display and the first width corresponds to a width of the display.
17. The computer-readable storage medium of claim 14, wherein detecting the first input comprises: detecting a contact on the touch-sensitive surface corresponding to a location on the display at a first location adjacent a vertical edge of the display; as well as Continued movement of the contact on the touch-sensitive surface corresponding to a second location on the display adjacent a horizontal edge of the display that intersects the vertical edge is detected without the contact breaking contact with the touch-sensitive surface.
18. The computer-readable storage medium of claim 14, wherein detecting the first input comprises: A gesture is detected on a button of the electronic device corresponding to a request to display the application in the second mode, wherein a single press of the button is associated with performing a different function.
19. The computer-readable storage medium of claim 14, wherein displaying the application in the second mode comprises: A blank area is displayed in an area of the display adjacent to the application.
20. The computer-readable storage medium of claim 14, wherein displaying the application in the second mode comprises: displaying a notification area of the display adjacent to an area displaying the application, the notification area including a user interface element associated with displaying a notification view including one or more notifications for the user; detecting a contact on the touch-sensitive surface corresponding to the user interface element; detecting a continuous movement of the contact, the continuous movement of the contact moving the user interface element in a direction of the continuous movement, the continuous movement having a vertical component; If it is detected that the vertical component is less than the threshold distance: determining a magnitude of the vertical component of the continuous movement; as well as displaying a portion of the notification view in the notification area of the display adjacent to the area where the application is displayed in the second mode, wherein a height of the portion of the notification view is proportional to the determined size of the vertical component; and If it is detected that the vertical component is greater than the threshold distance, the notification view is displayed in its entirety.
21. The computer-readable storage medium of any one of claims 14 to 20, wherein displaying the application in the second mode comprises: While the application is displayed in the second mode, an image of a home screen view of the electronic device is displayed in an area of the display adjacent to the application.
22. The computer-readable storage medium of any one of claims 14 to 20, wherein the first mode of the application comprises a toolbar for the application displayed in an area of the display, the toolbar comprising a plurality of user interface elements, each user interface element being associated with a function of the application, and wherein displaying the application in the second mode comprises: Determining a view of the application displayed in the first mode and having the first height, the view comprising a first portion displaying content of the application and a second portion displaying the toolbar of the application; as well as The view of the application is shifted in a vertical direction until the view is displayed to have the second height, the shifted view including the first portion displaying the content of the application and excluding the second portion displaying the toolbar.
23. The computer-readable storage medium of any one of claims 14 to 20, wherein the first mode of the application comprises a toolbar for the application displayed in an area of the display, the toolbar comprising a plurality of user interface elements, each user interface element being associated with a function of the application, and wherein displaying the application in the second mode comprises: determining a first view of the application, the first view of the application being associated with the application in the first mode, the first view comprising a first portion displaying content of the application and a second portion displaying the toolbar of the application; as well as Determine a second view of the application, the second view of the application being associated with the application in the second mode, the second view comprising the first portion displaying content of the application and resized according to the second height and the second portion displaying the toolbar of the application and resized according to the second height.
24. The computer-readable storage medium of claim 14, wherein the one or more programs further comprise instructions for: while the application is displayed in the second mode, determining an amount of time since input to the application was last received; and If it is determined that the amount of time exceeds a threshold, automatically switching from the second mode back to the first mode.
25. The computer-readable storage medium of claim 14, wherein after entering the second mode and before detecting the second input while in the second mode, the electronic device uses a first time threshold to determine whether to automatically switch back to the first mode, and wherein the one or more programs further include instructions for: After detecting the second input, a second time threshold different from the first time threshold is used to determine whether to automatically switch back to the first mode.
26. The computer-readable storage medium of claim 14, wherein the one or more programs further comprise instructions for: receiving a request to prevent switching from the second mode back to the first mode; locking the application in the second mode; receiving an input for deactivating the application while the application is locked in the second mode; canceling the application according to the input for canceling the application; receiving input for opening the application; as well as The application is displayed in the second mode according to the input for opening the application.
27. An electronic device comprising: monitor; Touch-sensitive surfaces; one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: Displaying the application in a first mode, wherein the application is displayed to have a first height when in the first mode; detecting a first input while the application is displayed in the first mode; In response to detecting the first input: If it is determined that the first input corresponds to a request to display the application in a second mode: displaying the application in a second mode by offsetting display of the application, wherein in the second mode the application is displayed having a second height that is less than the first height; while the application is displayed in the second mode and subsequently detecting an end of the first input, detecting a second input on the touch-sensitive surface; In response to detecting the second input: if it is determined that the second input is a navigation input detected on the touch-sensitive surface that corresponds to a location on the display of the application, scrolling within the application without switching back to the first mode; as well as switching from the second mode back to the first mode if it is determined that the second input is an input at a location on the touch-sensitive surface that corresponds to an external location of the application on the display; detecting a repetition of the first input while the application is displayed in the second mode; as well as In response to detecting a repetition of the first input, switching from the second mode back to the first mode.
28. An electronic device according to claim 27, wherein in the first mode, the application includes a graphical object located at the following position on the display: the position corresponds to a distance across the touch-sensitive surface that is greater than a predefined reachable measure, and the position indicates that the graphical object is not reachable by the user's fingers of the user's hand unless the electronic device is repositioned in the user's hand; and wherein in the second mode, the graphical object is located at the following position on the display: the position corresponds to a distance across the touch-sensitive surface that is less than the predefined reachable measure, and the position indicates that the graphical object is reachable by the user's fingers of the user's hand without the electronic device being repositioned in the user's hand.
29. An electronic device according to claim 27, wherein the first mode includes a full-screen mode, wherein when in the full-screen mode, the application occupies the entirety of the display except for an area of a status bar for the electronic device, and the application is displayed as having the first height and a first width, wherein the first height corresponds to the height of the display and the first width corresponds to the width of the display.
30. The electronic device of claim 27, wherein detecting the first input comprises: detecting a contact on the touch-sensitive surface corresponding to a location on the display at a first location adjacent a vertical edge of the display; as well as Continued movement of the contact on the touch-sensitive surface corresponding to a second location on the display adjacent a horizontal edge of the display that intersects the vertical edge is detected without the contact breaking contact with the touch-sensitive surface.
31. The electronic device of claim 27, wherein detecting the first input comprises: A gesture is detected on a button of the electronic device corresponding to a request to display the application in the second mode, wherein a single press of the button is associated with performing a different function.
32. The electronic device of claim 27, wherein displaying the application in the second mode comprises: A blank area is displayed in an area of the display adjacent to the application.
33. The electronic device of claim 27, wherein displaying the application in the second mode comprises: displaying a notification area of the display adjacent to an area displaying the application, the notification area including a user interface element associated with displaying a notification view including one or more notifications for the user; detecting a contact on the touch-sensitive surface corresponding to the user interface element; detecting a continuous movement of the contact, the continuous movement of the contact moving the user interface element in a direction of the continuous movement, the continuous movement having a vertical component; If it is detected that the vertical component is less than the threshold distance: determining a magnitude of the vertical component of the continuous movement; as well as displaying a portion of the notification view in the notification area of the display adjacent to the area where the application is displayed in the second mode, wherein a height of the portion of the notification view is proportional to the determined size of the vertical component; and If it is detected that the vertical component is greater than the threshold distance, the notification view is displayed in its entirety.
34. The electronic device according to any one of claims 27 to 33, wherein displaying the application in the second mode comprises: While the application is displayed in the second mode, an image of a home screen view of the electronic device is displayed in an area of the display adjacent to the application.
35. An electronic device according to any one of claims 27 to 33, wherein the first mode of the application comprises a toolbar for the application displayed in an area of the display, the toolbar comprising a plurality of user interface elements, each user interface element being associated with a function of the application, and wherein displaying the application in the second mode comprises: Determining a view of the application displayed in the first mode and having the first height, the view comprising a first portion displaying content of the application and a second portion displaying the toolbar of the application; as well as The view of the application is shifted in a vertical direction until the view is displayed to have the second height, the shifted view including the first portion displaying the content of the application and excluding the second portion displaying the toolbar.
36. An electronic device according to any one of claims 27 to 33, wherein the first mode of the application comprises a toolbar for the application displayed in an area of the display, the toolbar comprising a plurality of user interface elements, each user interface element being associated with a function of the application, and wherein displaying the application in the second mode comprises: determining a first view of the application, the first view of the application being associated with the application in the first mode, the first view comprising a first portion displaying content of the application and a second portion displaying the toolbar of the application; as well as Determine a second view of the application, the second view of the application being associated with the application in the second mode, the second view comprising the first portion displaying content of the application and resized according to the second height and the second portion displaying the toolbar of the application and resized according to the second height.
37. The electronic device of claim 27, wherein the one or more programs further comprise instructions for: while the application is displayed in the second mode, determining an amount of time since input to the application was last received; and If it is determined that the amount of time exceeds a threshold, automatically switching from the second mode back to the first mode.
38. The electronic device of claim 27, wherein after entering the second mode and before detecting the second input while in the second mode, the electronic device uses a first time threshold to determine whether to automatically switch back to the first mode, and wherein the one or more programs further include instructions for: After detecting the second input, a second time threshold different from the first time threshold is used to determine whether to automatically switch back to the first mode.
39. The electronic device of claim 27, wherein the one or more programs further comprise instructions for: receiving a request to prevent switching from the second mode back to the first mode; locking the application in the second mode; receiving an input for deactivating the application while the application is locked in the second mode; canceling the application according to the input for canceling the application; receiving input for opening the application; as well as The application is displayed in the second mode according to the input for opening the application.
40. An electronic device comprising: monitor; Touch-sensitive surfaces; as well as Device for carrying out the method according to any one of claims 1 to 7 and 11 to 13.
41. A computer program product comprising one or more programs configured to be executed by one or more processors of an electronic device having a display and a touch-sensitive surface, the one or more programs including instructions for executing a method according to any one of claims 1 to 7 and 11 to 13.
Citation Information
Patent Citations
Touch screen device, method, and graphical user interface for determining commands by applying heuristics
AU2008100179A4
Human-computer interaction method and interface
CA2891909A1
Image processing apparatus and image processing method
CN101616258A
Portable electronic equipment and method capable of dynamically regulating graphical controls
CN103064629A
Mobile terminal and operation control method thereof
CN103186325A
