Systems, devices, and methods for dynamically providing user interface controls at touch-sensitive secondary display

Abstract

Embodiments of the present disclosure relate to systems, devices, and methods for dynamically providing user interface controls at a touch-sensitive secondary display. An exemplary method includes receiving a request to open an application, and in response: (i) displaying on a primary display a plurality of user interface (UI) objects associated with the application, the plurality of UI objects including a first UI object displayed with associated content and other UI objects not displayed with the associated content; and (ii) displaying a set of affordance representations representing the plurality of UI objects on the touch-sensitive secondary display. The method further includes detecting, by the touch-sensitive secondary display, a swipe gesture in a direction from a first affordance to a second affordance, the first affordance representing a first UI object and the second affordance representing a second, different UI object. In response to detecting the swipe gesture, the method includes updating the primary display to display associated content of the second UI object.

Description

[0001] Description of the case

[0002] This application is a divisional application of the invention patent application with the application date of July 28, 2017, the national application number 202110182280.2, and the invention name “System, device and method for dynamically providing user interface controls at a touch-sensitive secondary display”. Technical Field

[0003] The disclosed embodiments relate to keyboards and, more particularly, to improved techniques for receiving input via dynamic input and output (I / O) devices. Background Art

[0004] A conventional keyboard includes any number of physical keys for inputting information (e.g., characters) into a computing device. Typically, a user presses a key or otherwise movably actuates a key to provide input corresponding to the key. In addition to providing character input, a keyboard may include movably actuated keys associated with function input. For example, a keyboard may include an "escape" or "esc" key to allow a user to activate an escape or exit function. In many keyboards, a set of function keys for function input is located in a "function row." Typically, a set of keys for alphanumeric characters is located in the portion of the keyboard closest to the user, and the function row is located in the portion of the keyboard away from the user but adjacent to the alphanumeric characters. A keyboard may also include function keys that are not part of the function row.

[0005] With the advent and popularity of portable computing devices such as laptop computers, the area occupied by a dedicated keyboard may be limited by the corresponding size of the display. Compared to peripheral keyboards used for desktop computers, dedicated keyboards as components of portable computing devices may have fewer keys, smaller keys, or keys that are closer together to allow the portable computing device to have a smaller overall size.

[0006] Conventional dedicated keyboards are static and fixed in time, regardless of what is happening on the display. Furthermore, the functionality of software applications displayed on the screen is typically accessed through toolbars and menus that the user interacts with using a mouse. This requires the user to regularly switch modes and move their hand position between the keyboard and the mouse. Alternatively, the functionality of the application is accessed through complex key combinations that require memorization and practice. Therefore, it is desirable to provide an I / O device (and method for an I / O device) that can address the shortcomings of conventional systems. Summary of the Invention

[0007] The embodiments described herein solve the above-mentioned shortcomings by providing dynamic and space-efficient I / O devices and methods. Such devices and methods optionally supplement or replace conventional input devices and methods. Such devices and methods also reduce the mode switching amount required by the user (e.g., moving both hands between the keyboard and the mouse, and also moving eyes from the keyboard to the display), thereby reducing the number of inputs required to activate the desired function (e.g., reducing the number of inputs required to select a menu option, as explained in more detail below). Such devices and methods also provide more information on a limited screen (e.g., using a touch-sensitive secondary display to provide more information to the user, and effectively presenting the information using limited screen space). Such devices and methods also provide improved human-computer interfaces, such as by providing emphasis effects to make information on the touch-sensitive secondary display 104 more legible, by providing continuous interaction so that continuous input from a user directed to the touch-sensitive secondary display or the primary display causes the device to provide output that is then used to facilitate further input from the user (e.g., providing a color selector that allows a user to quickly preview how information will be presented on the primary display by providing input on the touch-sensitive secondary display, as described below), and by requiring less interaction from the user to achieve a desired result (e.g., allowing a user to log in to their device using a single biometric input, as described below). For these reasons, and those described below, the devices and methods described herein reduce power usage and extend battery life of electronic devices.

[0008] (A1) According to some embodiments, in a computing system (e.g., computing system 100 or system 200, Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is included as a peripheral input mechanism 222).

[0009] part of a touch-sensitive display (i.e., a separate display), or a touch-sensitive display coupled with another device (such as a Figure 2CThe method includes: displaying a first user interface on a primary display, the first user interface including one or more user interface elements; identifying an active user interface element of the one or more user interface elements that is focused on the primary display; determining whether the active user interface element that is focused on the primary display is associated with an application executed by the computing system; and based on determining that the active user interface element that is focused on the primary display is associated with the application executed by the computing system, displaying a second user interface on the touch screen display including: (A) a first set of one or more affordances corresponding to the application;

[0010] and (B) at least one system-level affordance corresponding to at least one system-level function.

[0011] Displaying application-specific and system-level affordances in a touch-sensitive secondary display in response to a focus change occurring on the primary display provides the user with accessible affordances directly available through the touch-sensitive secondary display. Providing the user with accessible affordances directly accessible through the touch-sensitive secondary display enhances the operability of the computing system and makes the user device interface more efficient (e.g., by helping the user access desired functionality directly through the touch-sensitive secondary display with less interaction rather than having to waste time digging through hierarchical menus to locate the desired functionality), and also reduces power usage and extends the battery life of the device by enabling the user to access desired functionality more quickly and efficiently. Similarly, displaying application-specific affordances on the touch-sensitive secondary display indicates the internal state of the device by providing affordances associated with the application currently focused on the primary display.

[0012] (A2) In some embodiments of the method of A1, the computing system further comprises: (i) a host computing device comprising a host display, a processor, a memory, and host computing device communication circuitry;

[0013] and (ii) an input device comprising a housing, a touch screen display, a physical input mechanism, and input device communication circuitry for communicating with a host computing device communication circuitry, wherein the input device is distinct and separate from the host computing device.

[0014] (A3) In some embodiments of the method of any of A1-A2, the physical input mechanism includes a plurality of physical keys.

[0015] (A4) In some embodiments of the method of any one of A1-A3, the physical input mechanism includes a touchpad.

[0016] (A5) In some embodiments of the method of any one of A1-A4, the application is executed by the processor in the foreground of the first user interface.

[0017] (A6) In some embodiments of the method of any of items A1-A5, at least one system-level enable representation is configured to cause multiple system-level enable representations corresponding to system-level functions to be displayed on the touch screen display upon selection.

[0018] (A7) In some implementations of the method of any of A1-A3, at least one system-level indication corresponds to one of power control or evacuation control.

[0019] (A8) In some embodiments of the method of any one of items A1-A7, at least one enable indication displayed on the second user interface is a multi-function enable indication.

[0020] (A9) In some embodiments of the method of A8, the method also includes: detecting a user touch input that selects a multi-function enable indication; executing a first function associated with the multi-function enable indication based on determining that the user touch input corresponds to a first type; and executing a second function associated with the multi-function enable indication based on determining that the user touch input corresponds to a second type different from the first type.

[0021] (A10) In some embodiments of the method of any one of A1-A9, the method further comprises:

[0022] Based on determining that the active user interface element is not associated with an application executed by the computing system, displaying a third user interface on the touch screen display, comprising: (C) a second set of one or more affordances corresponding to operating system controls of the computing system, wherein the second set of one or more affordances is different from the first set of one or more affordances. (A11) In some embodiments of the method of A10, the second set of one or more affordances is an extended set of operating system controls that includes (B) (i.e., at least one system-level affordance corresponding to at least one system-level function).

[0023] (A12) In some embodiments of the method of any one of A1-A11, the method further comprises:

[0024] Detecting user touch input that selects one of the first set of enable representations; and in response to detecting the user touch input, displaying a different set of enable representations corresponding to functions of the application, and maintaining display of at least one system-level enable representation.

[0025] (A13) In some embodiments of the method A12, the method further includes: detecting a subsequent user touch input that selects at least one system-level enable representation; and in response to detecting the subsequent user touch input, displaying a plurality of system-level enable representations corresponding to system-level functions and at least one application-level enable representation corresponding to an application.

[0026] (A14) In some embodiments of the method of any one of A1-A13, the method further comprises:

[0027] After displaying a second user interface on the touch screen display, identifying a second active user interface element that is focused on the main display among the one or more user interface elements; determining whether the second active user interface element corresponds to a different application executed by the computing device; and based on determining that the second active user interface element corresponds to a different application, displaying a fourth user interface on the touch screen display, including: (D) a third group of one or more enable representations corresponding to the different applications; and (E) at least one system-level enable representation corresponding to at least one system-level function.

[0028] (A15) In some embodiments of the method of any one of A1-A14, the method further comprises:

[0029] After identifying the second active user interface element, determining whether the computing system is playing a media item, wherein the media item is not associated with a different application; and based on determining that the computing system is playing the media item, displaying at least one persistent enable representation for controlling the media item on the touch screen display.

[0030] (A16) In some embodiments of the method of A15, at least one persistent indicator displays feedback corresponding to the media item.

[0031] (A17) In some embodiments of the method of any one of A1-A16, the method further comprises:

[0032] Detecting user input corresponding to a copy key; and in response to detecting the user input: ceasing to display at least a first set of one or more enable representations of a second user interface on the touch screen display; and displaying a first set of default function keys.

[0033] (A18) In some embodiments of the method of A17, the method further includes: after displaying the first set of default function keys, detecting a swipe gesture on the touch screen display in a direction substantially parallel to the main axis of the touch screen display; and in response to detecting the swipe gesture, displaying a second set of default function keys having at least one different function key.

[0034] (A19) In another aspect, a computing system is provided, the computing system comprising one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods A1-A18.

[0035] (A20) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods A1-A18, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0036] (A21) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims A1-A18.

[0037] (A22) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display; a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display; and means for performing or causing performance of any of the methods of claims A1-A18.

[0038] (B1) According to some embodiments, an input device is provided. The input device includes:

[0039] A housing at least partially enclosing a plurality of components, the plurality of components including: (i) a plurality of physical keys (e.g., Figure 1A (i) a second touch-sensitive display (also referred to as a "touch screen display") positioned adjacent to the plurality of physical keys; and (ii) a second touch-sensitive display (also referred to as a "touch screen display") positioned adjacent to the plurality of physical keys; and (iii) a short-range communication circuit configured to communicate with a computing device (e.g., computing system 100 or 200) positioned adjacent to the input device, wherein the computing device includes a computing device display, a processor, and a memory, and the short-range communication circuit is configured to: transmit key presses of any of the plurality of physical keys and touch inputs on the touch screen display to the computing device;

[0040] and receiving, based on the current focus of the computing device display, instructions for changing the display of the affordances on the touch screen display.In some embodiments, when an application is focused on the computing device display, the touch screen display is configured to display the following: (A) one or more affordances corresponding to the focused application;

[0041] and (B) at least one system-level affordance, wherein the at least one system-level affordance is configured to, upon selection, cause display of a plurality of affordances corresponding to system-level functions.

[0042] Displaying application-specific and system-level affordances in a touch-sensitive secondary display in response to changes in focus occurring on a primary display provides the user with accessible affordances that are directly available through the touch-sensitive secondary display. Providing the user with accessible affordances that are directly accessible through the touch-sensitive secondary display enhances the operability of the computing system and makes the user device interface more efficient (e.g., by helping the user access desired functionality directly through the touch-sensitive secondary display with less interaction and without having to waste time digging through hierarchical menus to locate the desired functionality), and also reduces power usage and extends the battery life of the device by enabling the user to access desired functionality more quickly and efficiently. Furthermore, by dynamically updating affordances displayed in the touch-sensitive secondary display based on changes in focus on the primary display, the touch-sensitive secondary display is able to provide more information on a limited screen and helps ensure that the user is provided with the desired options when the user needs them (thereby reducing power usage and extending battery life because the user does not need to waste power and battery life searching through hierarchical menus to locate those desired options).

[0043] (B2) In some embodiments of the input device of B1, when the application is focused on the computing device display, the touch screen display is further configured to display at least one of a power control enable indication and a de-enable indication.

[0044] (B3) In some embodiments of the input device of any of B1-B2, the input device is a keyboard.

[0045] (B4) In some embodiments of the input device of B3, the computing device is a laptop computer including a keyboard.

[0046] (B5) In some embodiments of the input device of B3, the computing device is a desktop computer,

[0047] And the keyboard is different from a desktop computer.

[0048] (B6) In some embodiments of the input device of any of B1-B5, the input device is integrated into a laptop computer.

[0049] (B7) In some embodiments of the input device of any of B1-B6, the plurality of physical keys includes a QWERTY keyboard.

[0050] (B8) In some embodiments of the input device of any of B1-B7, the alphabet corresponds to the Latin alphabet.

[0051] (B9) In some embodiments of the input device of any one of B1-B8, the input device includes a touchpad.

[0052] (B10) In some embodiments of the input device of any of B1-B9, the input device has a major dimension of at least 18 inches long.

[0053] (B11) In some embodiments of the input device of any of B1-B10, the short-range communication circuit is configured to communicate with the computing device at a distance of less than 15 feet.

[0054] (B12) In some embodiments of the input device of any of B1-B11, the short-range communication circuit corresponds to a wired or wireless connection to the computing device.

[0055] (B13) In some embodiments of the input device of any of B1-B12, the input device includes a fingerprint sensor embedded in the touch screen display.

[0056] (C1) According to some embodiments, in a computing system (e.g., system 100 or system 200,

[0057] Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A keyboard 106) and a touch-sensitive secondary display

[0058] (For example, Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a touchscreen display). Figure 2CThe method includes: displaying a first user interface of an application executed by the computing system on a main display; displaying a second user interface on the touch screen display, the second user interface including a first set of one or more affordances corresponding to the application, wherein the first set of one or more affordances corresponds to a first portion of the application; detecting a swipe gesture on the touch screen display; based on determining that the swipe gesture is performed in a first direction, displaying a second set of one or more affordances corresponding to the application on the touch screen display, wherein at least one affordance of the second set of one or more affordances is different from the first set of one or more affordances, and wherein the second set of one or more affordances also corresponds to the first portion of the application; and based on determining that the swipe gesture is performed in a second direction substantially perpendicular to the first direction, displaying a third set of one or more affordances corresponding to the application on the touch screen display, wherein the third set of one or more affordances is different from the second set of one or more affordances, and wherein the third set of one or more affordances corresponds to a second portion of the application, the second portion being different from the first portion of the application.

[0059] Allowing a user to quickly navigate through application-specific affordances in a touch-sensitive secondary display in response to swipe gestures provides the user with a convenient way to scroll through and quickly locate desired functionality via the touch-sensitive secondary display. Providing a convenient way for the user to quickly scroll through and quickly locate desired functionality via the touch-sensitive secondary display enhances the operability of the computing system and makes the user device interface more efficient (e.g., by helping the user access desired functionality directly through the touch-sensitive secondary display with less interaction and without having to waste time digging through hierarchical menus to locate the desired functionality), and further reduces power usage and extends the battery life of the device by enabling the user to access desired functionality more quickly and efficiently. Furthermore, by dynamically updating affordances displayed in the touch-sensitive secondary display in response to swipe gestures on the secondary display, the secondary display is able to provide more information on a limited screen and helps ensure that the user's desired options are provided when the user needs them (thereby reducing power usage and extending battery life because the user does not need to waste power and battery life searching through hierarchical menus to locate those desired options).

[0060] (C2) In some embodiments of the method of C1, before the swipe gesture is detected, the second part is displayed on the main display in a compact view within the first user interface, and the method includes: based on determining that the swipe gesture is performed in a second direction substantially perpendicular to the first direction, displaying the second part on the main display in an expanded view within the first user interface.

[0061] (C3) In some embodiments of the method of C1, a first user interface of an application executed by the computing system is displayed in full-screen mode on the main display, and a first set of one or more enabling representations displayed on the touch screen display includes controls corresponding to the full-screen mode.

[0062] (C4) In some embodiments of the method of any of items C1-C3, the second set of one or more enable representations and the third set of one or more enable representations include at least one system-level enable representation corresponding to at least one system-level function.

[0063] (C5) In some embodiments of the method of any one of items C1-C4, the method includes: after displaying a third group of one or more enable representations on the touch screen display: detecting user input selecting the first portion on the first user interface; and in response to detecting the user input: stopping displaying the third group of one or more enable representations on the touch screen display, wherein the third group of one or more enable representations corresponds to the second portion of the application; and displaying a second group of one or more enable representations, wherein the second group of one or more enable representations corresponds to the first portion of the application.

[0064] (C6) In some embodiments of the method of any of C1-C5, the first direction is substantially parallel to a major dimension of the touch screen display.

[0065] (C7) In some embodiments of the method of any of C1-C5, the first direction is substantially perpendicular to a major dimension of the touch screen display.

[0066] (C8) In some embodiments of the method of any of C1-C7, the first part is one of a menu, tab, folder, toolset, or toolbar of the application, and the second part is one of a menu, tab, folder, toolset, or toolbar of the application.

[0067] (C9) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods C1-C8.

[0068] (C10) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods C1-C8, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0069] (C11) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims C1-C8.

[0070] (C12) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims C1-C8.

[0071] (D1) According to some embodiments, in a computing system (e.g., system 100 or system 200, Figures 1A to 2D ), wherein the method of performing a function of maintaining an application in full screen mode in a computing system includes one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a touchscreen display). Figure 2CThe method includes: displaying a first user interface of an application executed by the computing system on a primary display in a normal mode, the first user interface including a first set of one or more affordances associated with the application; detecting user input for displaying at least a portion of the first user interface of the application in a full-screen mode on the primary display; and in response to detecting the user input: ceasing to display the first set of one or more affordances associated with the application in the first user interface on the primary display; displaying a portion of the first user interface of the application in a full-screen mode on the primary display; and automatically displaying a second set of one or more affordances for controlling the application on the touch screen display without human intervention, wherein the second set of one or more affordances corresponds to the first set of one or more affordances.

[0072] Providing an affordance for controlling an application via a touch-sensitive secondary display while displaying a portion of the application in full-screen mode on the primary display allows the user to continue to access functionality that may no longer be directly displayed on the primary display. Allowing the user to continue to access functionality that may no longer be directly displayed on the primary display provides the user with a quick way to access functionality that may have been buried on the primary display, thereby enhancing the operability of the computing system and making the user device interface more efficient (e.g., by helping the user access desired functionality directly through the touch-sensitive secondary display with less interaction rather than having to waste time digging through hierarchical menus to locate the desired functionality), and also reducing power usage and extending the battery life of the device by enabling the user to access desired functionality more quickly and efficiently. Therefore, by transitioning menu options from the primary display to the touch-sensitive secondary display to ensure that content can be presented in full-screen mode (where accessible), the user is able to maintain interaction with the device and their workflow is not interrupted when transitioning to full-screen mode. Additionally, accessing menu options while viewing full-screen content requires less interaction as menu options that may have been buried behind content on the primary display are presented on the touch-sensitive secondary display for quick access (without having to exit full-screen mode and then dig around to find the menu option), thereby reducing power usage and extending the battery life of the device.

[0073] (D2) In some embodiments of the method of D1, the second set of one or more enable representations is the first set of one or more enable representations.

[0074] (D3) In some embodiments of the method of any of D1-D2, the second set of one or more enable representations includes controls corresponding to a full-screen mode.

[0075] (D4) In some embodiments of the method of any one of D1-D3, the method includes: detecting user touch input that selects one of a second set of enabled representations displayed on the touch screen display; and in response to detecting the user touch input, changing a portion of a first user interface of an application displayed in full screen mode on the main display according to the selected one of the second set of enabled representations.

[0076] (D5) In some embodiments of the method of any one of D1-D4, the method includes: after displaying a portion of a first user interface of an application in full screen mode on a primary display: detecting a subsequent user input for exiting full screen mode; and in response to detecting the subsequent user input: displaying a first user interface of an application executed by the computing system in normal mode on the primary display, the first user interface including a first set of one or more enable representations associated with the application; and maintaining display on the touch screen display at least a subset of a second set of one or more enable representations for controlling the application, wherein the second set of one or more enable representations corresponds to the first set of one or more enable representations.

[0077] (D6) In some embodiments of the method of any of D1-D5, the user input for displaying at least a portion of the first user interface of the application in full screen mode on the primary display is at least one of a touch input detected on the touch screen display and a control selected within the first user interface on the primary display.

[0078] (D7) In some embodiments of the method of any of D1-D6, the second set of one or more enable representations includes at least one system-level enable representation corresponding to at least one system-level function.

[0079] (D8) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods of D1-D7.

[0080] (D9) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods in D1-D7, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0081] (D10) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims D1-D7.

[0082] (D11) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims D1-D7.

[0083] (E1) According to some embodiments, in a computing system (e.g., system 100 or system 200,

[0084] Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A keyboard 106) and a touch-sensitive secondary display

[0085] (For example, Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a touchscreen display). Figure 2C The method includes: displaying a first user interface of an application executed by the computing system on a main display; displaying a second user interface on the touch screen display, the second user interface including a set of one or more affordances corresponding to the application; detecting a notification;

[0086] And in response to detecting the notification, simultaneously displaying the group of one or more enabling representations corresponding to the application and at least a portion of the detected notification on the touch screen display in a second user interface, wherein the detected notification is not displayed on the main display.

[0087] Displaying received notifications on the touch-sensitive secondary display allows the user to continue working on the primary display in an uninterrupted manner and allows the user to interact with the received notifications via the touch-sensitive secondary display. Allowing the user to continue working on the primary display in an uninterrupted manner and allowing the user to interact with the received notifications via the touch-sensitive secondary display provides the user with a quick way to view and interact with the received notifications, thereby enhancing the operability of the computing system and making the user device interface more efficient (for example, by helping the user to conveniently access the received notifications directly through the touch-sensitive secondary display without having to interrupt their workflow to process the received notifications). In addition, displaying the received notifications on the touch-sensitive secondary display provides an emphasis effect for the notifications received on the touch-sensitive secondary display (because in some embodiments, the received notifications are displayed as covering other enable representations in the touch-sensitive secondary display), thereby ensuring that the received notifications are visible and easily accessible on the touch-sensitive secondary display.

[0088] (E2) In some embodiments of the method of E1, the method includes: before detecting the notification, detecting user input selecting a notification setting so that the notification is displayed on the touch screen display and not on the main display.

[0089] (E3) In some embodiments of the method of any one of items E1-E2, the method includes: detecting user touch input on the touch screen display corresponding to a portion of the detected notification; based on determining that the user touch input corresponds to a first type, stopping displaying the portion of the detected notification on the touch screen display in a second user interface; and based on determining that the user touch input corresponds to a second type different from the first type, performing an action associated with the detected notification.

[0090] (E4) In some embodiments of the method of any of E1-E3, the portion of the notification displayed on the touch screen display prompts a user of the computing system to select one of a plurality of options for responding to the detected notification.

[0091] (E5) In some embodiments of the method of any of E1-E4, the portion of the notification displayed on the touch screen display includes one or more suggested responses to the detected notification.

[0092] (E6) In some embodiments of the method of any of E1-E5, the notification corresponds to at least one of an incoming instant message, SMS, email, voice call, or video call.

[0093] (E6) In some embodiments of the method of any of E1-E5, the notification corresponds to a modal alert issued by an application executed by a processor of the computing system in response to user input to close the application or perform an action within the application.

[0094] (E7) In some embodiments of the method of any of items E1-E7, the group of one or more enable representations includes at least one system-level enable representation corresponding to at least one system-level function, and notifies user input corresponding to one or more portions of a selection input mechanism or at least one of the system-level enable representations.

[0095] (E8) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods of E1-E7.

[0096] (E9) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods E1-E7, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0097] (E10) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims E1-E7.

[0098] (E11) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display; a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display; and means for performing or causing performance of any of the methods of claims E1-E7.

[0099] (F1) According to some embodiments, in a computing system (e.g., system 100 or system 200, Figures 1A to 2D ), wherein the computing system comprises one or more processors, a memory, a first housing including a main display,

[0100] (e.g., housing 110 including display 102 or housing 204 including display 102), and at least partially containing a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1AIn some embodiments, the touch-sensitive secondary display is separate from the physical keyboard.

[0101] (For example, the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as Figure 2C The method includes: displaying a user interface on a primary display, the user interface including one or more user interface elements; identifying an active user interface element of the one or more user interface elements that is focused on the primary display, wherein the active user interface element is associated with an application executed by the computing system; in response to identifying the active user interface element, displaying a set of one or more affordances corresponding to the application on the touch screen display; detecting user input that moves a corresponding portion of the user interface; and in response to detecting the user input and based on determining that the user input satisfies predefined action criteria: ceasing to display the corresponding portion of the user interface on the primary display; ceasing to display at least a subset of the set of one or more affordances on the touch screen display; and displaying a representation of the corresponding portion of the user interface on the touch screen display.

[0102] Allowing a user to quickly move a user interface portion (e.g., menus, notifications, etc.) from a primary display to a touch-sensitive secondary display provides the user with a convenient and customized way to access the user interface portion. Providing the user with a convenient and customized way to access the user interface portion via the touch-sensitive secondary display enhances the operability of the computing system and makes the user device interface more efficient (e.g., by helping the user access the user interface portion directly through the touch-sensitive secondary display with less interaction and without having to waste time searching for a previously viewed (possibly buried) user interface portion), and further reduces power usage and extends the battery life of the device by enabling the user to more quickly and efficiently access the desired user interface portion. Furthermore, displaying the user interface portion on the touch-sensitive secondary display in response to user input provides an emphasis effect for the user interface portion at the touch-sensitive secondary display (because, in some embodiments, the corresponding user interface portion is displayed as overlaying other affordances in the touch-sensitive secondary display), thereby ensuring that the corresponding user interface portion is visible and easily accessible on the touch-sensitive secondary display.

[0103] (F2) In some embodiments of the method of F1, the corresponding portion of the user interface is a menu corresponding to an application executed by the computing system.

[0104] (F3) In some embodiments of the method of any of F1-F2, the corresponding portion of the user interface is one of a notification and a modal alert.

[0105] (F4) In some embodiments of the method of any one of F1-F3, the predefined action criteria are met when the user input is a drag gesture of dragging the corresponding portion of the user interface to a predefined position of the main display.

[0106] (F5) In some embodiments of the method of any one of F1-F3, the predefined action criterion is met when the user input is a predefined input corresponding to moving a corresponding portion of the user interface to the touch screen display.

[0107] (F6) In some embodiments of the method of any one of F1-F5, the method includes: in response to detecting the user input, and based on determining that the user input does not meet the predefined action criteria: maintaining the corresponding portion of the user interface displayed on the primary display; and

[0108] And keep displaying one or more enabling indications of the group on the touch screen display.

[0109] (F7) In some embodiments of the method of any one of F1-F6, the group of one or more enable indications includes at least one system-level enable indication corresponding to at least one system-level function, and the method includes: after displaying a representation of a corresponding portion of the user interface on the touch screen display, maintaining the display of the at least one system-level enable indication on the touch screen display.

[0110] (F8) In some embodiments of the method of any of F1-F7, a representation of the corresponding portion of the user interface is overlaid on the set of one or more enabling representations on the touch screen display.

[0111] (F9) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods F1-F8.

[0112] (F10) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods F1-F8, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0113] (F11) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims F1-F8.

[0114] (F12) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display; a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display; and means for performing or causing performance of any of the methods of claims F1-F8.

[0115] (G1) According to some embodiments, in a computing system (e.g., system 100 or system 200,

[0116] Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C 108 in the touch-sensitive secondary display). The method includes: receiving a request to open an application. In response to receiving the request, the method includes: (i) displaying a plurality of user interface objects associated with an application executed on the computing system (e.g., a plurality of user interface objects corresponding to tabs in Safari, individual photos in a photo browsing application, individual video frames in a video editing application, etc.) on the primary display, the plurality of user interface objects including a first user interface object displayed with content associated therewith and other user interface objects displayed without content associated therewith; and (ii) displaying a set of enable representations on the touch-sensitive secondary display, each representing (i.e., corresponding to) one of the plurality of user interface objects. The method also includes: detecting, by the touch-sensitive secondary display, a swipe gesture in a direction from a first enable representation of the set of enable representations to a second enable representation of the set of enable representations. In some embodiments, the first enable representation represents a first user interface object, and the second enable representation represents a second user interface object that is different from the first user interface object.

[0117] In response to detecting the swipe gesture, the method includes updating the primary display (e.g., during the swipe gesture) to cease displaying the associated content of the first user interface object,

[0118] And display the associated content of the second user interface object.

[0119] Allowing a user to quickly navigate through user interface objects (e.g., browser tabs) on a primary display by providing input on a touch-sensitive secondary display provides a convenient way for the user to quickly navigate through user interface objects. Providing a convenient way for the user to quickly navigate through user interface objects via the touch-sensitive secondary display (and reducing the number of inputs required to navigate through user interface objects, thereby requiring fewer interactions to navigate through user interface objects) enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on the touch-sensitive secondary display to navigate through user interface objects on the primary display). Furthermore, by enabling the user to more quickly and efficiently navigate through user interface objects on the primary display, power usage is reduced and the battery life of the device is extended. Furthermore, when the user provides input (e.g., a swipe gesture) on the touch-sensitive display to navigate through user interface objects on the primary display, each contacted affordance (corresponding to one of the user interface objects) on the touch-sensitive display is visually distinguishable from the other affordances (e.g., the corresponding contacted affordance is enlarged and a border is highlighted), thereby making information displayed on the touch-sensitive secondary display more discernible to the user.

[0120] (G2) In some embodiments of the method of G1, the method includes: detecting continued progression of the swipe gesture across the touch-sensitive secondary display, including the swipe gesture contacting a third affordance representing a third user interface object. In response to detecting the swipe gesture contacting the third affordance, the method includes: updating the primary display to display content associated with the third user interface object.

[0121] (G3) In some embodiments of the method of any of G1-G2, each enable representation in the set of enable representations includes a representation of corresponding associated content of a corresponding user interface object in a plurality of user interface objects.

[0122] (G4) In some embodiments of the method of any of G1-G3, the method includes: before detecting the swipe gesture, detecting initial contact with the touch-sensitive secondary display over the first indicator. In response to detecting the initial contact, the method includes: increasing the magnification level (or display size) of the first indicator.

[0123] (G5) In some embodiments of the method of any of G1-G4, the application is a web browsing application, and the plurality of user interface objects each corresponds to a web browsing tab.

[0124] (G6) In some embodiments of the method of G6, the method includes: detecting input at a URL input portion of a web browsing application on the primary display. In response to detecting the input, the method includes: updating the touch-sensitive secondary display to include a representation of the favorite URL.

[0125] (G7) In some embodiments of the method of any of G1-G4, the application is a photo browsing application, and the plurality of user interface objects each correspond to a respective photo.

[0126] (G8) In some embodiments of the method of any of G1-G4, the application is a video editing application, and the multiple user interface objects each correspond to a respective frame in the corresponding video.

[0127] (G9) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods of G1-G8.

[0128] (G10) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods G1-G8, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0129] (G11) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims G1-G8.

[0130] (G12) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims G1-G8.

[0131] (H1) According to some embodiments, in a computing system (e.g., system 100 or system 200,

[0132] Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C 108 in the touchpad). The method includes: receiving a request to search within content displayed on a primary display of the computing device (for example, the request corresponds to a request to search for text within displayed web page content). In response to receiving the request, the method includes: (i) displaying multiple search results responsive to the search on the primary display, with a focus on a first search result of the multiple search results; and (ii) displaying respective representations of respective search results corresponding to the multiple search results on the touch-sensitive secondary display. The method also includes: detecting, via the touch-sensitive secondary display, a touch input (for example, a tap or a swipe) selecting a representation in the respective representations, the representation corresponding to a second search result in the multiple search results that is different from the first search result. In response to detecting the input,

[0133] The method includes changing focus on the primary display to the second search result.

[0134] Allowing a user to quickly navigate through search results on a primary display by providing input at a touch-sensitive secondary display provides a convenient way for the user to quickly navigate through search results. Providing a convenient way for the user to quickly navigate through search results via the touch-sensitive secondary display (and reducing the number of inputs required to navigate through search results, thereby requiring less user interaction to quickly browse through multiple search results) enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on the touch-sensitive secondary display to navigate through multiple search results on the primary display), and further reduces power usage and extends the battery life of the device by enabling the user to more quickly and efficiently navigate through search results on the primary display. Furthermore, when the user provides input (e.g., a swipe gesture) on the touch-sensitive display to navigate through searches on the primary display, each contacted affordance (corresponding to one of the search results) on the touch-sensitive display is visually distinguishable from the other affordances (e.g., the corresponding contacted affordance is enlarged and a border is highlighted), thereby making the information displayed on the touch-sensitive secondary display more discernible to the user.

[0135] (H2) In some embodiments of the method of H1, changing the focus includes modifying a visual characteristic of a particular search result on the primary display (eg, displaying the particular search result in a larger font size).

[0136] (H3) In some embodiments of the method of any of H1-H2, the method includes: detecting a gesture that moves across at least two of the corresponding representations on the touch-sensitive secondary display. In response to detecting the gesture, the method includes: when the swipe gesture moves across the at least two of the corresponding representations, changing the focus on the primary display to the corresponding search results corresponding to the at least two of the corresponding representations.

[0137] (H4) In some embodiments of the method of H3, the method includes: based on determining that the speed of the gesture is above a threshold speed, changing the focus on the main display to corresponding search results other than corresponding search results corresponding to at least two of the corresponding representations (for example, if above the threshold speed, cycling through more search results other than those contacted during the swipe).

[0138] (H5) In some embodiments of the method of any of H3-H4, the gesture is a swipe gesture. (H6) In some embodiments of the method of any of H3-H4, the gesture is a flick gesture. (H7) In some embodiments of the method of any of H1-H6, the representations are tick marks that each correspond to a respective search result in the search results.

[0139] (H8) In some embodiments of the method of H7, the tick marks are displayed in rows on the touch-sensitive secondary display in an order corresponding to the ranking of the search results on the primary display.

[0140] (H9) In some embodiments of the method of any of H1-H8, the request to search within the content is a request to locate a search string within the content, and the plurality of search results each include at least the search string.

[0141] (H9) In some embodiments of the method of H8, displaying the plurality of search results includes highlighting a search string for each of the plurality of search results.

[0142] (H10) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods described in H1-H9.

[0143] (H11) On the other hand, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause the performance of any of the methods H1-H9, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0144] (H12) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims H1-H9.

[0145] (H13) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims H1-H9.

[0146] (I1) According to some embodiments, in a computing system (e.g., system 100 or system 200, Figures 1A to 2D), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A keyboard 106) and a touch-sensitive secondary display

[0147] (For example, Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C The method includes displaying a calendar application on the primary display. The method also includes receiving a request to display information about an event associated with the calendar application (e.g., the request corresponds to a selection of an event displayed within the calendar application on the primary display). In response to receiving the request, the method includes: (i) displaying event details of a first event on the primary display, the event details including a start time and an end time of the event; and (ii) displaying an affordance on the touch-sensitive secondary display, the affordance (e.g., a user interface control)

[0148] Indicates a time range that includes at least a start time and an end time.

[0149] Allowing a user to quickly and easily edit event details on a touch-sensitive secondary display provides a user with a quick way to quickly edit event details without having to perform additional input (e.g., having to jump back and forth between using a keyboard and using a touchpad to modify event details). Providing a convenient way for a user to quickly edit event details via a touch-sensitive secondary display (and reducing the number of inputs required to edit event details, thereby requiring less interaction to achieve the desired result of editing event details) enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on the touch-sensitive secondary display to quickly edit certain event details), and also reduces power usage and extends the battery life of the device by enabling the user to edit event details more quickly and efficiently. Additionally, by updating the primary display in response to input at the touch-sensitive secondary display (e.g., to show updated event start and end times), the user is able to maintain interaction with the device in an efficient manner by providing input to modify the event and then immediately see the modifications reflected on the primary display so that the user can decide whether to provide additional input.

[0150] (I2) In some embodiments of the method of I1, the method includes: detecting, via a touch-sensitive secondary display, an input at a user interface control that modifies a time range. In response to detecting the input: (i) modifying at least one of a start time and an end time of the event based on the input; and (ii) displaying the modified time range of the event on the primary display based on the input.

[0151] (I3) In some embodiments of the method of I2, the method includes: saving the event with the modified start and / or end time to the memory of the computing system.

[0152] (I4) In some embodiments of the method of any of items I1-I3, the input that modifies the time range is a press input that remains in contact with the enable representation for more than a threshold amount of time and then moves at least a portion of the enable representation on the touch-sensitive secondary display.

[0153] (I5) In some embodiments of the method of any of items I1-I3, the input for modifying the time range is a swipe gesture that moves across the touch-sensitive secondary display and causes the computing system to select a new start time and a new end time for the event, where the new start time and the new end time correspond to time slots of the same duration covered by the start time and the end time.

[0154] (I6) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing that includes a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and are configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods I1-I5. (I7) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs that include instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause the performance of any of the methods I1-I5, the computing system having a memory, a first housing that includes a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display different from the primary display.

[0155] (I8) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims I1-I5.

[0156] (I9) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display; a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display; and means for performing or causing performance of any of the methods of claims I1-I5.

[0157] (J1) According to some embodiments, in a computing system (e.g., system 100 or system 200, Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A keyboard 106) and a touch-sensitive secondary display

[0158] (For example, Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C The method includes detecting a new connection between the computing system and an external device other than the computing system (e.g., connecting a new monitor, connecting headphones via Bluetooth or a headphone jack, etc.). In response to detecting the new connection, the method includes displaying a plurality of affordances corresponding to functions available via the external device on the touch-sensitive secondary display.

[0159] Allowing a user to effectively utilize an external device through a touch-sensitive secondary display provides the user with a convenient way to access functionality that might otherwise be buried in menus. Providing a convenient way for the user to access functionality of an external device that might otherwise be buried in menus (and thereby reducing the number of inputs required to access functionality, thereby requiring less interaction to use the external device) enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on the touch-sensitive secondary display to perform a function of the external device), and further reduces power usage and extends the battery life of the device by enabling the user to interact with the external device more quickly and efficiently. In this way, the touch-sensitive secondary display also conveys information about the internal state of the device (by reflecting the connection status between the device and the external device and allowing the user to easily interact with the external device).

[0160] (J2) In some embodiments of the method of J1, the method includes: receiving, via the touch-sensitive secondary display, a selection of a first affordance corresponding to a first function available via the external device. In response to receiving the selection, the method includes: initiating execution of the first function.

[0161] (J3) In some embodiments of the method of any of J1-J2, the external device is an additional display that is different from the main display and the touch-sensitive display.

[0162] (J4) In some embodiments of the method of J3, the plurality of enable representations include a first enable representation that, when selected, causes the computing system to initiate execution of a display mirroring function through an additional display.

[0163] (J5) In another aspect, a computing system is provided, comprising one or more processors, a memory, a first housing comprising a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing or causing the performance of any of the methods in J1-J4. (J6) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause the performance of any of the methods in J1-J4, the computing system having a memory, a first housing comprising a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display different from the primary display.

[0164] (J7) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims J1-J4.

[0165] (J8) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims J1-J4.

[0166] (K1) According to some embodiments, in a computing system (e.g., system 100 or system 200, Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A keyboard 106) and a touch-sensitive secondary display

[0167] (For example, Figure 1AIn some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C The method includes: displaying a user interface of an application executed on the computing system on a primary display; and detecting a first input at a specific location within the user interface. In response to detecting the first input, the method includes:

[0168] A set of affordances, each corresponding to a different character, is displayed on the touch-sensitive secondary display.

[0169] Allowing a user to quickly and easily preview how characters will appear within an application on a primary display by providing intuitive input on a touch-sensitive secondary display provides a convenient way for the user to quickly preview how characters will appear within the application. Providing a convenient way for the user to quickly preview how characters will appear within the application enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on the touch-sensitive secondary display to quickly preview how characters will appear, thereby requiring less interaction to preview how those characters will appear on the primary display), and also reduces power usage and extends the battery life of the device by enabling the user to more quickly and efficiently preview characters. In this way, the user is enabled to interact effectively and continuously with their device because the user is allowed to continue previewing characters and then continue providing input until the desired character is located.

[0170] (K2) In some embodiments of the method of K1, the method includes: detecting, via the touch-sensitive secondary display, a second input on the first affordance corresponding to a first character among the different characters. In response to detecting the second input, the method includes: displaying a preview of the first character at a specific location on the primary display while the input remains in contact with the first affordance.

[0171] (K3) In some embodiments of the method of K2, the method includes: detecting, via the touch-sensitive secondary display, movement of a second input from a first affordance to a second affordance corresponding to a second character among different characters. In response to detecting movement of the second input from the first affordance to the second affordance, the method includes: replacing the preview of the first character with the preview of the second character.

[0172] (K4) In some embodiments of the method of any one of K1-K3, the method includes: determining an enable representation included in the set of enable representations based at least in part on text content included in the user interface.

[0173] (K5) In some embodiments of the method of K4, the determination is made in response to detecting that a user has modified textual content included in the user interface.

[0174] (K6) In some embodiments of the method of any of K1-K5, the method includes: detecting a lift-off of the second input while in contact with the second indicator. In response to detecting the lift-off, the method includes: updating the user interface to include the second user interface element.

[0175] (K7) In some embodiments of the method of any of K1-K5, the method includes: detecting an additional input while the second input is in contact with the second indicator. In response to detecting the additional input, the method includes: updating the user interface to include the second user interface element.

[0176] (K8) In some embodiments of the method of any of K1-K7, the method includes: as the second input continues to move across the touch-sensitive secondary display, while the second input contacts a corresponding affordance in the set of affordances, displaying a preview of the corresponding character in the different characters. (K9) In some embodiments of the method of any of K1-K8, the preview of the second character remains displayed on the primary display while the input remains in contact with the second affordance.

[0177] (K10) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods of K1-K9.

[0178] (K11) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods K1-K9, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0179] (K12) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods of claims K1-K9.

[0180] (K13) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims K1-K9.

[0181] (L1) According to some embodiments, in a computing system (e.g., system 100 or system 200, Figures 1A to 2D ), wherein the method is executed in a computing system including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102), and a computer system at least partially containing a physical keyboard (e.g., Figure 1A keyboard 106) and a touch-sensitive secondary display

[0182] (For example, Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C The method includes receiving a request to open a content editing application (an application for formatting and editing documents, drawings, photos, etc.). In response to receiving the request, the method includes: (i) displaying the content editing application on the primary display; and (ii) displaying a user interface control for modifying at least one visual feature (e.g., a color selector including a slider with a color value for selecting a color for content displayed in the content editing application) on the touch-sensitive secondary display. The at least one visual feature is used to present content within the content editing application.

[0183] Thus, a user is provided with an intuitive way to modify visual features used to present content in a content editing application on a primary display by providing input on a touch-sensitive secondary display. Providing a user with an intuitive way to modify visual features in this manner enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on the touch-sensitive secondary display to quickly preview how certain visual features will appear when used to present content on the primary display), and additionally reduces power usage and extends the battery life of the device by enabling the user to preview changes to visual features in a faster and more efficient manner. In this manner, the user is enabled to interact with their device efficiently and continuously because the user is allowed to continue to preview how modifications to the visual feature will appear on the primary display and then continue to provide input until the desired modification to the visual feature is located.

[0184] (L2) In some embodiments of the method of L1, the method includes: detecting input of a first value of at least one visual feature selected at a user interface control via a touch-sensitive secondary display.

[0185] After detecting the input, the method includes presenting content in the content editing application using the first value of the at least one visual feature (e.g., presenting all new content added to the content editing application using the first value and / or presenting currently selected content using the first value).

[0186] (L3) In some embodiments of the method of any of L1-L2, the user interface controls include respective controls that each correspond to a respective value of the at least one visual characteristic along a sliding scale of values.

[0187] (L4) In some embodiments of the method of L3, the sliding scale of values ​​represents different color shades.

[0188] (L5) In some embodiments of the method of L4, the first value corresponds to a first chromaticity of the first color, and the method includes: based on determining that the input meets a predetermined criterion (maintaining contact for more than a threshold amount of time), modifying the user interface controls on the touch-sensitive secondary display to include options for selecting other chromaticities of the first color that are different from the first chromaticity of the first color.

[0189] (L6) In some embodiments of the method of any of L1-L5, the method includes: before presenting the content, receiving a selection of the content, and presenting the content includes presenting a preview of the content using a first value of the at least one visual feature.

[0190] (L7) In some embodiments of the method of L6, the preview is presented while the input remains in contact with the touch-sensitive secondary display.

[0191] (L8) In some embodiments of the method of L7, the method includes: in response to detecting a lift-off of the input, stopping displaying the preview.

[0192] (L9) In some embodiments of the method of L8, the method includes: in response to detecting a liftoff of the input, displaying a portion of the editable content having a modified value of at least one visual characteristic.

[0193] (L10) In another aspect, a computing system is provided that includes one or more processors, a memory, a first housing including a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods described in L1-L9.

[0194] (L11) On the other hand, a non-transitory computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause the performance of any of the methods L1-L9, the computing system having a memory, a first housing including a primary display, and a second housing that at least partially contains a physical keyboard and a touch-sensitive secondary display distinct from the primary display.

[0195] (L12) On the other hand, a graphical user interface is provided on a computing system having one or more processors, a memory, a first housing including a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display different from the primary display, the graphical user interface including a user interface displayed according to any of the methods described in claims L1-L9.

[0196] (L13) In another aspect, a computing device is provided. The computing device includes: a first housing including a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims L1-L9.

[0197] (M1) According to some embodiments, in an electronic device (e.g., system 100 or system 200,

[0198] Figures 1A to 2D ), wherein the method is executed in an electronic device including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102, also referred to as a "display"), and a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard.

[0199] (For example, the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as Figure 2C In some embodiments, the electronic device further comprises a biometric sensor that is integrated with the touch-sensitive secondary display or is a separate component positioned adjacent to the touch-sensitive secondary display within the second housing. The method comprises: when the device is in a locked state

[0200] (For example, a locked state is a state in which one or more features of a device are disabled and access to sensitive information or the ability to change or delete information is prohibited), displaying a corresponding login user interface associated with logging into a plurality of user accounts including a first user account and a second user account. When displaying the login user interface, the method includes: receiving biometric information about a user. In response to receiving the biometric information, the method includes: (i) displaying a prompt on the display to enter login credentials for the first user account based on determining that the biometric information is consistent with biometric information of a first user account among the plurality of user accounts, while the first user account does not have an active session on the device; and (ii) displaying a prompt on the display to enter login credentials for the second user account based on determining that the biometric information is consistent with biometric information of a second user account among the plurality of user accounts, while the second user account does not have an active session on the device.

[0201] Thus, an intuitive way is provided for the user to access a user-specific login page by providing a single input on a biometric sensor. Providing an intuitive way for the user to access a user-specific login page in this manner enhances the operability of the computing system and makes the user device interface more efficient (e.g., by requiring a single input or gesture on a biometric sensor to quickly access the appropriate user-specific login page, thereby requiring less interaction to reach the user-specific login page), and further reduces power usage and extends the battery life of the device by enabling the user to access the login page with a single input.

[0202] (M2) In some embodiments of the method of M1, the method includes: in response to receiving biometric information: based on determining that the biometric information is consistent with the biometric information of a first user account among multiple user accounts, and the first user account has an active session on the device, unlocking the device relative to the first user account (for example, without requiring additional user input).

[0203] (M3) In some embodiments of the method of any one of M1-M2, the method includes: in response to receiving biometric information: based on determining that the biometric information is consistent with the biometric information of a second user account in multiple user accounts, and the second user account has an active session on the device, unlocking the device relative to the second user account (for example, without requiring additional user input).

[0204] (M4) In some embodiments of the method of any of M1-M3, the method includes: in response to receiving the biometric information: maintaining the device in a locked state based on determining that the biometric information is inconsistent with biometric information of any user account of the device. (M5) In some embodiments of the method of any of M1-M4, the login user interface includes a plurality of selectable affordances corresponding to the plurality of user accounts.

[0205] (M6) In some embodiments of the method of any one of M1-M5, the method includes: when displaying a prompt to enter login credentials for a first user account, receiving input of login credentials. In response to receiving the input of the login credentials, the method includes: (i) unlocking the device relative to the first user account based on determining that the login credentials are consistent with the login credentials of the first user account; and (ii) maintaining the device in a locked state based on determining that the login credentials are inconsistent with the login credentials of the first user account.

[0206] (M7) In some embodiments of the method of any one of M1-M6, the method includes: when displaying a prompt to enter login credentials for a second user account, receiving input of login credentials. In response to receiving the input of the login credentials: (i) unlocking the device relative to the first user account based on a determination that the login credentials are consistent with the login credentials of the second user account; and (ii) maintaining the device in a locked state based on a determination that the login credentials are inconsistent with the login credentials of the second user account.

[0207] (M8) In some embodiments of the method of any one of M1-M7, the login user interface includes instructions to provide biometric information.

[0208] (M9) In some embodiments of the method of any one of M1-M8, the device includes a secondary display (e.g., a touch-sensitive secondary display) adjacent to the biometric sensor, and the method includes displaying instructions on the secondary display to provide biometric information through the biometric sensor while displaying a login user interface on the display of the device.

[0209] (M10) In some embodiments of the method of any one of M1-M9, the biometric sensor is a fingerprint sensor.

[0210] (M11) In some embodiments of the method of any one of M1-M9, the biometric sensor is a facial detection sensor.

[0211] (M12) In some embodiments of the method of any one of M1-M9, the biometric sensor is a retinal scanner.

[0212] (M13) In another aspect, an electronic device is provided, comprising one or more processors, a memory, a display, and a biometric sensor. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods of M1-M12.

[0213] (M14) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods M1-M12, the computing system having a display and a biometric sensor.

[0214] (M15) On the other hand, a graphical user interface on a computing system is provided, the computing system having one or more processors, memory, display and biometric sensors, the graphical user interface comprising a user interface displayed according to any of the methods of claims M1-M12.

[0215] (M16) In another aspect, a computing device is provided, comprising a display and a biometric sensor, and means for performing or causing performance of any of the methods of claims M1-M12.

[0216] (N1) According to some embodiments, in an electronic device (e.g., system 100 or system 200,

[0217] Figures 1A to 2D ), wherein the method is executed in an electronic device including one or more processors, a memory, a first housing including a primary display (e.g., housing 110 including display 102 or housing 204 including display 102, also referred to as a "display"), and a physical keyboard (e.g., Figure 1A 106) and a touch-sensitive secondary display (e.g., Figure 1A In some embodiments, the touch-sensitive secondary display is separate from the physical keyboard (e.g., the touch-sensitive secondary display is a separate display 222, or the touch-sensitive display is coupled to another device such as a Figure 2C108 in the touch-sensitive secondary display). In some embodiments, the electronic device further comprises a biometric sensor that is integrated with the touch-sensitive secondary display or is a separate component positioned adjacent to the touch-sensitive secondary display within the second housing. The method comprises: when the device is logged into a first user account, displaying a user interface associated with the first user account (e.g., a home screen or desktop of the first user account). The device is associated with a plurality of user accounts including a first user account and a second user account, and the second user account is associated with biometric information capable of logging into the second user account. When displaying the user interface associated with the first user account, the method comprises: receiving input through an input element having an integrated biometric sensor. In response to receiving the input through an input element having an integrated biometric sensor, the method comprises: based on determining that the input satisfies a second user switching criterion while the second user account has an active session on the device, wherein the second user switching criterion comprises requiring that the biometric information detected during input using the input element is consistent with the biometric information of a second user account among the plurality of user accounts: (i) unlocking the device relative to the second user account;

[0218] (ii) locking the device relative to the first user account; and (iii) replacing display of the user interface associated with the first account with a user interface associated with the second user account.

[0219] Thus, an intuitive way is provided for the user to access an active user-specific session on the electronic device by providing a single input on a biometric sensor. Providing an intuitive way for the user to access an active user-specific session in this manner enhances the operability of the electronic device and makes the user-device interface more efficient (e.g., by requiring a single input or gesture on the biometric sensor to gain immediate access to the active session, thereby requiring less interaction to switch user accounts and log into the device), and further reduces power usage and extends the battery life of the device by enabling the user to access the active session with a single input.

[0220] (N2) In some embodiments of the method of N1, the method includes: in response to receiving biometric information: based on determining that the input meets a second user switching criterion and the second user account does not have an active session on the device, wherein the second user switching criterion includes requiring that the biometric information detected during input using the input element is consistent with the biometric information of the second user account among multiple user accounts, displaying a prompt on the display to enter the login credentials of the second user account.

[0221] (N3) In some embodiments of the method of any one of N1-N2, the method includes: in response to receiving biometric information: based on determining that the input satisfies a third user switching criterion while a third user account has an active session on the device, wherein the third user switching criterion includes requiring that the biometric information detected during input using the input element is consistent with the biometric information of a third user account among multiple user accounts: (i) unlocking the device relative to the third user account; (ii) locking the device relative to the first user account; and (iii) replacing the display of a user interface associated with the first account with a user interface associated with the third user account.

[0222] (N4) In some embodiments of the method of any one of N1-N3, the method includes: in response to receiving biometric information: based on determining that the input meets a third user switching criterion and the third user account does not have an active session on the device, wherein the third user switching criterion includes requiring that the biometric information detected during the input using the input element is consistent with the biometric information of the third user account in the plurality of user accounts, displaying a prompt on the display to enter login credentials for the third user account. (N5) In some embodiments of the method of any one of N1-N2, the input element is a button,

[0223] And input through an input element with an integrated biometric sensor includes detecting a press input through a button.

[0224] (N6) In some embodiments of the method of N5, the second user switching criterion includes a criterion that the press input continues for less than a first threshold amount of time; and the method includes: in response to receiving a press input performed through an input element with an integrated biometric sensor, placing the device in a lower power mode (for example, the low power mode corresponds to a pause state of the electronic device in which the display is turned off) based on determining that the press input performed using the button continues for longer than the first threshold amount of time.

[0225] (N7) In some embodiments of the method of N5, the second user switching criterion includes a criterion that the button is pressed for less than a first threshold amount of time; and the method includes: in response to receiving input performed through an input element with an integrated biometric sensor: (i) placing the device in a low power mode (for example, the low power mode corresponds to a sleep / suspend state of the electronic device in which the display is turned off, and the low power mode is entered when the press input ends) based on determining that the press input performed using the button lasts for longer than the first threshold amount of time and less than a second threshold amount of time; and (ii) displaying an options menu for changing the device state (for example, the options menu includes shutdown, restart, and sleep / suspend options, which, when selected, cause the device to shut down, restart, or sleep / suspend, respectively).

[0226] (N8) In some embodiments of the method of N7, the method includes: in response to receiving input through an input element having an integrated biometric sensor: restarting the device based on determining that the press input using the button lasts longer than a third threshold time amount that is greater than the second threshold time amount.

[0227] (N9) In some embodiments of the method of any one of N1-N8, the method includes: after replacing display of a user interface associated with the first account with a user interface associated with the second user account: receiving a second input through an input element with an integrated biometric sensor while the user interface associated with the second user account is displayed. In response to receiving the second input through the input element with an integrated biometric sensor: based on determining that the second input satisfies a first user switching criterion while the first user account has an active session on the device, wherein the first user switching criterion includes requiring that biometric information detected during input using the input element is consistent with biometric information of a first user account among a plurality of user accounts: (i) unlocking the device relative to the first user account; (ii) locking the device relative to the second user account; and (iii) replacing display of the user interface associated with the second account with the user interface associated with the first user account.

[0228] (N10) In another aspect, an electronic device is provided, comprising one or more processors, a memory, a display, and a biometric sensor. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing the performance of any of the methods described in N1-N9.

[0229] (N11) In another aspect, a non-transitory computer-readable storage medium is provided that stores one or more programs comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform or cause execution of any of the methods N1-N9, the computing system having a display and a biometric sensor.

[0230] (N12) On the other hand, a graphical user interface on a computing system is provided, the computing system having one or more processors, memory, display and biometric sensors, the graphical user interface including a user interface displayed according to any one of the methods of claims N1-N9.

[0231] (N13) In another aspect, a computing device is provided. The computing device includes a display and a biometric sensor, and means for performing or causing performance of any of the methods of claims N1-N9. BRIEF DESCRIPTION OF THE DRAWINGS

[0232] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals refer to corresponding parts throughout.

[0233] Figure 1A is an illustrative diagram of a portable computing system (e.g., a laptop computer) according to some embodiments.

[0234] Figure 1B According to some embodiments Figure 1A An illustration of the main body of the portable computing system in FIG.

[0235] Figure 2A is an illustrative diagram of a first implementation of a desktop computing system according to some embodiments.

[0236] Figure 2B is an illustrative diagram of a second implementation of a desktop computing system according to some embodiments.

[0237] Figure 2C is an illustrative diagram of a third implementation of a desktop computing system according to some embodiments.

[0238] Figure 2D is an illustrative diagram of a fourth implementation of a desktop computing system according to some embodiments.

[0239] Figure 3A is a block diagram of an electronic device according to some embodiments.

[0240] Figure 3B According to some embodiments, Figure 3A A block diagram of the components of event processing.

[0241] Figures 3C to 3E Examples of dynamic intensity thresholds are shown according to some embodiments.

[0242] Figure 4 is a block diagram of a peripheral electronic device according to some embodiments.

[0243] Figures 5A to 5E 、 Figures 6A to 6F 、 7A to 7D 、 Figures 8A to 8H 、 Figure 9 、 Figures 10A to 10H 、 Figures 11A to 11H 、 12A to 12H 、 13A to 13C 、 Figures 14A to 14E 、 Figures 15A to 15H 、 16A to 16T 、 Figures 17A to 17G 、 18A to 18I 、 Figures 19A to 19K 、 Figures 20A to 20G 、 Figures 21A to 21J 、 Figures 22A to 22C 、 FIG. 23A to FIG. 23B 、 Figures 24A to 24N 、 Figures 25A to 25C 、 Figures 26A to 26E 、 27A to 27O 、 Figures 28A to 28P 、 29A to 29T 、 Figures 30A to 30F 、 Figures 31A to 31B 、 Figures 32A to 32E 、 Figures 33A to 33K 、 Figures 34A to 34C 、 Figures 35A to 35B 、 Figures 36A to 36W 、 Figures 37A to 37M 、 Figures 38A to 38J 、 Figure 39 、 Figure 40 、 Figures 41A to 41E 、 Figures 42A to 42B as well as Figures 43A to 43D An exemplary user interface for dynamically providing user interface controls at a touch-sensitive secondary display according to some embodiments is shown.

[0244] Figures 44A to 44D is a flow chart of a method of updating dynamic input and output devices according to some embodiments.

[0245] Figures 45A to 45C is a flow chart of a method of updating dynamic input and output devices according to some embodiments.

[0246] Figures 46A to 46B is a flowchart of a method of maintaining functionality of an application in full screen mode according to some embodiments.

[0247] Figures 47A to 47B is a flowchart of a method for displaying notifications on a touch screen display according to some embodiments.

[0248] Figures 48A to 48C is a flowchart of a method of moving a user interface portion according to some embodiments.

[0249] Figure 49 is a flowchart of a method for navigating user interface objects on a primary display by providing input at a touch-sensitive secondary display according to some embodiments.

[0250] Figure 50 is a flowchart of a method for browsing search results on a primary display by providing input at a touch-sensitive secondary display according to some embodiments.

[0251] Figure 51 is a flowchart of a method for modifying details of an event displayed on a primary display by providing input at a touch-sensitive secondary display according to some embodiments.

[0252] Figure 52 is a flowchart of a method for presenting executable information about an external device at a touch-sensitive secondary display according to some embodiments, the external device being connected to a computing system including a touch-sensitive secondary display.

[0253] Figure 53 is a flowchart of a method for previewing characters displayed within an application on a primary display by providing input at a touch-sensitive secondary display according to some embodiments.

[0254] Figure 54 is a flowchart of a method for modifying visual characteristics used to present content within a content editing application on a primary display by providing input at a touch-sensitive secondary display according to some embodiments.

[0255] Figures 55 to 65 A functional block diagram of an electronic device according to some embodiments is shown.

[0256] Figure 66 is a flow chart of a method for implementing valid login using a biometric sensor, according to some embodiments.

[0257] Figure 67 is a flow chart of a method for enabling rapid switching between logged-in user accounts using a biometric sensor, according to some embodiments.

[0258] Figures 68 to 69 A functional block diagram of an electronic device according to some embodiments is shown.

[0259] Figures 70A to 70Bis a schematic diagram of a primary display and a touch-sensitive secondary display according to some embodiments, which are used to illustrate a user interface for dynamically providing controls at the touch-sensitive secondary display for use with a map application. Figures 70C to 70D is a schematic diagram of a touch-sensitive secondary display according to some embodiments, which is used to illustrate a user interface for dynamically providing controls at the touch-sensitive secondary display for use with a map application.

[0260] Figures 71A to 71B is a schematic diagram of a primary display and a touch-sensitive secondary display according to some embodiments, which are used to illustrate a user interface for dynamically providing controls at the touch-sensitive secondary display for use with a video editing application. Figures 71C to 71M is a schematic diagram of a touch-sensitive secondary display according to some embodiments, which is used to illustrate a user interface for dynamically providing controls at the touch-sensitive secondary display for use with a video editing application.

[0261] Figure 72A is a schematic diagram of a primary display and a touch-sensitive secondary display according to some embodiments, which are used to illustrate a user interface for dynamically providing controls for use with a video editing application. Figures 72B to 72K is a schematic diagram of a touch-sensitive secondary display according to some embodiments, which is used to illustrate a user interface for dynamically providing controls for use with a video editing application.

[0262] Figures 73A to 73N is a schematic diagram of a touch-sensitive secondary display according to some embodiments, which is used to illustrate a user interface for dynamically providing controls for use with a video editing application.

[0263] Figures 74A to 74H is a schematic diagram of a touch-sensitive secondary display according to some embodiments, which is used to illustrate a user interface for dynamically providing controls for use with a video editing application.

[0264] Figure 75 is a schematic diagram illustrating example glyphs that may be displayed at a touch-sensitive secondary display for use with various video editing applications according to some embodiments. DETAILED DESCRIPTION

[0265] Figure 1A to Figure 1B 、 Figures 2A to 2D 、 Figures 3A to 3E as well as Figure 4 A description of the example device is provided. Figures 5A to 14E An exemplary user interface for updating dynamic input and output devices is shown. Figures 44A to 44D is a flow chart of a method 600 of updating dynamic input and output devices. Figures 45A to 45C is a flow chart of a method 700 of updating dynamic input and output devices. Figures 46A to 46B is a flow chart of a method 800 of maintaining functionality of an application in full screen mode. Figures 47A to 47B is a flow chart of a method 900 of displaying notifications on a touch screen display. Figures 48A to 48C is a flow chart of a method 1000 of moving a user interface portion. Figures 5A to 14E The user interface in the Figures 44A to 44D 、 Figures 45A to 45C 、 Figures 46A to 46B 、 Figures 47A to 47B as well as Figures 48A to 48C The method and / or process in.

[0266] Figures 49 to 54 as well as Figures 66 to 67 A flow chart of a method for utilizing a touch-sensitive secondary display to implement efficient interaction at a computing system is included. Figures 15A to 43D The user interface in the Figures 49 to 54 as well as Figures 66 to 67 The method and / or process in.

[0267] Exemplary Devices and Systems

[0268] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks have not been described in detail in order to avoid unnecessarily obscuring aspects of the embodiments.

[0269] It will also be understood that, although in some cases, the terms "first," "second," etc. are used herein to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, a first contact can be named a second contact, and similarly, a second contact can be named a first contact without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact.

[0270] The terms used in the description of the various described embodiments herein are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms "a", "an", and "the" are intended to also include the plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and encompass any and all possible combinations of one or more items in the items listed in association. It will also be understood that the terms "includes", "including", "comprises", and / or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, elements, and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0271] As used herein, the term "if" is optionally interpreted to mean "when" or "upon," or "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that..." or "if [stated condition or event] is detected" are optionally interpreted to mean "upon determining that..." or "in response to determining that..." or "upon detecting [stated condition or event]," or "in response to detecting [stated condition or event]," depending on the context.

[0272] Figure 1A is an illustrative diagram of a portable computing system 100 according to some embodiments. The portable computing system 100 may be, for example, a laptop computer such as device or any other portable computing device. The portable computing system 100 includes: (A) a display portion 110 (also referred to herein as a first housing 110 or housing 110) having a main display 102 and (B) a main body portion 120 (also referred to as a second housing 120 or housing 120) having a dynamic function row 104, a set of physical (i.e., movably actuatable) keys 106, and a touchpad 108 partially contained within the same housing. The display portion 110 is typically mechanically, electrically, and communicatively coupled to the main body 120 of the portable computing system 100. For example, the portable computing system 100 may include a hinge that allows the display portion 110 to rotate relative to the main body portion 120. The portable computing system 100 includes one or more processors and a memory storing one or more programs executed by the one or more processors to perform any of the embodiments described herein. In some embodiments, reference Figure 1BThe dynamic function row 104 described in more detail is a touch screen display that detects user touch input and selections using resistive sensing, acoustic sensing, capacitive sensing, optical sensing, infrared sensing, etc. In some embodiments, the main display 102 of the display portion 110 is also a touch screen display.

[0273] Figure 1B is an illustrative diagram of a main body portion 120 of a portable computing system 100 according to some embodiments. The main body portion 120 includes a set of physical keys 106 (also referred to herein as "physical keys 106" and "keyboard 106"), a dynamic function row 104, and a touchpad 108, partially contained within the same housing. In some embodiments, the dynamic function row 104, which is a touch screen, replaces the function row of the set of physical keys 106, thereby allowing the space occupied by the set of physical keys 106 to be reduced, allowing the entire main body portion 120 to be smaller or allowing other portions, such as the touchpad 108, to be larger. In some embodiments, the length of the dynamic function row 104 relative to the major dimension of the set of physical keys 106 is approximately 18 inches. For ease of illustration, although the included Figure 1A The touch screen of the row of dynamic functions 104 in FIG. 1 is referred to as a "row," but in some other embodiments, the touch screen may take any other form, such as a square, a circle, multiple rows, a column, multiple columns, multiple separate sectors, etc. Figure 1A to Figure 1B A dynamic function row 104 is shown that replaces the function row of the set of physical keys 106, but in some other embodiments, the dynamic function row 104 may additionally and / or alternatively replace a numeric keypad portion, an editing / function portion, etc., of the set of physical keys 106.

[0274] Each physical key of the set of physical keys 106 has at least one associated input. The input can be a printable character, a non-printable character, a function, or other input. The input associated with a physical key can be displayed (e.g., printed) on the surface of the key by a letter, word, symbol, or other indicia in Latin, Arabic, Chinese, or any other alphabet. For example, the particular physical key indicated at 138 is associated with the alphabetic character "z," as indicated by the letter z displayed on the key. As another example, a physical key labeled "Command" may be associated with a command function. For example, the set of physical keys 106 is associated with a QWERTY, Dvorak, or other keyboard layout having alphanumeric, numeric, and / or editing / function portions (e.g., standard, extended, or compressed) in accordance with ISO / IEC 9995, ANSI-INCITS 154-1988, JIS X 6002-1980, or other similar standards.

[0275] When a user has activated a key, a signal corresponding to an input associated with the physical key may be processed by the processor (or Figures 2A to 2DThe computing device 202 or Figures 2A to 2B In the illustrated example, each key of the set of physical keys 106 includes two plates and a spring. A user may activate the key by pressing the key, which compresses the spring. When the spring is compressed, the two plates may contact, thereby allowing current to flow through the connected plates. In response to the current flowing through the connected plates, an input corresponding to the key may be provided to the processor. For example, in response to Figure 2C Upon activation of the set of keys 106 of the peripheral keyboard 206 in the computing device 202, input corresponding to the activated keys is provided to the computing device 202. It should be appreciated that other systems for movably actuated keys may be used.

[0276] In some embodiments, the dynamic function row 104 is a touch screen display (also referred to herein as a "touch-sensitive secondary display") that displays one or more user-selectable symbols 142 (also sometimes referred to herein as "user interface elements," "user interface components," "indicators," "buttons," or "soft keys"). For example, the dynamic function row 104 replaces the function row keys on a typical keyboard. A user can select a particular one of the one or more user-selectable symbols 142 by touching a location on the touch screen display corresponding to the particular one of the one or more user-selectable symbols 142. For example, a user can select a user-selectable symbol indicated by the magnifying glass symbol 144 by tapping the dynamic function row 104 so that the user's finger contacts the dynamic function row 104 at the location of the magnifying glass indicator 214. In some embodiments, the tap contact or tap gesture includes touching down and lifting off the contact within a predetermined amount of time (e.g., 250ms, etc.). In some embodiments, the touch screen display of the dynamic function row 104 is implemented using resistive sensing, acoustic sensing, capacitive sensing, optical sensing, infrared sensing, etc. to detect user touch input and selection.

[0277] When the user selects a particular one of the one or more user-selectable symbols 142, a signal corresponding to the particular one of the one or more user-selectable symbols 142 is generated by the dynamic function row 104. For example, when the user taps "esc" on the dynamic function row 104, the dynamic function row 104 sends a signal indicating the user input corresponding to the escape function to the processor (or processors) of the portable computing system 100. Figures 2A to 2D computing device 202, or Figures 2A to 2B the processor of the peripheral keyboard 206, or Figure 2C a processor of the first input mechanism 212, or Figure 2D a processor of the peripheral input mechanism 222 in the processor).

[0278] In some embodiments, when a particular one of the one or more user-selectable symbols 142 is selected, the dynamic function row 104 sends a signal corresponding to the location on the touch screen display where the particular one of the one or more user-selectable symbols 142 is displayed to the processor (or processor) of the portable computing system 100. Figures 2A to 2D computing device 202, or Figures 2A to 2B the processor of the peripheral keyboard 206, or Figure 2C a processor of the first input mechanism 212, or Figure 2D For example, the dynamic function row 104 may send a signal including a position value (0 to 20) based on the position of a particular one of the one or more user-selectable symbols 142 selected on the touch screen display. Figure 1B In the illustrative example of , the "esc" symbol may have a position value of 0, the magnifying glass symbol 144 may have a position value of 16, etc. The processor (or Figures 2A to 2D computing device 202, or Figures 2A to 2B the processor of the peripheral keyboard 206, or Figure 2C a processor of the first input mechanism 212, or Figure 2D The processor of the peripheral input mechanism 222 in the display portion 110 may receive a signal indicating the position value of the selected user-selectable symbol and use context information (such as the main display 102 (or Figures 2A to 2D The position value is interpreted by referring to the currently active or focused graphical user interface element displayed on the peripheral display device 204 in the image.

[0279] Each of the one or more user-selectable symbols 142 may include an indicator (such as a symbol, e.g., a magnifying glass symbol shown at 144), an abbreviation (e.g., "esc"), a non-abbreviated word, a character, an image, an animated image, a video, etc. In some embodiments, a corresponding one of the one or more user-selectable symbols 142 is capable of receiving user input.

[0280] An input may be associated with each of the one or more user-selectable symbols 142. The input may be a function, a character, a numerical value, etc. A corresponding one of the one or more user-selectable symbols 142 may include an indicator of the input corresponding to the corresponding one of the one or more user-selectable symbols 142. For example, in Figure 1B, the user-selectable symbol with the abbreviation "esc" indicates to the user that an escaping function is associated with the user-selectable symbol. When the user selects the user-selectable symbol, the function associated with one or more user-selectable symbols 142 may be activated. For example, when the user selects the user-selectable symbol with the indicator "esc", the escaping function may be activated. Figures 2A to 2D 202 in the computing device 202), activation of the function may have different effects. For example, when the dialog box is on the display portion 110 (or Figures 2A to 2D When the dialog box is opened on the primary display 102 of the portable computing system 100 (or the peripheral display device 204), activating the exit function on the dynamic function row 104 can close the dialog box. For another example, when the game application is being executed by the processor (or Figures 2A to 2D Activating the exit function on the dynamic function row 104 pauses the game when executed on the computing device 202 in the game.

[0281] In some embodiments, functions may be associated with combinations of movably actuatable keys and / or user-selectable symbols. For example, simultaneously actuating the command key and the "c" key (i.e., command+c) may be associated with a "copy" function. For another example, actuating the command key and simultaneously selecting the user-selectable symbol with the indicator "esc" (i.e., command+esc) may activate a function that opens a specific application (such as a media player application). For another example, simultaneously selecting two user-selectable symbols (e.g., a user-selectable symbol with the indicator "esc" and a user-selectable symbol 144 with a magnifying glass indicator) may result in the activation of a function (such as a specialized search function).

[0282] In some embodiments, a first subset 146 of one or more user-selectable symbols 142 of the dynamic function row 104 may be associated with one set of functions, and a second subset 148 of one or more user-selectable symbols 142 of the dynamic function row 104 may be associated with a second set of functions. For example, the user-selectable symbols in the first subset 146 may be overall functions (e.g., system-level functions or affordances), and the user-selectable symbols in the second subset 148 may be application-specific functions. Thus, when the focus shifts from a first element of a graphical user interface displayed on the main display 102 (e.g., a first window corresponding to an internet browser application) to a second element of the graphical user interface (e.g., a second window corresponding to an email application), the user-selectable symbols in the second subset 148 change. In contrast, when the focus shifts from the first element of the graphical user interface to the second element of the graphical user interface, the user-selectable symbols in the first subset 146 remain unchanged.

[0283] In some embodiments, the user-selectable symbols in the second subset 148 are determined based on the active user interface element display that is focused on the primary display 102. In some embodiments, the term "focused" may refer to an active element of a user interface (e.g., a window associated with an application, a particular toolbar or menu associated with an application, or the operating system) that is currently actively running in the foreground or that can be controlled by input received from a user of the computing system (such as a key press, a mouse click, a voice command, a gesture motion, etc.).

[0284] In some embodiments, a first subset 146 of one or more user-selectable symbols 142 corresponding to the overall user-selectable symbols occupies a first area of ​​the dynamic function row 104 (e.g., the left half of the dynamic function row 104), and a second subset 148 of one or more user-selectable symbols 142 occupies a second area of ​​the dynamic function row 104 (e.g., the right half of the dynamic function row 104). It should be appreciated that the dynamic function row 104 may be allocated to the first subset 146 and the second subset 148 in other proportions. In some embodiments, when no application has focus, the second area of ​​the dynamic function row 104 may not include any user-selectable symbols. In some embodiments, the dynamic function row 104 includes three or more subsets of user-selectable symbols. In some embodiments, the dynamic function row 104 includes a single set of user-selectable symbols that are not divided into subsets. Although in Figure 1B A single row of user-selectable symbols is shown in the dynamic function row 104 in FIG, but it should be recognized that the dynamic function row 104 may include multiple rows of user-selectable symbols.

[0285] In some embodiments, a change in focus may change the focus on display portion 110 (or Figures 2A to 2D The user input may be received from a keyboard, mouse, trackpad, or other user input device. Additionally and / or alternatively, in some embodiments, a change in focus may change the focus displayed on the display portion 110 (or peripheral display device 204). Figures 2A to 2D Elements displayed in the foreground of a graphical user interface displayed on the primary display 102 of the peripheral display device 204 .

[0286] In some embodiments, in response to user input, for example, in response to a user selecting to display a Figures 2A to 2DThe focus changes in response to an element of a graphical user interface (e.g., a different window) displayed on primary display 102 (or peripheral display device 204) or in response to a user selecting a user-selectable symbol (e.g., one of the affordances / symbols displayed on dynamic function row 104). The user selection can be a keystroke, a mouse click, a mouse hover, a command+tab input, etc. In some embodiments, in response to an operating system (or Figures 2A to 2D The focus changes based on the determination of the computing device 202 in the operating system. For example, when a user closes the application window with focus, the operating system may focus on a different application, such as the application that had focus before the closed application window. For another example, when a user closes the application window with focus, the operating system may focus on a dialog box prompting the user to save changes made to the document through the application.

[0287] In some embodiments, the focus change can be a change from an element associated with an application to another element associated with the same application (e.g., a change from an email composition window of an email application to an inbox list window of an email application, or a change from one tab of an internet browser application to another tab of an internet browser application). In some embodiments, the focus change can be a change from an element associated with one application to an element associated with another application (e.g., a change from an internet browser window to an email application window). In addition, in some embodiments, the focus change can be a change from an element associated with an application to an element associated with an operating system (such as a system dialog box, a system settings control (e.g., a volume control), a window associated with a file / folder navigation application (e.g., Apple's FINDER application)). In addition, the focus can also be directed to a dialog box, a file directory, a settings control (e.g., a volume control), or any other element in a graphical user interface that can present its information to the user and / or can receive user input for its information.

[0288] Figure 2A is an illustrative diagram of a first specific implementation of a desktop computing system 200 according to some embodiments. The desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a primary display 102, a peripheral keyboard 206, and a peripheral mouse 208. The computing device 202 includes one or more processors and a memory storing one or more programs executed by the one or more processors. In some embodiments, the peripheral display device 204 can be connected to the computing device 202, such as Device integration. In some embodiments, the primary display 102 of the peripheral display device 204 is a touch screen display. Figure 2AIn the embodiment, a peripheral display device 204 (also referred to herein as a first housing 204 or housing 204), a peripheral keyboard 206, and a peripheral mouse 208 are communicatively coupled to the computing device 202 via a wired connection (such as USB or PS / 2) or via a wireless communication link using a communication protocol (such as Bluetooth, Wi-Fi, etc.). For example, the peripheral keyboard 206 (also referred to herein as a second housing 206 or housing 206) is no more than fifteen feet away from the computing device 202 (e.g., approximately three feet away). Figure 2A In some embodiments, the peripheral keyboard 206 includes a dynamic function row 104 and a set of physical keys 106 at least partially contained within the same housing. Figure 1B ) is a touch screen display. In some embodiments, the peripheral keyboard 206 includes one or more processors and a memory storing one or more programs executable by the one or more processors of the peripheral keyboard 206 to perform any of the embodiments described herein. In some embodiments, the peripheral keyboard 206 relays signals indicative of user input (e.g., keystrokes and selections represented by user-selectable symbols / affordances displayed by the dynamic function row 104) to the computing device 202.

[0289] Figure 2B is an illustrative diagram of a second specific implementation of a desktop computing system 200 according to some embodiments. Figure 2B In FIG, a desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a main display 102, and a peripheral keyboard 206. Figure 2B In , the peripheral display device 204 and the peripheral keyboard 206 are communicatively coupled to the computing device 202 via a wired connection (such as USB or PS / 2) or via a wireless communication link using a communication protocol (such as Bluetooth, Wi-Fi, etc.). Figure 2B In some embodiments, the peripheral keyboard 206 includes a dynamic function row 104, a set of physical keys 106, and a touchpad 108 at least partially contained within the same housing. Figure 1B 108 ) is a touch screen display. In some embodiments, the peripheral keyboard 206 includes one or more processors and a memory storing one or more programs executable by the one or more processors of the peripheral keyboard 206 to perform any of the embodiments described herein. In some embodiments, the peripheral keyboard 206 relays signals indicative of user input (e.g., keystrokes, user interactions with the trackpad 108, and selections represented by user-selectable symbols / affordances displayed by the dynamic function row 104) to the computing device 202.

[0290] Figure 2Cis an illustrative diagram of a third implementation of a desktop computing system 200 according to some embodiments. Figure 2C In FIG, a desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a primary display 102, a peripheral keyboard 206, and a first peripheral input mechanism 212. Figure 2C In FIG, the peripheral display device 204, the peripheral keyboard 206, and the first peripheral input mechanism 212 are communicatively coupled to the computing device 202 via a wired connection (such as USB or PS / 2) or via a wireless communication link using a communication protocol (such as Bluetooth, Wi-Fi, etc.). Figure 2C In some embodiments, the peripheral keyboard 206 includes a set of physical keys 106, and the first peripheral input mechanism 212 includes a dynamic function row 104 and a touchpad 108 at least partially contained within the same housing. Figure 1B In some embodiments, the first peripheral input mechanism 212 includes one or more processors and a memory storing one or more programs executable by the one or more processors of the first peripheral input mechanism 212 to perform any of the embodiments described herein. In some embodiments, the first peripheral input mechanism 212 relays signals indicating user input (e.g., user interaction with the trackpad 108 and user selection of user-selectable symbols / indications displayed by the dynamic function row 104) to the computing device 202.

[0291] Figure 2D is an illustrative diagram of a fourth implementation of a desktop computing system 200 according to some embodiments. Figure 2D In FIG, a desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a primary display 102, a peripheral keyboard 206, a peripheral mouse 208, and a second peripheral input mechanism 222. Figure 2D In the embodiment, the peripheral display device 204, the peripheral keyboard 206, the peripheral mouse 208, and the second peripheral input mechanism 222 are communicatively coupled to the computing device 202 via a wired connection (such as USB or PS / 2) or via a wireless communication link using a communication protocol (such as Bluetooth, Wi-Fi, etc.). Figure 2A In FIG, the peripheral keyboard 206 includes a dynamic function row 104 and a set of physical keys 106. Figure 2D In some embodiments, the peripheral keyboard 206 includes a set of physical keys 106, and the second peripheral input mechanism 222 includes a dynamic function row 104 at least partially contained within a housing of the second peripheral input mechanism 222. Figure 1BIn some embodiments, the second peripheral input mechanism 222 includes one or more processors and a memory storing one or more programs executable by the one or more processors of the second peripheral input mechanism 222 to perform any of the embodiments described herein. In some embodiments, the second peripheral input mechanism 222 relays a signal indicative of a user input (e.g., a user selection represented by a user-selectable symbol / indicator displayed by the dynamic function row 104) to the computing device 202.

[0292] Figure 3A is a block diagram of an electronic device 300 according to some embodiments. In some embodiments, the electronic device 300 is a portable electronic device, such as a laptop computer (e.g., Figure 1A In some embodiments, the electronic device 300 is not a portable device, but a desktop computer (e.g., Figures 2A to 2D 202 of the desktop computing system 200) that is connected to a peripheral display system (e.g., Figures 2A to 2D ) and an optional peripheral touch-sensitive surface (e.g., Figures 2B to 2C The touch pad 108 and / or touch-sensitive display (such as Figures 2A to 2D The peripheral display device 204 and / or Figures 2A to 2D The dynamic function row 104)) in is communicatively coupled.

[0293] The electronic device 300 typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a video conferencing application, an email application, an instant messaging application, an image management application, a digital camera application, a digital video camera application, a web browser application, and / or a media player application.

[0294] Various applications executed on the electronic device 300 optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed by the electronic device 300 optionally adapt and / or vary from one application to the next and / or within an application. In this way, the common physical architecture of the electronic device 300 (such as the touch-sensitive surface) optionally supports the various applications with a user interface that is intuitive and clear to the user.

[0295] The electronic device 300 includes a memory 302 (which optionally includes one or more computer-readable storage media), a memory controller 322, one or more processing units (CPUs) 320, a peripheral device interface 318, an RF circuit 308, an audio circuit 310, a speaker 311, a microphone 313, an input / output (I / O) subsystem 306, other input or control devices 316, and an external port 324. The electronic device 300 optionally includes a display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B The electronic device 300 optionally includes one or more optical sensors 364. The electronic device 300 optionally includes one or more intensity sensors 365 for detecting the intensity of contacts on a touch-sensitive surface, such as a touch-sensitive display or a touchpad. The electronic device 300 optionally includes one or more intensity sensors 365 for detecting the intensity of contacts on a touch-sensitive surface, such as a touch-sensitive display or a touchpad (e.g., Figure 1A to Figure 1B One or more tactile output generators 367 that generate tactile output on the touch panel 108 in . These components optionally communicate via one or more communication buses or signal lines 303.

[0296] As used in this specification, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or to a surrogate (surrogate) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds (e.g., at least 256) different values. The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, the force measurements from multiple force sensors are combined (e.g., weighted averaged) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measurement). In some implementations, the surrogate measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in pressure units).

[0297] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device that will be detected by a user using the user's sense of touch. For example, when a device or a component of the device is in contact with a surface that is touch-sensitive to a user (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" on a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click," even when the physical actuation button associated with the touch-sensitive surface that was physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the touch-sensitive surface does not change, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. While such interpretation of touch by the user will be limited by the user's individualized sensory perceptions, there are many sensory perceptions of touch that are common to most users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., "press click," "release click," "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or a component thereof that would generate that sensory perception for a typical (or average) user.

[0298] It should be understood that electronic device 300 is merely one example, and electronic device 300 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 3A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing and / or application specific integrated circuits.

[0299] Memory 302 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices. The access of other components of electronic device 300 (such as, CPU 320 and peripheral device interface 318) to memory 302 is optionally controlled by memory controller 322. Peripheral device interface 318 can be used for coupling input peripheral devices and output peripheral devices to CPU 320 and memory 302. One or more processing units 320 run or execute various software programs and / or instruction sets stored in memory 302 to perform various functions of electronic device 300 and process data. In some embodiments, peripheral device interface 318, CPU 320 and memory controller 322 are optionally implemented on a single chip (such as chip 304). In some other embodiments, they are optionally implemented on a separate chip.

[0300] RF (radio frequency) circuitry 308 receives and transmits RF signals, also known as electromagnetic signals. RF circuitry 308 converts electrical signals into / from electromagnetic signals and communicates with communication networks and other communication devices via the electromagnetic signals. RF circuitry 308 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. RF circuitry 308 optionally communicates with networks and other devices via wireless communications, such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). Wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11d), 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilization Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol, including those not yet developed as of the filing date of this document.

[0301] The audio circuit 310, the speaker 311, and the microphone 313 provide an audio interface between the user and the electronic device 300. The audio circuit 310 receives audio data from the peripheral device interface 318, converts the audio data into electrical signals, and sends the electrical signals to the speaker 311. The speaker 311 converts the electrical signals into sound waves audible to humans. The audio circuit 310 also receives electrical signals converted from sound waves by the microphone 313. The audio circuit 310 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 318 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 302 and / or the RF circuit 308 by the peripheral device interface 318. In some embodiments, the audio circuit 310 also includes a headset jack. The headset jack provides an interface between the audio circuit 310 and a removable audio input / output peripheral device (such as an output-only headset or a headset with both output (e.g., a single-ear or dual-ear headset) and input (e.g., a microphone)).

[0302] The I / O subsystem 306 couples the input / output peripherals of the electronic device 300 (such as the display system 312) and other input or control devices 316 to the peripherals interface 318. The I / O subsystem 306 optionally includes a display controller 356, an optical sensor controller 358, an intensity sensor controller 359, a tactile feedback controller 361, and one or more other input controllers 360 for other input or control devices. The one or more other input controllers 360 receive / send electrical signals from / to other input or control devices 316. Other input or control devices 316 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, and the like. In some alternative embodiments, the other input controllers 360 are optionally coupled to (or not coupled to) any of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more physical buttons optionally include up / down buttons for volume control of speaker 311 and / or microphone 313 .

[0303] Display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1BThe dynamic function row 104 in the electronic device 300 provides an output interface between the electronic device 300 and the user (optionally providing an input interface when the display system is a touch-sensitive display). The display controller 356 receives electrical signals from the display system 312 and / or sends electrical signals to the display system 312. The display system 312 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects / elements.

[0304] In some embodiments, the display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B The dynamic function row 104 in the display system 312 is a touch-sensitive display having a touch-sensitive surface, sensor, or set of sensors that receives input from a user based on tactile contact. Thus, the display system 312 and the display controller 356 (together with any associated modules and / or instruction sets in the memory 302) detect contact on the display system 312 (and any movement or interruption of that contact) and convert the detected contact into an interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the display system 312. In one exemplary embodiment, the point of contact between the display system 312 and the user corresponds to the area under the user's finger.

[0305] Display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B The dynamic function rows 104 in the display system 312 optionally use LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, LED (light emitting diode) technology, or OLED (organic light emitting diode) technology, although other display technologies may be used in other embodiments. In some embodiments, when the display system 312 is a touch-sensitive display, the display system 312 and the display controller 356 optionally use any of a variety of touch sensing technologies now known or later developed (including but not limited to capacitive technology, resistive technology, infrared technology, and surface acoustic wave technology) and other proximity sensor arrays or other elements for determining one or more contact points with the display system 312 to detect contact and any movement or interruption thereof. In one exemplary embodiment, projected mutual capacitance sensing technology is used, such as the one from Apple Inc. (Cupertino, California). and Technology found in.

[0306] Display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B The dynamic function row 104 in the image processing unit 310 optionally has a video resolution of more than 400 dpi (e.g., 500 dpi, 800 dpi or greater). In some embodiments, the display system 312 is a touch-sensitive display, and the user optionally uses a stylus, finger, etc. to contact the touch-sensitive display. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures. In some embodiments, the electronic device 300 converts rough finger-based input into precise pointer / cursor positions or commands for performing the actions desired by the user.

[0307] In some embodiments, in addition to the display system 312, the electronic device 300 optionally includes a touchpad (e.g., Figure 1A to Figure 1B In some embodiments, the touchpad is a touch-sensitive area of ​​the electronic device 300 that is distinct from the display system 312 and does not display visual output. In some embodiments, when the display system 312 is a touch-sensitive display, the touchpad is optionally a touch-sensitive surface separate from the display system 312 or an extension of the touch-sensitive surface formed by the display system 312.

[0308] The electronic device 300 also includes a power system 362 for powering the various components. The power system 362 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC), etc.), a recharging system, power fault detection circuitry, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0309] The electronic device 300 optionally also includes one or more optical sensors 364 coupled to the optical sensor controller 358 in the I / O subsystem 306. The optical sensor 364 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 364 receives light projected from the environment through one or more lenses and converts the light into data representing an image. In conjunction with the imaging module 343, the optical sensor 364 optionally captures still images or video. In some embodiments, the optical sensor is located on the front of the electronic device 300, so that the user can optionally obtain an image of the user for video conferencing while viewing other video conference participants on the display system 312.

[0310] The electronic device 300 optionally also includes one or more contact force sensors 365 coupled to the force sensor controller 359 in the I / O subsystem 306. The contact force sensors 365 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). The contact force sensors 365 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is coupled to a touch-sensitive surface (e.g., Figure 1A to Figure 1B The touch panel 108, or the display system 312 (when it is a touch-sensitive display)) is arranged in parallel or adjacent to each other.

[0311] The electronic device 300 optionally also includes one or more tactile output generators 367 coupled to the tactile feedback controller 361 in the I / O subsystem 306. The tactile output generators 367 optionally include one or more electroacoustic devices, such as speakers or other audio components; and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). The contact force sensor 365 receives tactile feedback generation instructions from the tactile feedback module 333 and generates tactile outputs that can be sensed by a user of the electronic device 300. In some embodiments, at least one tactile output generator is coupled to a touch-sensitive surface (e.g., Figure 1A to Figure 1B The touch-sensitive surface is arranged in parallel or adjacent to the touch pad 108, or the display system 312 (when it is a touch-sensitive display), and tactile output is optionally generated by moving the touch-sensitive surface vertically (e.g., inward / outward to the surface of the electronic device 300) or laterally (e.g., back and forth in the same plane as the surface of the electronic device 300).

[0312] The electronic device 300 optionally further includes one or more proximity sensors 366 coupled to the peripherals interface 318. Alternatively, the proximity sensors 366 are coupled to other input controllers 360 in the I / O subsystem 306. The electronic device 300 optionally further includes one or more accelerometers 368 coupled to the peripherals interface 318. Alternatively, the accelerometers 368 are coupled to other input controllers 360 in the I / O subsystem 306.

[0313] In some embodiments, the software components stored in memory 302 include an operating system 326, a communication module 328 (or instruction set), a contact / motion module 330 (or instruction set), a graphics module 332 (or instruction set), an application 340 (or instruction set), and a dynamic function row module 350 (or instruction set). In addition, in some embodiments, memory 302 stores device / global internal state 357 (or instruction set), such as Figure 3A Device / global internal state 357 includes one or more of the following: an active application state indicating which applications (if any) are currently active and / or focused; a display state indicating which applications, views, or other information occupy the display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B Dynamic function row 104 in the display system) and / or peripheral display system (e.g., Figures 2A to 2D The primary display 102 and / or the peripheral display device 204 in Figures 2A to 2D sensor state, including information obtained from various sensors and input or control devices 316 of the electronic device 300; and position information about the position and / or posture of the electronic device 300.

[0314] The operating system 326 (e.g., DARWIN, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VXWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware components and software components.

[0315] The communication module 328 facilitates communication with other devices (e.g., Figures 2A to 2D The computing device 202, Figure 2A and Figure 2D Peripheral mouse 208, Figures 2A to 2B The peripheral keyboard 206, Figure 2C The first peripheral input mechanism 212 and / or Figure 2D The second peripheral input mechanism 222 in the embodiment of the present invention is used to communicate with the user and also includes various software components for sending / receiving data via the RF circuit 308 and / or the external port 324. The external port 324 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port 324 is connected to the external device. A multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used on the device.

[0316] The contact / motion module 330 optionally detects contact with the display system 312 (when it is a touch-sensitive display) (in conjunction with the display controller 356) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 330 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger lift event or contact interruption). The contact / motion module 330 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact) or multiple points of contact simultaneously (e.g., "multi-touch" / multiple finger contact). In some embodiments, the contact / motion module 330 also detects the touch pad (e.g., Figure 1A to Figure 1B Contact on the touchpad 108).

[0317] In some embodiments, the contact / motion module 330 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has made a selection or "clicked" on an enable representation). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and are adjustable without changing the physical hardware of the electronic device 300). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once by making a system-level click on the "intensity" parameter).

[0318] The contact / motion module 330 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, times, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap contact includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). For another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event and then detecting one or more finger drag events, and in some embodiments, subsequently detecting a finger lift (lift-off) event.

[0319] The graphics module 332 includes a graphics module for displaying the main display 102 (e.g., Figure 1A The main display 102 or the display portion 110 in Figures 2A to 2D Various known software components for presenting and displaying graphics on the main display 102 of the peripheral display device 204 in the peripheral display device 204 or other displays, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc. In some embodiments, the graphics module 332 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 332 receives one or more codes specifying the graphics to be displayed from an application, etc., and also receives coordinate data and other graphics attribute data if necessary, and then generates screen image data for output to the display controller 356.

[0320] The tactile feedback module 333 includes various software components for generating instructions used by the tactile output generator 367 to produce tactile output at one or more locations on the electronic device 300 in response to user interaction with the electronic device 300 .

[0321] Application 340 optionally includes the following modules (or instruction sets), or a subset or superset thereof:

[0322] Email client module 341 for receiving, sending, composing and viewing emails

[0323] (also sometimes referred to herein as a "mail application" or "email application");

[0324] An imaging module 342 for capturing still images and / or video images;

[0325] An image management module 343 (also sometimes referred to herein as a "photo application") for editing and viewing still images and / or video images;

[0326] A media player module 344 (also sometimes referred to herein as a "media player application") for playing back audio and / or video; and

[0327] A web browsing module 345 (also referred to herein as

[0328] "web browser").

[0329] Examples of other applications 340 optionally stored in memory 302 include messaging and communication applications, word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption applications, digital rights management applications, speech recognition applications, and speech replication applications.

[0330] In combination with RF circuit 308, display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B The email client module 341 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions, including one or more of the dynamic functionality row 104 in the display controller 356, the contact module 330, and the graphics module 332. In conjunction with the image management module 343, the email client module 341 makes it very easy to create and send emails with still images or video images captured by the imaging module 342.

[0331] In conjunction with display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B One or more of the dynamic function row 104), the display controller 356, the optical sensor 364, the optical sensor controller 358, the contact module 330, the graphics module 332 and the image management module 343, the imaging module 342 includes executable instructions for capturing still images or videos (including video streams) and storing them in the memory 302, modifying the characteristics of the still images or videos, or deleting the still images or videos from the memory 302.

[0332] In conjunction with display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B104), one or more of the display controller 356, the contact module 330, the graphics module 332, and the imaging module 342, the image management module 343 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, marking, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.

[0333] In conjunction with display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B 104), one or more of the display controller 356, the contact module 330, the graphics module 332, the audio circuit 310, the speaker 311, the RF circuit 308, and the web browsing module 345, the media player module 344 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as, MP3 or AAC files), and for displaying, presenting, or otherwise playing back video (e.g., on a computer connected via the external port 324). Figure 1A The main display 102 or the display portion 110 in Figures 2A to 2B Executable instructions for the peripheral display device 204 in the main display 102).

[0334] In combination with RF circuit 308, display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B The web browsing module 345 includes one or more of the dynamic function row 104 in the display controller 356, the contact module 330 and the graphics module 332, and the web browsing module 345 includes executable instructions for browsing the Internet according to user instructions (including searching, linking, receiving and displaying web pages or parts thereof, as well as attachments and other files linked to web pages).

[0335] The dynamic function row (DFR) module 350 includes a focus determination module 351, a DFR determination module 352, and a DFR rendering module 353. In some embodiments, the focus determination module 351 is configured to determine the focus of the display system 312 (e.g., Figure 1A The main display 102 of the display portion 110) or a peripheral display system (e.g., Figures 2A to 2D In some embodiments, the DFR determination module 352 is configured to determine a graphic (e.g., a set of one or more affordances) based on the focused active user interface element. In some embodiments, the DFR rendering module 353 is configured to display the focused active user interface element on the display system 312 (e.g., the peripheral display device 204). Figure 1A to Figure 1BThe DFR rendering module 353 includes a module for rendering the graphics determined by the DFR determination module 352 on the dynamic function row 104 in the display system 312 (e.g., Figure 1A to Figure 1B Various known software components for presenting and displaying graphics on the dynamic function row 104 in the dynamic function row 104, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as, user interface objects including soft keys), digital images, videos, animations, etc. In some embodiments, the DFR module 350 includes other modules for the following operations: adjusting the sensitivity of the dynamic function row 104; adjusting the audible and / or tactile feedback provided by the dynamic function row 104; adjusting the settings of the enable representations and information displayed by the dynamic function row 104 (e.g., size, brightness, font, language, etc.); adjusting the current power mode (e.g., normal mode and low power mode) of the dynamic function row 104; and so on.

[0336] In some embodiments, the dynamic feature row module 350 interacts with components that enable the provision of predicted / proactive / suggested content items, including predicted recipients, suggested text completion strings, proactively suggested applications, etc. Proactively suggested content items are discussed in more detail in U.S. application serial number 15 / 167,713, the entire contents of which are hereby incorporated by reference.

[0337] Each of the above modules and applications corresponds to an executable instruction set for performing one or more functions and the methods described in the application (e.g., computer-implemented methods and other information processing methods as described herein). These modules (i.e., instruction sets) need not be implemented with independent software programs, processes, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 302 optionally stores a subset of the above modules and data structures. In addition, memory 302 optionally stores additional modules and data structures not described above.

[0338] Figure 3B According to some embodiments, Figure 3A In some embodiments, the memory 302 ( Figure 3A ) includes an event classifier 370 (e.g., in the operating system 326) and an application 340-1 (e.g., any of the above-mentioned applications 341, 342, 343, 344 or 345).

[0339] The event classifier 370 receives event information and determines the application 340-1 and the application view 391 of the application 340-1 to which the event information is to be delivered. The event classifier 370 includes an event monitor 371 and an event dispatcher module 374. In some embodiments, the application 340-1 includes an application internal state 392 that indicates the display system 312 (e.g., when the application is active or being executed). Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B In some embodiments, the device / global internal state 357 is used by the event classifier 370 to determine which application(s) are currently active or focused, and the application internal state 392 is used by the event classifier 370 to determine the application view 391 to which the event information is to be delivered.

[0340] In some embodiments, the application internal state 392 includes additional information, such as one or more of the following: resumption information to be used when the application 340-1 resumes execution, user interface state information indicating information being displayed by the application 340-1 or information ready for display by the application 340-1, a state queue for enabling the user to return to a previous state or view of the application 340-1, and a redo / undo queue of previous actions taken by the user.

[0341] The event monitor 371 receives event information from the peripherals interface 318. The event information includes information about sub-events (e.g., user touches on the display system 312 as part of a multi-touch gesture when the display system 312 is a touch-sensitive display). The peripherals interface 318 transmits information it receives from the I / O subsystem 306 or sensors such as the proximity sensor 366, accelerometer 368, and / or microphone 313 (through the audio circuit 310). The information that the peripherals interface 318 receives from the I / O subsystem 306 includes information from the display system 312 (when it is a touch-sensitive display) or another touch-sensitive surface (e.g., Figure 1A to Figure 1B information of the touch panel 108).

[0342] In some embodiments, event monitor 371 sends requests to peripheral device interface 318 at predetermined intervals. In response, peripheral device interface 318 transmits event information. In other embodiments, peripheral device interface 318 transmits event information only when there is a significant event (e.g., receiving an input that is above a predetermined noise threshold and / or exceeds a predetermined duration).

[0343] In some embodiments, the event classifier 370 also includes a hit view determination module 372 and / or an active event identifier determination module 373.

[0344] Hit view determination module 372 provides software processes for determining where a sub-event occurs within one or more views when display system 312 displays more than one view, where a view consists of controls and other elements visible to a user on the display.

[0345] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the application) in which a touch is detected optionally correspond to programmatic levels within the application's programmatic or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are considered to be valid input is optionally determined based at least in part on the hit view of the initial touch that started the touch-based gesture.

[0346] The hit view determination module 372 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, the hit view determination module 372 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0347] Active event recognizer determination module 373 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 373 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 373 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, views higher in the hierarchy will still remain actively participating views.

[0348] Event dispatcher module 374 dispatches event information to event recognizers (e.g., event recognizer 380). In embodiments that include active event recognizer determination module 373, event dispatcher module 374 delivers the event information to the event recognizer determined by active event recognizer determination module 373. In some embodiments, event dispatcher module 374 stores the event information in an event queue, which is retrieved by corresponding event receiver 382.

[0349] In some embodiments, operating system 326 includes event classifier 370. Alternatively, application 340-1 includes event classifier 370. In other embodiments, event classifier 370 is a standalone module or is part of another module stored in memory 302, such as contact / motion module 330.

[0350] In some embodiments, application 340-1 includes multiple event handlers 390 and one or more application views 391, each of which includes instructions for handling touch events that occur within a corresponding view of the application's user interface. Each application view 391 of application 340-1 includes one or more event recognizers 380. Typically, an application view 391 includes multiple event recognizers 380. In other embodiments, one or more of the event recognizers 380 is part of a separate module, such as a user interface toolkit (not shown) or a higher-level object from which application 340-1 inherits methods and other properties. In some embodiments, the corresponding event handler 390 includes one or more of the following: a data updater 376, an object updater 377, a GUI updater 378, and / or event data 379 received from an event classifier 370. The event handler 390 optionally utilizes or calls the data updater 376, the object updater 377, or the GUI updater 378 to update the application internal state 392. Alternatively, one or more of the application views in application view 391 include one or more corresponding event handlers 390. In addition, in some embodiments, one or more of data updater 376, object updater 377, and GUI updater 378 are included in application view 391.

[0351] A corresponding event identifier 380 receives event information (e.g., event data 379) from event classifier 370 and identifies an event from the event information. Event identifier 380 includes an event receiver 382 and an event comparator 384. In some embodiments, event identifier 380 also includes metadata 383 and at least a subset of event delivery instructions 388 (which optionally include sub-event delivery instructions).

[0352] The event receiver 382 receives event information from the event classifier 370. The event information includes information about sub-events (e.g., touch or touch movement). Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0353] The event comparator 384 compares the event information with a predefined event or sub-event definition and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, the event comparator 384 includes an event definition 386. The event definition 386 includes a definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (387-1), event 2 (387-2), and other events. In some embodiments, the sub-events in event 387 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. For example, the definition of event 1 (387-1) is a double-click on a displayed object. A double-click, for example, includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off (touch end) of a predetermined duration. For another example, the definition of event 2 (387-2) is a drag on a displayed object. Drag, for example, includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch across the display system 312 (when it is a touch-sensitive display), and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 390.

[0354] In some embodiments, event definition 387 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 384 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view in which three user interface objects are displayed on display system 312 (when it is a touch-sensitive display), when a touch is detected on display system 312 (when it is a touch-sensitive display), event comparator 384 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 390, the event comparator uses the result of the hit test to determine which event handler 390 should be activated. For example, event comparator 384 selects an event handler that is associated with a sub-event and the object that triggered the hit test.

[0355] In some embodiments, the definition of the corresponding event 387 also includes a delay action that delays the delivery of the event information until it is determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0356] When a corresponding event recognizer 380 determines that a series of sub-events does not match any event in event definition 386, the corresponding event recognizer 380 enters the event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process the sub-events of the ongoing touch-based gesture.

[0357] In some embodiments, the corresponding event recognizer 380 includes metadata 383 having configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery for actively participating event recognizers. In some embodiments, metadata 383 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 383 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0358] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 380 activates the event handler 390 associated with the event. In some embodiments, the corresponding event recognizer 380 delivers event information associated with the event to the event handler 390. Activating the event handler 390 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 380 throws a flag associated with the identified event, and the event handler 390 associated with the flag retrieves the flag and performs a predefined process.

[0359] In some embodiments, event delivery instructions 388 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with the sub-event series or to an actively participating view. The event handler associated with the sub-event series or with the actively participating view receives the event information and performs a predetermined process.

[0360] In some embodiments, data updater 376 creates and updates data used in application 340-1. For example, data updater 376 stores video files used by media player module 344. In some embodiments, object updater 377 creates and updates objects used by application 340-1. For example, object updater 376 creates new user interface objects or updates the position of user interface objects. GUI updater 378 updates the GUI. For example, GUI updater 378 prepares display information and sends it to graphics module 332 for display on display system 312 (e.g., Figure 1A The main display 102 and / or the display portion 110 Figure 1A to Figure 1B Displayed on the dynamic function row 104).

[0361] In some embodiments, event handler 390 includes or has access to data updater 376, object updater 377, and GUI updater 378. In some embodiments, data updater 376, object updater 377, and GUI updater 378 are included in a single module of application 340-1 or application view 391. In other embodiments, they are included in two or more software modules.

[0362] It should be understood that the above discussion of event handling for user touches on a touch-sensitive display also applies to other forms of user input for operating the electronic device 300 using an input device, and not all user input is initiated on a touch screen. For example, mouse movement and mouse button presses, optionally combined with single or multiple keyboard presses or holds; contact movement on a trackpad, such as taps, drags, scrolls, etc.; stylus input; movement of the device; spoken commands; detected eye movement; biometric input; and / or any combination thereof, are optionally used as input corresponding to a sub-event defining an event to be recognized.

[0363] Figure 4 A block diagram of a peripheral electronic device 400 according to some embodiments is shown. In some embodiments, the peripheral electronic device 400 is a peripheral input and output device that at least partially includes a dynamic function row 104 and a physical input mechanism (such as a set of physical keys (e.g., Figures 2A to 2B The set of physical keys 106 in Figures 2B to 2C Examples of the peripheral electronic device 400 include: a peripheral keyboard (e.g., Figures 2A to 2B peripheral keyboard 206 in the peripheral touch-sensitive surface (e.g., Figure 2C ) or other peripheral input mechanisms (e.g., Figure 2DThe peripheral electronic device 400 and the computing device 202 ( Figures 2A to 2D ) is communicatively coupled. For example, the peripheral electronic device 400 is communicatively coupled to the computing device 202 via a wired connection (such as USB or PS / 2) or via a wireless communication link using a communication protocol (such as Bluetooth, Wi-Fi, etc.). The peripheral electronic device 400 may rely on the electronic device 300 ( Figure 3A ), or some of these components or processes may be completed by the peripheral electronic device 400 instead of the electronic device 300, located in the peripheral electronic device 400 instead of the electronic device 300, or accommodated by the peripheral electronic device 400 instead of the electronic device 300.

[0364] In some embodiments, the peripheral electronic device 400 includes one or more memories 402 (which optionally include one or more computer-readable storage media), a memory controller 422, one or more processing units (CPUs) 420, a peripheral device interface 418, RF circuitry 408, audio circuitry 410, a speaker 411, a microphone 413, an input / output (I / O) subsystem 406, other input or control devices 416, and an external port 424. The peripheral electronic device 400 includes a touch-sensitive display system 412 (e.g., Figures 2A to 2D row of dynamic functions 104 in the display) (also sometimes referred to herein as a "touch-sensitive display," "touch screen," or "touch screen display").

[0365] The peripheral electronic device 400 optionally includes a system for detecting a touch-sensitive surface such as a touch-sensitive display system 412 or a touchpad (e.g., Figures 2B to 2C The peripheral electronic device 400 optionally includes one or more intensity sensors 465 for measuring the intensity of a contact on a touch-sensitive surface (such as a touch-sensitive display system 412 or a touchpad (e.g., Figures 2B to 2C One or more tactile output generators 467 that generate tactile output on the touch panel 108 in . These components optionally communicate via one or more communication buses or signal lines 403.

[0366] Memory 402 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices. The access of other components of peripheral electronic device 400 (such as, CPU 420 and peripheral device interface 418) to memory 402 is optionally controlled by memory controller 422. Peripheral device interface 418 can be used for coupling CPU 420 and memory 402 to I / O subsystem 406 and other circuits. One or more processing units 420 run or execute various software programs and / or instruction sets stored in memory 402, to perform the various functions of peripheral electronic device 400 and process data. In some embodiments, peripheral device interface 418, CPU 420 and memory controller 422 are optionally implemented on a single chip (such as chip 404). In some other embodiments, they are optionally implemented on a separate chip.

[0367] RF (radio frequency) circuitry 408 receives and transmits RF signals, also known as electromagnetic signals. RF circuitry 408 converts electrical signals into / from electromagnetic signals and communicates with communication networks and other communication devices via the electromagnetic signals. RF circuitry 408 optionally includes well-known circuitry for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. Wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including, but not limited to, near field communication (NFC), Bluetooth, wireless fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Wi-MAX, or any other suitable communication protocol, including those not yet developed as of the filing date of this document.

[0368] Optional audio circuit 410, speaker 411 and microphone 413 provide an audio interface between the user and the peripheral electronic device 400. Audio circuit 410 receives audio data from peripheral device interface 418, converts the audio data into electrical signals, and sends the electrical signals to speaker 411. Speaker 411 converts the electrical signals into sound waves audible to humans. Audio circuit 410 also receives electrical signals converted from sound waves by microphone 413. Audio circuit 410 converts the electrical signals into audio data and transmits the audio data to peripheral device interface 418 for processing. Audio data is optionally retrieved from and / or transmitted to memory 402 and / or RF circuit 408 by peripheral device interface 418. In some embodiments, audio circuit 410 also includes a headset jack. The headset jack provides an interface between audio circuit 410 and removable audio input / output peripheral devices (such as, output-only headphones or headphones with both output (e.g., single-ear or dual-ear headphones) and input (e.g., microphone)).

[0369] The I / O subsystem 406 connects the input / output peripherals of the peripheral electronic device 400 (such as a touch-sensitive display system 412, for example Figures 2A to 2D 104) is coupled to a peripherals interface 418. The I / O subsystem 406 optionally includes a display controller 456, an intensity sensor controller 459, a haptic feedback controller 461, and one or more input controllers 460 for other input or control devices 416. The one or more other input controllers 460 receive / send electrical signals from / to other input or control devices 416. Other input or control devices 416 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, a set of physical keys, a trackpad, and the like.

[0370] Touch-sensitive display system 412 (e.g., Figures 2A to 2D The dynamic function row 104 in the peripheral electronic device 400 provides an input / output interface between the user and the touch sensitive display system 412. The touch sensitive display system 412 receives electrical signals from the touch sensitive display system 412 and / or sends electrical signals to the touch sensitive display system 412. The touch sensitive display system 412 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects / elements.

[0371] Touch-sensitive display system 412 (e.g., Figures 2A to 2DThe dynamic functionality row 104 in the example embodiment includes a touch-sensitive surface, sensor, or group of sensors that receives input from a user based on tactile contact. Thus, the touch-sensitive display system 412 and the TSD controller 456 (together with any associated modules and / or instruction sets in the memory 402) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 412 and convert the detected contact into a signal for selecting or controlling a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 412. In one exemplary embodiment, a point of contact between the touch-sensitive display system 412 and the user corresponds to an area of ​​the touch-sensitive display system 412 that is in contact with the user's finger.

[0372] Touch-sensitive display system 412 (e.g., Figures 2A to 2D The dynamic function rows 104 in the touch sensitive display 412 optionally use LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, LED (light emitting diode) technology, or OLED (organic light emitting diode) technology, although other display technologies may be used in other embodiments. The touch sensitive display system 412 and TSD controller 456 optionally use any of a variety of touch sensing technologies now known or later developed (including but not limited to capacitive technology, resistive technology, infrared technology, and surface acoustic wave technology) and other proximity sensor arrays or other elements for determining one or more contact points with the touch sensitive display 412 to detect contact and any movement or interruption thereof. In one exemplary embodiment, projected mutual capacitance sensing technology is used, such as the one from Apple Inc. (Cupertino, California). and Technology found in.

[0373] Touch-sensitive display system 412 (e.g., Figures 2A to 2D The dynamic function row 104 in the image processing unit 104 optionally has a video resolution greater than 400 dpi (e.g., 500 dpi, 800 dpi, or greater). In some embodiments, the user uses a stylus, finger, or the like to contact the touch-sensitive display system 412. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures.

[0374] In some embodiments, in addition to the touch-sensitive display system 412, the peripheral electronic device 400 optionally includes a touchpad (e.g., Figures 2B to 2CIn some embodiments, the touchpad is a touch-sensitive area of ​​the peripheral electronic device 400 that does not display visual output, unlike the touch-sensitive display system 412. In some embodiments, the touchpad is optionally a touch-sensitive surface that is separate from the touch-sensitive display system 412, or an extension of the touch-sensitive surface formed by the touch-sensitive display system 412.

[0375] The peripheral electronic device 400 also includes a power system 462 for powering the various components. The power system 462 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC), etc.), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0376] The peripheral electronic device 400 optionally also includes one or more contact force sensors 465 coupled to the force sensor controller 459 in the I / O subsystem 406. The contact force sensor 465 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). The contact force sensor 465 receives contact force information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact force sensor is coupled to a touch-sensitive surface (e.g., touch-sensitive display system 412 and / or Figures 2B to 2C The touch panel 108) is arranged in parallel or adjacent to each other.

[0377] The peripheral electronic device 400 optionally also includes one or more tactile output generators 467 coupled to the tactile feedback controller 461 in the I / O subsystem 406. The tactile output generators 467 optionally include one or more electroacoustic devices, such as speakers or other audio components; and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). The contact force sensor 465 receives tactile feedback generation instructions from the tactile feedback module 433 and generates tactile outputs that can be sensed by a user of the peripheral electronic device 400. In some embodiments, at least one tactile output generator is connected to a touch-sensitive surface (e.g., the touch-sensitive display system 412 and / or Figures 2B to 2C The touch-sensitive surface is arranged in juxtaposition with or adjacent to the touch pad 108 in the peripheral electronic device 400, and optionally generates tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward toward the surface of the peripheral electronic device 400) or laterally (e.g., back and forth in the same plane as the surface of the peripheral electronic device 400).

[0378] In some embodiments, the software components stored in memory 402 include an operating system 426, a communication module 428 (or instruction set), a contact / motion module 430 (or instruction set), and a dynamic function row module 450 (or instruction set). In addition, in some embodiments, memory 402 stores device state 457, including information indicating which views or other information occupy the touch-sensitive display system 412 (e.g., Figures 2A to 2D The display status of each area of ​​the dynamic function row 104).

[0379] Operating system 426 includes various software components and / or drivers for controlling and managing general system tasks (eg, memory management, storage device control, power management, etc.) and facilitates communications between various hardware and software components.

[0380] The communication module 428 facilitates communication with other devices (e.g., Figures 2A to 2D 2 (in the example embodiment) and further includes various software components for sending / receiving data via RF circuitry 408 and / or external port 424. External port 424 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.).

[0381] The contact / motion module 430 optionally detects contact with the touch-sensitive display system 412 and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 430 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there has been movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger up event or contact interruption). The contact / motion module 430 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single-finger contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some embodiments, the contact / motion module 430 also detects touch pad (e.g., Figures 2B to 2C Contact on the touchpad 108).

[0382] In some embodiments, the contact / motion module 430 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has made a selection or "clicked" on an enable representation). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and are adjustable without changing the physical hardware of the peripheral electronic device 400). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once by making a system-level click on the "intensity" parameter).

[0383] The contact / motion module 430 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, times, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap contact includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). For another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event and then detecting one or more finger drag events, and in some embodiments, subsequently detecting a finger lift (lift-off) event.

[0384] The tactile feedback module 433 includes various software components for generating instructions that are used by one or more tactile output generators 467 to produce tactile output at one or more locations on the peripheral electronic device 400 in response to user interaction with the peripheral electronic device 400.

[0385] The dynamic function row (DFR) module 450 includes a focus acquisition module 451, a DFR determination module 452, and a DFR presentation module 453. In some embodiments, the focus acquisition module 451 is configured to obtain an indication of an active user interface element that is displayed in a video from the computing device 202 ( Figures 2A to 2D ) of the peripheral display device 204 ( Figures 2A to 2D ) of the graphical user interface on the primary display 102. In some embodiments, the DFR determination module 452 is configured to determine a graphic (e.g., a set of one or more affordances) based on the active user interface element that is currently in focus. Alternatively, in some embodiments, the computing device 202 ( Figures 2A to 2D) determines a graphic (e.g., the set of one or more affordances) based on the focused active user interface element, and provides the graphic to the peripheral electronic device 400 or a component thereof (e.g., the DFR module 450) for display on the touch-sensitive display system 412 (e.g., Figures 2A to 2D In some embodiments, the DFR rendering module 453 is configured to render the graphics determined by the DFR determination module 452 (or provided by the computing device 202) on the touch-sensitive display system 412 (e.g., Figures 2A to 2D The DFR rendering module 453 includes various known software components for rendering and causing graphics to be displayed on the touch-sensitive display system 412, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. In some embodiments, the DFR module 450 includes modules for: adjusting the sensitivity of the dynamic function row 104; adjusting the audible and / or tactile feedback provided by the dynamic function row 104; adjusting the settings (e.g., size, brightness, font, language, etc.) of the enable representations and information displayed by the dynamic function row 104; adjusting the current power mode (e.g., normal mode and low power mode) of the dynamic function row 104; and the like.

[0386] In some embodiments, memory 402 includes an event classifier 470 (e.g., in operating system 426). In some embodiments, event classifier 470 performs the same operations as event classifier 370 ( Figure 3B ) and includes an event classifier 370 ( Figure 3B ) is a subset or superset of the modules, processes, and instructions of the event classifier 470. Therefore, for the sake of brevity, the event classifier 470 will not be described.

[0387] It should be understood that the peripheral electronic device 400 is only one example, and that the peripheral electronic device 400 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 4 The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing and / or application specific integrated circuits.

[0388] Each of the above modules corresponds to an executable instruction set for performing one or more functions and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods as described herein). These modules (i.e., instruction sets) need not be implemented with independent software programs, processes, or modules, so the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 402 optionally stores a subset of the above modules and data structures. In addition, memory 402 optionally stores additional modules and data structures not described above.

[0389] As used herein, the term "focus selector" refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the focus selector is displayed on a touch-sensitive surface (e.g., trackpad 355 or 356 in FIG. 3 ). Figure 4 In some implementations, when input (e.g., a press input) is detected on the touch-sensitive surface 451 in the touchscreen display, the particular user interface element is adjusted based on the detected input. In some implementations of a touchscreen display that enables direct interaction with user interface elements on the touchscreen display, a contact detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by a contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted based on the detected input. In some implementations, focus moves from one area of ​​the user interface to another area of ​​the user interface without corresponding movement of a cursor or movement of a contact on the touchscreen display (e.g., by using a tab key or arrow keys to move focus from one button to another); in these implementations, the focus selector moves based on the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user interface element (or contact on the touchscreen display) that is controlled by the user to convey the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user intends to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch screen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).

[0390] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, the force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated force of the contact. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in pressure units). Using the intensity of the contact as an attribute of the user input allows the user to access additional device functionality that the user would not otherwise be able to easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls such as knobs or buttons).

[0391] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether a user has performed an action (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and are adjustable without changing the physical hardware of the system 100). For example, a mouse "click" threshold for a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using a system-level click on an "intensity" parameter).

[0392] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact is detected to increase in intensity, and / or before or after contact is detected to decrease in intensity). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half-maximum value of the contact intensity, the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used in determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an operation. For example, the set of one or more intensity thresholds may include a first intensity threshold and a second intensity threshold. In this example, a contact whose characteristic intensity does not exceed the first threshold results in a first operation, a contact whose characteristic intensity exceeds the first intensity threshold but does not exceed the second intensity threshold results in a second operation, and a contact whose characteristic intensity exceeds the second threshold results in a third operation. In some embodiments, the comparison between the characteristic intensity and the one or more intensity thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0393] In some embodiments, a portion of a gesture is identified for use in determining the characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an ending position (e.g., a drag gesture) where the strength of the contact increases. In this example, the characteristic strength of the contact at the ending position may be based on only a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the intensity of the swipe contact in order to determine the characteristic strength.

[0394] In some embodiments, one or more predefined intensity thresholds are used to determine whether a particular input meets the intensity-based criteria. For example, the one or more predefined intensity thresholds include: (i) a contact detection intensity threshold IT0, (ii) a light press intensity threshold IT1, (iii) a touch detection intensity threshold IT2, (iv) a light press intensity threshold IT3, (v) a touch detection intensity threshold IT4, (v) a light press intensity threshold IT5, (v) a touch detection intensity threshold IT6, (v) a light press intensity threshold IT L , (iii) Deep pressing intensity threshold IT D (e.g., at least initially above I L ), and / or (iv) one or more other intensity thresholds (e.g., below I L The intensity threshold I H ). In some embodiments, the light press strength threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or a touchpad. In some embodiments, the deep press strength threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or a touchpad. In some embodiments, when a contact is detected having a characteristic intensity lower than the light press strength threshold (e.g., and higher than a nominal contact detection intensity threshold IT0, contacts lower than the nominal contact detection intensity threshold are no longer detected), the device will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing an operation associated with the light press strength threshold or the deep press strength threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface illustrations.

[0395] In some embodiments, the device's response to an input detected by the device depends on criteria based on the intensity of the contact during the input. For example, for some "light press" inputs, the intensity of the contact exceeding a first intensity threshold during the input triggers a first response. In some embodiments, the device's response to an input detected by the device depends on criteria that include the intensity of the contact during the input and a time-based criterion. For example, for certain "deep press" inputs, when the intensity of the contact exceeding a second intensity threshold during the input is greater than the first intensity threshold for a light press, the second response is triggered only if a delay time elapses between the first intensity threshold being met and the second intensity threshold being met. This delay time is typically less than 200ms in duration (e.g., 40, 100, or 120ms, depending on the magnitude of the second intensity threshold, with the delay time increasing as the second intensity threshold increases). This delay time helps avoid accidental deep press inputs. As another example, for some "deep press" inputs, a period of reduced sensitivity occurs after reaching the first intensity threshold. During this period of reduced sensitivity, the second intensity threshold is increased. This temporary increase in the second intensity threshold also helps avoid accidental deep press inputs. For other deep press inputs, the response to detecting the deep press input does not depend on time-based criteria.

[0396] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of concurrent inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Exemplary factors are described in U.S. patent application Ser. Nos. 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entireties.

[0397] For example, Figure 3C A dynamic intensity threshold 480 is shown that varies over time based in part on the intensity of the touch input 476 over time. The dynamic intensity threshold 480 is the sum of two components, a first component 474 that decays over time after a predefined delay time p1 from the initial detection of the touch input 476; and a second component 478 that tracks the intensity of the touch input 476 over time. The initial high intensity threshold of the first component 474 reduces the accidental triggering of a "deep press" response while still allowing the "deep press" response to occur immediately if the touch input 476 provides sufficient intensity. The second component 478 reduces the accidental triggering of a "deep press" response by gradually fluctuating the intensity in the touch input. In some embodiments, when the touch input 476 meets the dynamic intensity threshold 480 (e.g., at Figure 3C ), triggering a “deep press” response.

[0398] Figure 3D Another dynamic intensity threshold 486 (eg, intensity threshold I D ). Figure 3D Two other intensity thresholds are also shown: a first intensity threshold I H and the second intensity threshold I L .exist Figure 3D Although the touch input 484 meets the first intensity threshold I before time p2, H and the second intensity threshold I L , but no response is provided until after the delay time p2 has elapsed at time 482. Similarly, Figure 3D In the embodiment, the dynamic intensity threshold 486 decays over time, and the decay starts at a time 488 after a predefined delay time p1 from the time 482 (at this time, the dynamic intensity threshold 486 decays over time). L This type of dynamic intensity threshold reduces the amount of time between triggering and a lower intensity threshold (e.g., the first intensity threshold I H Or the second intensity threshold I L ) after or at the same time as the response associated with the dynamic intensity threshold I D The associated response.

[0399] Figure 3E Another dynamic intensity threshold 492 (eg, intensity threshold I D ).exist Figure 3E In the middle, with the intensity threshold I L The associated response is triggered after a delay time p2 has elapsed from the initial detection of the touch input 490. Meanwhile, the dynamic intensity threshold 492 decays after a predefined delay time p1 has elapsed from the initial detection of the touch input 490. Thus, the dynamic intensity threshold 492 decays after a predefined delay time p1 has elapsed from the initial detection of the touch input 490. L After the associated response, the intensity of the touch input 490 decreases, followed by an increase in the intensity of the touch input 490 without releasing the touch input 490, which can trigger the touch input 490 associated with the intensity threshold I D (e.g., at time 494) even when the intensity of touch input 490 is below another intensity threshold, such as intensity threshold I L It is the same at this time.

[0400] The contact characteristic strength is lower than the light press strength threshold IT L The intensity increases to between the light press intensity threshold IT L and deep press intensity threshold IT D The intensity between the deep press intensity threshold IT is sometimes referred to as a "light press" input. D The intensity of the deep press increases to above the deep press intensity threshold IT D The intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. L A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting liftoff of the contact from the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some illustrations, shaded circles or ellipses are used to represent the intensity of contacts on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0401] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting an increase in the strength of the contact (or multiple contacts) to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting an increase in the strength of the corresponding contact to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and a subsequent decrease in the strength of the contact to below the press input strength threshold, and the corresponding operation is performed in response to detecting a subsequent decrease in the strength of the corresponding contact to below the press input strength threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0402] In some embodiments, the device employs intensity hysteresis to avoid unintended input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the intensity of the contact increases from an intensity equal to or less than a hysteresis intensity threshold to an intensity equal to or greater than a press input intensity threshold and, optionally, the intensity of the contact subsequently decreases to an intensity equal to or less than the hysteresis intensity, and a corresponding operation is performed in response to detecting the press input (e.g., an increase in the intensity of the contact or a decrease in the intensity of the contact, depending on the circumstances).

[0403] For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold, or in response to a gesture including a press input, are optionally triggered in response to detecting: the intensity of the contact increasing above the press input intensity threshold, the intensity of the contact increasing from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreasing below the press input intensity threshold, or the intensity of the contact decreasing below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting a decrease in the intensity of the contact below the press input intensity threshold, the operations are optionally performed in response to detecting a decrease in the intensity of the contact below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, the triggering of these responses is also dependent on satisfying time-based criteria (e.g., a delay time having elapsed between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0404] User interface and associated processes

[0405] Now note that the portable computing system 100 ( Figure 1A ) or desktop computing system 200 ( Figures 2A to 2D ) implements an embodiment of a user interface ("UI") and associated processes. In some embodiments, the main display 102 is on the display portion 110 ( Figure 1A Alternatively, in some embodiments, the main display 102 is implemented in the peripheral display device 204 ( Figures 2A to 2D ) is implemented in. In some embodiments, the dynamic function row 104 is implemented in the portable computing system 100 ( Figure 1A to Figure 1B Alternatively, in some embodiments, the dynamic function row 104 is a touch-sensitive display implemented in the main body portion 120 of the peripheral keyboard 206 ( Figures 2A to 2B ), the first peripheral input mechanism 212 ( Figure 2C ) or peripheral input mechanism 222 ( Figure 2D ) implemented in a touch-sensitive display.

[0406] Figures 5A to 14E FIGURES 10 and 11 illustrate exemplary user interfaces for displaying application-specific affordances on a dynamically updated touch screen display according to some embodiments. The user interfaces in these figures are used to illustrate the methods and / or processes described below, including Figures 44A to 44D 、 Figures 45A to 45C 、 Figures 46A to 46B 、 Figures 47A to 47B as well as Figures 48A to 48CThose skilled in the art will appreciate that the following user interfaces are merely examples. Furthermore, those skilled in the art will appreciate that additional affordances and / or user interface elements may be used in practice, or fewer affordances and / or user interface elements may be used.

[0407] Figure 5A The main display 102 is shown displaying a status tray 502 indicating that the system (i.e., operating system) is currently in focus, and an application (application) tray 514 having a plurality of executable / selectable application icons, including a mail application icon 506 (e.g., corresponding to Figure 3A Email client module 341 in ), web browser application icon 508 (e.g., corresponding to Figure 3A Web browsing module 345 in ), media player application icon 510 (eg, corresponding to Figure 3A ), application A icon 512 (eg, corresponding to a game), and photo application icon 515 (eg, corresponding to Figure 3A In some embodiments, the status tray 502 indicates the application currently running in the foreground and also includes multiple menus (e.g., Figure 5A ), each menu includes a corresponding set of controls for that application. Figure 5A The main display 102 is also shown displaying a cursor 504 at a location corresponding to the application A icon 512. In some embodiments, the cursor 504 is displayed by the portable computing system 100 ( Figure 1A to Figure 1B ) of the touch panel 108, the desktop computing system 200 ( Figure 2A and 2D ) of the peripheral mouse 208, the peripheral keyboard 206 ( Figure 2B ) of the touch panel 108, the first peripheral input mechanism 212 ( Figure 2C ) touch panel 108 and other controls.

[0408] Figure 5A Further shown is a dynamic functionality row 104 (e.g., a touch-sensitive display) displaying multiple affordances based on the current focus of the primary display 102 (i.e., the operating system is displayed since no application windows are open). Figure 5A , the system / operating system is currently focused on the primary display 102. Figure 5A, dynamic function row 104 includes persistent controls implemented as physical and / or soft keys, including: an away affordance 516 that, when activated (e.g., via a tap contact), invokes a corresponding function (e.g., exits an application currently focused on primary display 102 or pauses a game); and a power control 534 that, when activated (e.g., via a tap contact), causes a modal alert to be displayed on dynamic function row 104 and / or primary display 102 (e.g., Figure 14E ) to log out, restart, or shut down the portable computing system 100 or desktop computing system 200.

[0409] exist Figure 5A In the example, the dynamic function row 104 also includes a plurality of system-level affordances, including: a function for adjusting the primary display 102 ( Figure 1A and Figures 2A to 2B ) brightness can indicate 518; for adjusting the brightness of the set of physical keys 106 ( Figure 1A to Figure 1B and Figures 2A to 2B , when applicable) and / or the brightness of the dynamic function row 104; an exposure indication 522, which, when activated (e.g., via a tap contact), causes the primary display 102 ( Figure 1A and Figures 2A to 2B ) displays a preview window for the active application on main display 102 ( ); a search affordance 524 for performing a local search (e.g., for electronic documents) and / or an Internet search; a launch pad affordance 526 that, when activated (e.g., via a tap contact), causes a user to browse to the main display 102 ( Figure 1A and Figures 2A to 2B ) displays default or user-selected widgets and tools on the main display 102 ( ); a notification indicator 528 that, when activated (e.g., via a tap contact), displays a notification on the main display 102 ( Figure 1A and Figures 2A to 2B ) displays a notification center including recent messages, notifications, calendar events, etc.; a play / pause indicator 530 for initiating playback or pausing playback of a media item (e.g., a song, podcast, video, etc.); and a volume control indicator 532 for adjusting the volume of the media item being played. For example, when a tap is detected on the brightness indicator 520, the dynamic function row 104 displays a brightness slider (e.g., similar to a brightness slider) for adjusting the brightness of the set of physical keys 106 and / or the brightness of the dynamic function row 104. Figure 6F In some embodiments, the multiple system-level affordances also include a settings affordance (not shown) for accessing adjustment settings associated with the dynamic function row 104, such as symbol / icon size, touch detection sensitivity, tactile feedback, audible feedback, animations for focus changes, power modes, etc.

[0410] Figure 5B The main display 102 is shown in response to detecting Figure 5A In the example, the application A icon 512 is selected using the cursor 504 to display a window 536 of the application A (for example, a fantasy RPG game). Figure 5B , application A is in main menu mode (e.g., a fantasy RPG game is paused), and window 536 displays the main menu of application A. Window 536 of application A is focused on primary display 102. Figure 5B In , the status tray 502 indicates that application A is running in the foreground, and the application tray 514 also indicates that application A is running in the foreground based on the shadow behind application A icon 512. Figure 5B , the window 536 of application A includes three optional enabling indications (from left to right, respectively) in the upper left corner for closing the window 536, maximizing the window 536, and minimizing the window 536.

[0411] Figure 5B It also shows that in response to detecting Figure 5A 504, dynamic function row 104 displays an affordance 538 in addition to persistent controls (i.e., affordances 516 and 534) and a plurality of system-level affordances (i.e., affordances 518, 520, 522, 524, 526, 528, 530, and 532). When activated (e.g., via a tap contact), affordance 538 causes dynamic function row 104 to display a first set of affordances and / or indicators corresponding to application A (e.g., Figure 5C Control group A). Figure 5B The dynamic function row 104 is further shown receiving and detecting contact 540 (e.g., a tap contact) at a location corresponding to the enable indication 538.

[0412] Figure 5C Shows the response to detecting Figure 5B Upon selection of affordance 538 in the dynamic functionality row 104, the dynamic functionality row 104 displays a first set of affordances and / or indicators corresponding to application A (e.g., control group A) and at least one system-level affordance (e.g., affordance 542). Figure 5C , a first set of affordances and / or indicators (e.g., control set A) corresponding to application A (e.g., a fantasy RPG game) includes a health indicator 543 and a prestige indicator 545 associated with an in-game character / avatar controlled by a user of portable computing system 100 or desktop computing system 200 while playing application A. Figure 5C, the first set of affordances and / or indicators corresponding to application A (e.g., control set A) also includes control affordances 546-A, 546-B, and 546-C for controlling a character / avatar in the game. When activated (e.g., via a tap contact), affordance 542 causes dynamic function row 104 to display a plurality of system-level affordances (e.g., Figure 5A The indicators shown in FIG. 5 represent 518 , 520 , 522 , 524 , 526 , 528 , 530 and 532 ). Figure 5C The dynamic function row 104 is also shown receiving and detecting an upward swipe gesture with contact 544 moving from a first position 548 -A to a second position 548 -B.

[0413] Figure 5D Shows the response to detecting Figure 5C , dynamic functionality row 104 displays a second set of affordances and / or indicators corresponding to application A (e.g., control group B) and at least one system-level affordance (e.g., affordance 542). Figure 5D , a second set of enable representations and / or indicators corresponding to application A (e.g., control group B) includes control enable representations 546-D, 546-E, 546-F, 546-G, 546-H, 546-I, 546-J, and 546-K for controlling an in-game character / avatar controlled by a user of the portable computing system 100 or desktop computing system 200 when playing application A. Figure 5D Dynamic function row 104 is also shown receiving and detecting contact 552 (e.g., a tap contact) at a location corresponding to enable indication 542.

[0414] Figure 5E Shows the response to detecting Figure 5D With the enable representation 542 selected, the dynamic functionality row 104 displays persistent controls (i.e., enable representations 516 and 534), multiple system-level enable representations (i.e., enable representations 518, 520, 522, 524, 526, 528, 530, and 532), and an enable representation 538 corresponding to application A. Figure 5E The primary display 102 is also shown displaying a cursor 504 at a new position corresponding to the media player application icon 510 .

[0415] Figure 6A The main display 102 is shown in response to detecting Figure 5E 5. In the example, the media player application icon 510 is selected using the cursor 504 to display the window 554 of the media player application. For example, the window 554 is overlaid on the window 536. Figure 6A , window 554 displays a plurality of albums associated with the music subsection of the user's media library. Figure 6A , the music subsection of the user's media library is focused on the primary display 102, as indicated by the bold display of "Music" and albums A through L at least partially displayed in window 554. Figure 6A , the status tray 502 indicates that the media player application is running in the foreground, and the application tray 514 also indicates that the media player application is running in the foreground based on the shadow behind the media player application icon 510. Figure 6A The primary display 102 is also shown displaying a cursor 504 at a location corresponding to the podcast subsection of the user's media library.

[0416] Figure 6A Further shown is the dynamic function row 104 in response to detecting Figure 5E 542 . In the example shown in FIG. 54 , media player application icon 510 is selected using cursor 504 to display multiple album affordances 558 (e.g., album affordances 558-A through 558-G) and at least one system-level affordance (e.g., affordance 542 ). Figure 6A , multiple album affordances 558 correspond to a subset of the albums currently displayed in window 554. In some embodiments, multiple album affordances 558 reflect the albums currently displayed in window 554. For example, in response to selecting album affordance 558-D (e.g., via a tap contact), portable computing system 100 or computing device 200 causes audio circuit 310 ( Figure 3A ) plays back album D, and also causes the main display 102 to display album D in the "Now Playing" area of ​​window 554.

[0417] Figure 6B The main display 102 is shown in response to detecting Figure 6A 554. Figure 6B , window 554 displays a plurality of podcasts associated with the podcast subsection of the user's media library. Figure 6B , the podcasts sub-section of the user's media library is focused on the main display 102, as indicated by "Podcasts" displayed in bold in the menu and podcasts A through L at least partially displayed in window 554.

[0418] Figure 6B Further shown is the dynamic function row 104 in response to detecting Figure 6A , selecting a podcast sub-section with cursor 504 to display a first plurality of podcast affordances 560 (e.g., podcast affordances 560-A through 560-G) and at least one system-level affordance (e.g., affordance 542). Figure 6B, multiple podcast affordances 560 correspond to a subset of the podcasts currently displayed in window 554. For example, in response to selecting podcast affordance 560-D (e.g., via a tap contact), portable computing system 100 or computing device 200 causes audio circuitry 310 ( Figure 3A ) plays back podcast D, and also causes the main display 102 to display podcast D in the "Now Playing" area of ​​window 554. Figure 6B The dynamic function row 104 is further shown detecting a right-to-left swipe gesture with contact 562 moving from a first position 564-A to a second position 564-B.

[0419] Figure 6C The main display 102 is shown in response to detecting Figure 6B A right-to-left swipe gesture in the is performed to display a second plurality of podcast enable indications 560 (e.g., podcast enable indications 560-E to 560-P) in window 554. Figure 6C Also shown is a dynamic function row 104 in response to detecting Figure 6B A right-to-left swipe gesture in the is performed to display a second plurality of podcast enable indications 560 (e.g., podcast enable indications 560-E to 560-K) and at least one system-level enable indication (e.g., enable indication 542). Figure 6C Dynamic functionality row 104 is further shown receiving and detecting contact 566 (e.g., a tap contact) at a location corresponding to podcast enable indication 560-J.

[0420] Figure 6D The main display 102 is shown in response to detecting the selection Figure 6C The podcast indicator in 560-J shows the playback of podcast J in window 554. Figure 6D The primary display 102 is also shown displaying a cursor 504 at a location corresponding to a mail application icon 506 .

[0421] Figure 6D Further illustrating the dynamic function row 104 in response to detecting the selection Figure 6C The podcast icon in the figure can be represented as 560-J and displays the persistent volume control 568 and the playback controls and indicators. Figure 6D , the persistent volume control 568 indicates that Podcast J is not muted, and the equalizer feedback for Podcast J is also displayed. Figure 6D In , the playback controls include a rewind control 571, a pause control 572, and a fast forward control 573 for controlling podcast J. Figure 6D , the play indicator includes an image 574 corresponding to podcast J (e.g., a cover design or associated image), an indicator 576 displaying the author and title of podcast J, and a remaining time indicator 578.

[0422] Figure 6E The main display 102 is shown in response to detecting Figure 6D In the example, the mail application icon 506 is selected using the cursor 504 to display the mail application window 580. For example, the window 580 is overlaid on the windows 554 and 536. Figure 6E , window 580 displays a list of multiple emails (e.g., emails A to F) in the user's inbox and the contents of the selected email A. In some embodiments, the newest or most urgent email is displayed at the top of the list of multiple emails in the user's inbox, and the email at the top of the list is automatically selected. Figure 6E , email A is focused on the primary display 102 because the email is displayed in bold in a list of multiple emails, and the content of email A is displayed below the list in window 580. Figure 6E , status tray 502 indicates that the mail application is running in the foreground, and application tray 514 also indicates that the mail application is running in the foreground based on the shadow behind mail application icon 506.

[0423] Figure 6E Also shown is the dynamic function row 104 in response to the Figure 6D Detection of selection of mail application icon 506 using cursor 504 displays multiple affordances corresponding to email A (e.g., affordances 582, 584, 586, 588, 590, 592, 594, 596, and 598) and at least one system-level affordance (e.g., affordance 542). Figure 6E , the plurality of affordances corresponding to email A include affordance 582, which, when activated (e.g., via a tap contact), refreshes the inbox; affordance 584, which, when activated (e.g., via a tap contact), causes main display 102 to display a sub-window for composing a new email (e.g., as shown in FIG. 1 ); Figure 7B ); enable indication 586, which, when activated (e.g., via a tap contact), causes the main display 102 to display a sub-window for replying to the sender of email A; enable indication 588, which, when activated (e.g., via a tap contact), causes the main display 102 to display a sub-window for replying to all recipients of email A; enable indication 590, which, when activated (e.g., via a tap contact), causes the main display 102 to display a sub-window for forwarding email A; enable indication 592, which, when activated (e.g., via a tap contact), causes email A to be filed to a default mailbox or folder; enable indication 594, which, when activated (e.g., via a tap contact), causes email A to be deleted; enable indication 596, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display functions to be applied to email A (e.g., as Figure 7A a set of enable representations (shown) for selecting different tags, which optionally correspond to pre-existing folders; and an enable representation 598, which, when activated (e.g., via a tap contact), causes the main display 102 or the dynamic function row 104 to display a search dialog box for searching the user's inbox. Figure 6E The dynamic function row 104 is further shown receiving and detecting contact 599 (e.g., a long press gesture) at a location corresponding to the persistent volume control 568.

[0424] Figure 6F The dynamic function row 104 is shown in response to the corresponding Figure 6E A long press gesture is detected at the location of the persistent volume control 568 in the display to display a volume slider 5100 for adjusting the playback volume of podcast J, which is in the Figure 6C The circle / thumb in the volume slider 5100 can be dragged by the user of the portable computing system 100 or the desktop computing system 200 to adjust the volume. Alternatively, in some embodiments, the dynamic function row 104 is responsive to the action in the corresponding Figure 6E A long press gesture is detected at the location of the persistent volume control 568 in the display to display playback controls (e.g., pause, fast forward, rewind, next track, previous track, etc.) for controlling playback of podcast J, which is played back in the Figure 6C Initiated in Figure 6F , the volume slider 5100 is overlaid on multiple enable representations corresponding to email A. Figure 6F The dynamic function row 104 is also shown receiving and detecting contact 5102 (e.g., a tap contact) at a location corresponding to the enable indication 596.

[0425] In other embodiments, volume slider 5100 is displayed in a different area than the multiple affordances corresponding to email A. Thus, when volume slider 5100 is activated, the multiple affordances corresponding to email A (e.g., affordances 582, 584, 586, 588, 590, 592, 594, 596, and 598) scroll or shrink in an animated manner to ensure that there is sufficient display space to display volume slider 5100. In some embodiments, a circle / thumb is displayed under the user's finger in response to a long press gesture to allow the user to slide the circle / thumb without removing contact from the affordances.

[0426] Figure 7A The dynamic function row 104 is shown in response to detecting Figure 6F Selecting affordance 596 displays a set of affordances 5103, 5104, 5106, 5108, and 5110 for selecting different tags applicable to email A, which optionally correspond to pre-existing folders. Figure 7A, the dynamic function row 104 also displays a function for stopping displaying the set of enable representations 5103, 5104, 5106, 5108 and 5110 on the dynamic function row 104 and displaying multiple enable representations corresponding to email A on the dynamic function row 104 (for example, Figure 6E The exit indicator (as shown) indicates 5112. Figure 7A Main display 102 is also shown displaying cursor 504 in window 580 at a location corresponding to the email composition affordance.

[0427] Figure 7B The main display 102 is shown in response to detecting Figure 7A Selecting the email composition indicator with cursor 504 displays a sub-window for composing a new email in window 580. Figure 7B , the sub-window for composing a new email is in focus on the main display 102 as indicated by the email composition affordance displayed in a thicker line and also as indicated by the thick line surrounding the sub-window for composing a new email.

[0428] Figure 7B Also shown is a dynamic function row 104 in response to detecting Figure 7A Selecting an email composition affordance using cursor 504 displays a set of affordances corresponding to composing a new email. Figure 7B , the set of enable indications corresponding to composing a new email includes enable indication 5114, which, when activated (e.g., via a tap contact), causes the portable computing system 100 or desktop computing system 200 to send the newly composed email; enable indication 5116, which is used to change the text color of the text of the new email; enable indication 5118, which is used to bold selected text of the new email; enable indication 5120, which is used to italicize selected text of the new email; and enable indication 5122, which is used to underline selected text of the new email. Figure 7B The dynamic function row 104 is further shown receiving and detecting contact 5124 (e.g., a tap contact) at a location corresponding to the persistent volume control 568.

[0429] Figure 7C The main display 102 is shown displaying the text in the body of a new email in a sub-window for composing a new email within window 580, and an indicator at the current insertion position after the characters "pl".

[0430] Figure 7C Also shown is the dynamic function row 104 displaying a persistent volume control 568 that indicates the volume of the Figure 7B A tap contact is detected at the location of the persistent volume control 568 in the Podcast J to mute the podcast. Figure 7C , the persistent volume control 568 is available even in Figure 6C In the case where the podcast J that has been started to play is muted (i.e., it is shown that the podcast J is still playing but muted), the equalizer feedback of the podcast J is also displayed. Figure 7C , the dynamic function row 104 displays predicted words 5126 , 5128 , and 5130 to complete the word beginning with “pl” being typed in the body of the new email based on the insertion point in the subwindow within window 580 . Figure 7C The dynamic function row 104 is further shown receiving and detecting a contact 5132 (e.g., a tap contact) at a location corresponding to the predicted word 5126 (i.e., "planet").

[0431] Figure 7D Shows the response to detecting Figure 7C 580 , the main display 102 displays the word “planet” in the body of the new email in the sub-window for composing the new email. Figure 7D The main display 102 is also shown displaying a cursor 504 at a location corresponding to the "To:" field of the sub-window for composing a new email.

[0432] Figure 8A The main display 102 is shown in response to detecting Figure 7D 504 to select the "To:" field and display a menu 5134 corresponding to the user's address book. Figure 8A , menu 5134 includes a list of multiple contacts corresponding to an "all contacts" group of the user's address book (e.g., a pre-existing or auto-populated list of contacts). Figure 8A , the “All Contacts” group of the user's address book is focused on the main display 102, as indicated by the thick line surrounding the “All Contacts” group in menu 5134.

[0433] Figure 8A Also shown is a dynamic function row 104 in response to detecting Figure 7D 542 ). In FIG. 54 , the “To:” field is selected using cursor 504 to display a first plurality of contact affordances 5136 (e.g., contact affordances 5136-A through 5136-F) corresponding to an “All Contacts” group of the user’s address book and at least one system-level affordance (e.g., affordance 542). Figure 8A , the dynamic function row 104 also displays an exit enable indication 5112, which, when activated (e.g., via a tap contact), causes the main display 102 to stop displaying the menu 5134 on the main display 102, and also causes the dynamic function row 104 to stop displaying the first plurality of contacts enable indication 5136. Figure 8AThe dynamic functionality row 104 is also shown detecting a right-to-left swipe gesture with contact 5138 moving from a first position 5140-A to a second position 5140-B (e.g., the user scrolls from right to left through "All Contacts").

[0434] Figure 8B The dynamic function row 104 is shown in response to detecting Figure 8A A right-to-left swipe gesture in the display shows a second plurality of contact enable representations 5136 (e.g., contact enable representations 5136-E to 5136-J) corresponding to the "All Contacts" group of the user's address book and at least one system-level enable representation (e.g., enable representation 542). Figure 8B The dynamic function row 104 is also shown detecting an upward swipe gesture with contact 5142 moving from a first position 5144-A to a second position 5144-B.

[0435] Figure 8C The main display 102 is shown in response to detecting Figure 8B , and displays a list of multiple contacts corresponding to the "Home" group of the user's address book. Figure 8C , the “Home” group of the user's address book is focused on the main display 102, as indicated by the thick line surrounding the “Home” group in menu 5134.

[0436] Figure 8C The dynamic function row 104 is shown in response to detecting Figure 8B An upward swipe gesture in displays multiple contact enable representations 5146 (e.g., contact enable representations 5146-A to 5146-F) corresponding to the "Home" group of the user's address book and at least one system-level enable representation (e.g., enable representation 542). Figure 8C Dynamic functionality row 104 is further shown receiving and detecting contact 5148 (e.g., a tap contact) at a location corresponding to contact enable indication 5146-D, which is associated with a contact named "James H." within the "Home" group of the user's address book.

[0437] Figure 8D The main display 102 is shown in response to detecting Figure 8C Selecting the contact indicator 5146-D displays "James H." in the "To:" field of the sub-window for composing a new email in window 580. Figure 8D Also shown is the response to detecting the Figure 8CWhen contact enable representation 5146-D is selected, dynamic function row 104 replaces the display of multiple contact enable representations 5146 (e.g., contact enable representations 5146-A to 5146-F) corresponding to the "home" group of the user's address book with a set of enable representations corresponding to composing new emails (e.g., enable representations 5114, 5116, 5118, 5120 and 5122). Figure 8D The dynamic functionality row 104 is further shown receiving and detecting contact 5150 (e.g., a tap contact) at a location corresponding to the at least one system-level enable indication 542.

[0438] Figure 8E Shows the response to detecting Figure 8D With the enable indication 542 selected, the dynamic functionality row 104 displays persistent controls (i.e., enable indications 516 and 534), multiple system-level enable indications (i.e., enable indications 518, 520, 522, 524, 526, 528, 530, and 532), and an enable indication 5152 corresponding to the mail application. Figure 8E Dynamic functionality row 104 is also shown receiving and detecting contact 5154 (e.g., a tap contact) at a location corresponding to play / pause affordance 530. For example, in response to detecting selection of play / pause affordance 530, portable computing system 100 or desktop computing system 200 pauses playback of podcast J, which is accessed via the user interface associated with the podcast. Figure 6C The interaction described by Figure 7B The description of is muted. Figure 8E Then select the Play / Pause indicator 530 to restart playback of podcast J.

[0439] Figure 8F Primary display 102 is shown displaying cursor 504 at a position corresponding to an exit affordance for closing window 580 . Figure 8G The main display 102 is shown in response to detecting Figure 8F 504 is used to select the exit indicator and display the modal alert 5156. Figure 8G , the modal alert 5156 is focused on the primary display 102. Figure 8G , a modal alert 5156 displayed on the main display 102 prompts the user to save the draft email before closing the window 580 and includes a "Save" enable indication, a "Don't Save" enable indication, and a "Cancel" enable indication. Figure 8GMain display 102 is also shown displaying cursor 504 at a position corresponding to the "Cancel" affordance. Displaying modal alerts on the dynamic function row improves efficiency and provides a better user experience because it does not require the user to move their eyes between the keyboard and the screen, nor does it require the user to move their hands from the keyboard to another input device such as a mouse.

[0440] Figure 8G Further illustrating the dynamic function row 104 in response to detecting Figure 8F 542). In some embodiments, the modal alert is a notification corresponding to an email, SMS, etc., received by the portable computing system 100 or the desktop computing system 200, an alert associated with an application (e.g., as a save dialog box, an exit confirmation dialog box, or a send email confirmation dialog box), etc. Figure 8G , modal alert 5156 displayed on dynamic feature row 104 prompts the user to save the draft email before closing window 580, and includes a "Save" affordance 5158, a "Don't Save" affordance 5160, and a "Cancel" affordance 5162. Affordances 5158, 5160, and 5162 are merely examples, and other affordances may be used to control or respond to the modal alert.

[0441] Figure 8H The main display 102 is shown in response to the detection Figure 8G Selecting the "Cancel" indicator using cursor 504 in the example above stops displaying the modal alert 5156 and maintains the display of the sub-window for composing a new email (e.g., Figure 8F shown). Figure 8H It also shows that in response to detecting Figure 8G 504, dynamic functionality row 104 stops displaying modal alert 5156 and displays persistent controls (i.e., affordances 516 and 534), multiple system-level affordances (i.e., affordances 518, 520, 522, 524, 526, 528, 530, and 532), and affordance 5152 corresponding to the mail application (i.e., affordances 516 and 534). Figure 8F shown).

[0442] Figure 9 The set of physical keys 106 ( Figure 1A to Figure 1B ) or from the set of physical keys 106 ( Figures 2A to 2D ) receives a signal corresponding to a specified physical key combination (e.g., alt+tab), the main display 102 displays the application selection window 5164. Figure 9, the application selection window 5164 displayed on the main display 102 includes: a mail application icon 506, which, when activated (e.g., via selection of a cursor 504), causes the main display 102 to display a window 580 corresponding to the mail application in the foreground; a media player application icon 510, which, when activated (e.g., via selection of a cursor 504), causes the main display 102 to display a window 554 corresponding to the media player application in the foreground; and an application A icon 512, which, when activated (e.g., via selection of a cursor 504), causes the main display 102 to display a window 536 corresponding to application A in the foreground. Figure 9 The primary display 102 is also shown displaying a cursor 504 at a location corresponding to the photos application icon 515 .

[0443] Figure 9 Further shown is a diagram of a method of responsively accessing the set of physical keys 106 ( Figure 1A to Figure 1B ) or from the set of physical keys 106 ( Figures 2A to 2D ) receives a signal corresponding to a specified physical key combination (e.g., alt+tab), the dynamic function row 104 displays the application selection window 5164 and at least one system-level enable indication (e.g., enable indication 542). Figure 9 , the application selection window 5164 displayed on the dynamic function row 104 includes: a mail application icon 506, which, when activated (e.g., via a tap contact), causes the main display 102 to display a window 580 corresponding to the mail application in the foreground; a media player application icon 510, which, when activated (e.g., via a tap contact), causes the main display 102 to display a window 554 corresponding to the media player application in the foreground; and an application A icon 512, which, when activated (e.g., via a tap contact), causes the main display 102 to display a window 536 corresponding to application A in the foreground.

[0444] Figure 10A The main display 102 is shown in response to detecting Figure 9 5. In FIG. 5, the photo application icon 515 is selected using the cursor 504 to display the photo application window 5166. For example, the window 5166 is overlaid on the windows 580, 554, and 536. Figure 10A , window 5166 displays a plurality of photos associated with the entire photo subsection of the user's photo library. Figure 10A , the entire photos subsection of the user's photo library is in focus on the primary display 102, as indicated by the bold display of "Photos" and photos A through L at least partially displayed in window 5166. Figure 10A, status tray 502 indicates that the photos application is running in the foreground, and application tray 514 also indicates that the photos application is running in the foreground based on the shadow behind photos application icon 515. Figure 10A The main display 102 is also shown displaying a cursor 504 at a position corresponding to photo B within window 5166.

[0445] Figure 10A Further shown is the dynamic function row 104 in response to detecting Figure 9 516 , and at least one system-level affordance (e.g., affordance 542 ) is displayed corresponding to the entire photo subsection of the user's photo library. Figure 10A , dynamic functionality row 104 includes: a search affordance 5168 for searching the user's photo library; a slideshow affordance 5170, which, when activated (e.g., via a tap contact), initiates a slideshow in full screen mode of the selected photo or all photos in all photo subsections of the user's photo library (e.g., as shown in FIG. 1 ); and a slideshow affordance 5170 for searching the user's photo library. Figure 10C and a slider indicator 5172 for scrolling through the photos displayed from the entire photo sub-portion of the user's photo library displayed in window 5166.

[0446] Figure 10B The main display 102 is shown in response to detecting Figure 10A In the example, use cursor 504 to select photo B and the selected photo B is displayed in window 5166. Figure 10B , the selected photo B is focused on the main display 102, as shown by the thick line surrounding photo B in window 5166. Figure 10B Primary display 102 is also shown displaying cursor 504 at a location corresponding to the slideshow affordance.

[0447] Figure 10B Further shown is the dynamic function row 104 in response to detecting Figure 10A 5. Selecting photo B with cursor 504 displays a set of affordances corresponding to the selected photo B. Figure 10B , the set of affordances corresponding to the selected photo B includes: a search affordance 5168 for searching the user's photo library; a zoom affordance 5174 for zooming to the selected photo B; a like affordance 5176 for indicating a like for the selected photo B; a slideshow affordance 5170 which, when activated (e.g., via a tap contact), initiates a slideshow of the selected photo or all photos in all photo subsections of the user's photo library in full screen mode (e.g., as shown in FIG. 1 ); Figure 10C); information affordance 5178 for displaying information corresponding to the selected photo B on the dynamic function row 104 and / or the main display 102, such as size, location, time / date, etc.; and editing affordance 5180, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display tools for editing the selected photo B (e.g., Figure 10F as shown) and / or causes the main display 102 to display an editing interface for editing the selected photo B; a photo adding enable indication 5182 for adding the selected photo B to the photo album; a sharing enable indication 5184 for sharing the selected photo B via one or more communication modes (e.g., social media networks, SMS, email, etc.); and a deletion enable indication 5186 for deleting the selected photo B from the user's photo library.

[0448] Figure 10C Shows the response to detecting Figure 10B Selecting the slideshow indicator with cursor 504 in the main display 102 displays a slideshow of photos from all photo subsections of the user's photo library in window 5188. Figure 10C , the main display 102 displays window 5188 containing photo B in full screen mode.

[0449] Figure 10C It also shows that in response to detecting Figure 10B In the slide show indication selected using cursor 504, dynamic function row 104 displays a plurality of thumbnails (i.e., thumbnails 5192-Y, 5192-Z, 5192-A, 5192-B, 5192-C, 5192-D, and 5192-E) corresponding to photos in the entire photo subsection of the user's photo library. Figure 10C , the thick line surrounding thumbnail 5192-B indicates that photo B is currently being displayed by the main display 102. Figure 10C , the dynamic function row 104 also displays a pause enable indication 5190, which, when activated (e.g., via a tap contact), causes the slideshow to be paused and also causes the main display 102 to exit full screen mode. Figure 10C The dynamic functionality row 104 is further shown receiving and detecting contact 5194 (e.g., a tap contact) at a location corresponding to the pause enable indication 5190.

[0450] Figure 10D The main display 102 is shown in response to detecting Figure 10C The selection of pause indicator 5190 is displayed in window 5166 in expanded view to display photo B. Figure 10D , the expanded view of the selected photo B is focused on the main display 102, as shown by the thick line surrounding the expanded view of photo B in window 5166.

[0451] Figure 10D Also shown is a dynamic function row 104 in response to detecting Figure 10C The selection of the pause indication 5190 in the display shows a set of indications corresponding to the photo B. Figure 10D , the set of enable representations corresponding to photo B include: a zoom enable representation 5174, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 and / or the main display 102 to display zoom controls that enable a user of the portable computing system 100 or the desktop computing system 200 to zoom in or out of photo B; a full screen enable representation 5194, which, when activated (e.g., via a tap contact), causes the main display 102 to display photo B in full screen mode; and a slide show enable representation 5170, which, when activated (e.g., via a tap contact), initiates a slide show of the selected photo or all photos in all photo subsections of the user's photo library in full screen mode. mapping; an information enable representation 5178, which is used to display information corresponding to the selected photo B, such as size, location, time / date, etc., on the dynamic function row 104 and / or the main display 102; an editing enable representation 5180, which, when activated (for example, via a tap contact), causes the dynamic function row 104 to display tools for editing the selected photo B and / or causes the main display 102 to display an editing interface for editing the selected photo B; a photo adding enable representation 5182, which is used to add the selected photo B to the album; and a sharing enable representation 5184, which is used to share the selected photo B via one or more communication modes (for example, social media networks, SMS, email, etc.). Figure 10D The dynamic functionality row 104 is further shown receiving and detecting contact 5196 (e.g., a tap contact) at a location corresponding to the full screen enable representation 5196.

[0452] Figure 10E The main display 102 is shown in response to detecting Figure 10D The selection of full screen indication 5196 displays photo B in full screen mode in window 5200. Figure 10E Also shown is the dynamic function row 104 in response to detecting Figure 10D The selection of the full screen affordance 5196 in the display minimizes affordance 5198. When activated (e.g., via a tap contact), minimizes affordance 5198 causes the primary display 102 to display photo B in an expanded view within window 5166 (e.g., Figure 10D shown). Figure 10E The dynamic function row 104 is further shown receiving and detecting contact 5201 (e.g., a tap contact) at a location corresponding to the edit enable indication 5180.

[0453] Figure 10F Dynamic function row 104 is shown in response to detecting Figure 10E In response to the selection of the editing indicator 5180, a set of editing tools 5205 is displayed.

[0454] Figure 10F Also shown is the response to detecting the Figure 10E In the selection of the edit enable indication 5180, the dynamic function row 104 displays the edit enable indication 5180 with a bold / bold outline and Figure 10E All other enable representations displayed in the (e.g., zoom enable representation 5174, minimize enable representation 5198, information enable representation 5178, photo add enable representation 5182, and share enable representation 5184) have increased translucency.

[0455] exist Figure 10F , the set of editing tools 5205 includes: a rotation tool 5202, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display controls for rotating the photo B in the window 5200 clockwise or counterclockwise; an enhancement tool 5204, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display controls for enhancing the photo B, such as applying a filter to the photo B, adjusting the brightness of the photo B, adjusting the saturation of the photo B, and the like; and a red-eye reduction tool 5206, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display controls for enhancing the photo B. 4 displays controls for reducing red-eye of the person in photo B; a straighten tool 5208, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display controls for straightening the orientation of photo B within window 5200; a crop tool 5210, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display controls for cropping photo B within window 5200; and a retouch tool 5212, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 to display controls for retouching photo B, such as for erasing and airbrush effects. Figure 10F The dynamic function row 104 is further shown receiving and detecting contact 5214 (e.g., a tap contact) at a location corresponding to the straighten tool 5208.

[0456] Figure 10G The dynamic function row 104 is shown in response to detecting Figure 10F Selection of the straighten tool 5208 in the window displays a set of controls 5209 for straightening the orientation of photo B within the window 5200. Figure 10G, the set of controls 5209 corresponding to the straighten tool 2208 includes a slider 5210 for adjusting the orientation of the photo B within the window 5200, and a completion indicator 5212 that, when activated (e.g., via a tap contact), causes the dynamic function row 104 to stop displaying the set of controls 5209 and to display the set of editing tools 5205 (e.g., Figure 10F shown). Figure 10G The dynamic functionality row 104 is also shown receiving and detecting contact 5216 (e.g., a tap contact) at a location corresponding to leaving the enable indication 516.

[0457] For example, a user of the portable computing system 100 or the desktop computing system 200 can adjust the orientation of photo B within the window 5200 by performing a left-to-right swipe / drag gesture or a right-to-left swipe / drag gesture originating at a location on the slider 5210 or within the set of controls 5209. For example, in response to detecting an upward swipe gesture on the dynamic function row 104, the dynamic function row 104 displays a set of controls corresponding to the crop tool 5210. In another embodiment, in response to detecting a downward swipe gesture on the dynamic function row 104, the dynamic function row 104 displays a set of controls corresponding to the red-eye reduction tool 5206.

[0458] Figure 10H The main display 102 is shown in response to detecting Figure 10G The selection to leave indication 516 is displayed and photo B is displayed in an expanded view in window 5166. Figure 10H The dynamic function row 104 is shown in response to detecting Figure 10G The selection of leaving the affordance 516 displays a set of affordances corresponding to photo B (e.g. Figure 10D shown). Figure 10H, the set of enable representations corresponding to photo B include: a zoom enable representation 5174, which, when activated (e.g., via a tap contact), causes the dynamic function row 104 and / or the main display 102 to display zoom controls that enable a user of the portable computing system 100 or the desktop computing system 200 to zoom in or out of photo B; a full screen enable representation 5194, which, when activated (e.g., via a tap contact), causes the main display 102 to display photo B in full screen mode; and a slide show enable representation 5170, which, when activated (e.g., via a tap contact), initiates a slide show of the selected photo or all photos in all photo subsections of the user's photo library in full screen mode. mapping; an information enable representation 5178, which is used to display information corresponding to the selected photo B, such as size, location, time / date, etc., on the dynamic function row 104 and / or the main display 102; an editing enable representation 5180, which, when activated (for example, via a tap contact), causes the dynamic function row 104 to display tools for editing the selected photo B and / or causes the main display 102 to display an editing interface for editing the selected photo B; a photo adding enable representation 5182, which is used to add the selected photo B to the album; and a sharing enable representation 5184, which is used to share the selected photo B via one or more communication modes (for example, social media networks, SMS, email, etc.).

[0459] Figure 10H Also shown is that in response to receipt of notification 5218 by portable computing system 100 or desktop computing system 200, dynamic functionality row 104 displays notification 5218 overlaid on affordances 5178, 5180, 5182, and 5184. Figure 10H , notification 5218 corresponds to an SMS, instant message, etc. sent by Suzie S. to the user of the portable computing system 100 or the desktop computing system 200, where the content of the notification asks "Want to see a movie tonight?". Figure 10H The dynamic functionality row 104 is further shown detecting a left-to-right swipe gesture with contact 5220 from a first location 5222-A to a second location 5222-B within the notification 5128.

[0460] Figure 11A Dynamic function row 104 is shown in response to detecting Figure 10H 5218 by swiping from left to right in the notification. Figure 11A The primary display 102 is also shown displaying a cursor 504 at a location corresponding to a web browser application icon 508 .

[0461] Figure 11B The main display 102 is shown in response to detecting Figure 11A504 is used to select the web browser application icon 508 and display the window 5224 of the web browser application. For example, the window 5224 is overlaid on the window 5166. Figure 11B , window 5224 includes controls for a web browser application, including navigation controls (e.g., previous page, next page, refresh, and add to favorites), an address bar, a search bar, a show all bookmarks affordance (e.g., similar to an open book), a show all open tabs affordance (e.g., a grid of six squares), and affordances for specific bookmarks A, B, and C. Figure 11B , window 5224 shows the main interface of the web browser application, including multiple affordances 5227 linking to favorite or most frequently visited websites A to H. Figure 11B , window 5224 of application A is focused on primary display 102. Figure 11B , status tray 502 indicates that a web browser application is running in the foreground, and application tray 514 also indicates that a web browser application is running in the foreground based on the shadow behind web browser application icon 508.

[0462] Figure 11B It also shows that in response to detecting Figure 11A 504, dynamic functionality row 104 displays affordance 5226 in addition to persistent controls (i.e., affordances 516 and 534) and a plurality of system-level affordances (i.e., affordances 518, 520, 522, 524, 526, 528, 530, and 532). When activated (e.g., via a tap contact), affordance 5226 causes dynamic functionality row 104 to display a set of controls for the web browser application (e.g., Figure 11C The indicators shown are 5230, 5232 and 5238 and address bar 5234). Figure 11B The dynamic function row 104 is further shown receiving and detecting contact 5228 (e.g., a tap contact) at a location corresponding to the enable indication 5226.

[0463] Figure 11C The dynamic function row 104 is shown in response to detecting Figure 11B In the example, a set of controls for a web browser application is displayed for selection of the indication 5226. Figure 11CIn the embodiment, the set of controls for the web browser application includes: an affordance 5230 for displaying in window 5224 the web pages that the web browser application visited before the currently displayed web page; an affordance 5232 for displaying in window 5224 the web pages that the web browser application visited after the currently displayed web page; an affordance 5238 for adding the web page currently displayed by the web browser application to a favorites list or a bookmark folder; and an address bar 5234 for displaying the URL of the web page currently displayed by the web browser application. Figure 11C , the address bar 5234 also includes a refresh indicator 5236 for refreshing the web page currently displayed by the web browser application. Figure 11C Primary display 102 is also shown displaying cursor 504 at a location corresponding to affordance 5227-A linking to website A.

[0464] Figure 11D The main display 102 is shown in FIG. Figure 11C After selecting affordance 5227-A corresponding to website A using cursor 504, the interface of label A is displayed in window 5224. Figure 11D , the interface of Tab A is in focus on the primary display 102, as indicated by the bold line surrounding Tab A and the bold text of Tab A. Figure 11D In the example, the interface of tab A displays the checkout page of website A (e.g., associated with URL: www.website_A.com / checkout). The checkout page corresponds to the user's virtual shopping cart, which includes the purchased items A and B. Figure 11D The main display 102 is also shown displaying a cursor 504 at a position corresponding to the purchase enable representation within the window 5224. Figure 11C The dynamic function row 104 is further shown displaying the URL of the checkout webpage of website A in the address bar 5234 (e.g., www.website_A.com / checkout).

[0465] Figure 11E The main display 102 is shown in response to detecting Figure 11D Selecting the purchase affordance with cursor 504 displays a modal alert 5240 overlaid on window 5224. Figure 11E, modal alert 5240 displayed on main display 102 prompts a user of portable computing system 100 or desktop computing system 200 to provide their fingerprint on dynamic function row 104 and also includes a cancel affordance 5242 that, when activated (e.g., via selection of cursor 504), causes the purchase to be canceled. For example, modal alert 5240 is displayed according to a security setting (e.g., default or user-specified) that requires a fingerprint to authenticate purchases initiated by portable computing system 100 or desktop computing system 200. For example, in some embodiments, the main display 102 and / or the dynamic function row 104 displays a modal alert prompting a user of the portable computing system 100 or desktop computing system 200 to provide their fingerprint on the dynamic function row 104 when logging into the portable computing system 100 or desktop computing system 200, when entering a password to access an application or website, when entering a password to decrypt data stored by the portable computing system 100 or desktop computing system 200, when deleting folders and / or data from the portable computing system 100 or desktop computing system 200, when taking other destructive actions, and so on.

[0466] Figure 11E The dynamic function row 104 is shown in response to detecting Figure 11D Selecting the purchase affordance using cursor 504 displays a modal alert 5240. Figure 11E In some embodiments, the modal alert 5240 displayed on the dynamic function row 104 prompts the user of the portable computing system 100 or desktop computing system 200 to provide their fingerprint in the fingerprint area 5244 of the dynamic function row 104, and also includes a cancel enable indication 5242 that, when activated (e.g., via a tap contact), causes the purchase to be canceled. In some embodiments, the dynamic function row 104 is configured to detect a fingerprint within the fingerprint area 5244 of the dynamic function row 104, which also corresponds to 5A to 11D 534 in the power control. In some embodiments, the dynamic function row 104 is configured to detect a fingerprint at any location within its touch-sensitive area. Figure 11E The dynamic function row 104 is further shown receiving and detecting contact 5246 (e.g., a press and hold gesture) within the fingerprint area 5244.

[0467] Figure 11F The main display 102 is shown in FIG. Figure 11E After contact 5246 is made in the fingerprint area 5244, the interface of label A is displayed in the window 5224. Figure 11F In the example, the interface of label A displays the receipt webpage of website A (e.g., associated with URL: www.website_A.com / reciept), indicating that after verifying Figure 11EAfter the user of the portable computing system 100 or desktop computing system 200 provides the fingerprint, the purchase is completed.

[0468] Figure 11F Dynamic function row 104 is also shown displaying interface 5248 associated with an incoming voice call from C. Chung and at least one system-level affordance (e.g., affordance 542) in response to portable computing system 100 or desktop computing system 200 receiving the incoming voice call. Figure 11F In the embodiment, the interface 5248 includes a first enabling indication 5250 for answering an incoming call and a second enabling indication 5252 for rejecting an incoming call. Figure 11F Dynamic function row 104 is further shown receiving and detecting contact 5254 (e.g., a tap contact) at a location corresponding to first affordance 5250. For example, upon detecting selection of first affordance 5250, a communication connection (e.g., VoIP) is established between C. Chung and the user of portable computing system 100 or desktop computing system 200 for the voice call.

[0469] Figure 11G Shows the dynamic function row 104 when detecting Figure 11F After selection of first affordance 5250 in , interface 5256 associated with an ongoing voice call between C. Chung and a user of portable computing system 100 or desktop computing system 200 and at least one system-level affordance (e.g., affordance 542) is displayed. Figure 11G , interface 5256 includes an affordance 5258 for ending the voice call and an indicator of the total time of the voice call (e.g., 7 minutes and 29 seconds). In some embodiments, while the voice call is in progress, the affordance associated with the focus of primary display 102 is not displayed on dynamic function row 104. In some embodiments, after the voice call continues for a predefined amount of time, interface 5256 is displayed in compact mode, and the affordance associated with the focus of primary display 102 can be displayed on dynamic function row 104. Figure 11G The dynamic function row 104 is also shown receiving and detecting contact 5260 at a location corresponding to the enable indication 5258.

[0470] Figure 11H The main display 102 is shown displaying the interface of tab B in window 5224, where tabs A, B, and C are open in the web browser application. Figure 11H In the example, the interface of label B shows the homepage of website B (for example, the associated URL is: www.website_B.com / home). Figure 11H, the interface of Tab B is in focus on the primary display 102 , as indicated by the bold line surrounding Tab B and the bold text of Tab B.

[0471] Figure 11H Also shown is the dynamic function row 104 when detecting Figure 11G After selecting the option 5258, the display interface 5256 is stopped. Figure 11H In the example, the dynamic function row 104 includes the URL of the home page of website B in the address bar 5234 (e.g., www.website_B.com / home). Figure 11H , the dynamic function row 104 also includes: an enable representation 5262-A, which, when activated (for example, via a tap contact), causes the main display 102 to display the interface of label A, and also causes the dynamic function row 104 to display the URL corresponding to label B in the address bar 5234; and an enable representation 5262-B, which, when activated (for example, via a tap contact), causes the main display 102 to display the interface of label C, and also causes the dynamic function row 104 to display the URL corresponding to label C in the address bar 5234.

[0472] Figure 12A In response to the portable computing system 100 or the desktop computing system 200 receiving the notification 5264, the main display 102 displays the notification 5264 overlaid on the window 5264. Figure 12A ...

Claims

1. An electronic device, comprising: one or more processors; A first display portion; A second display portion; a hinge connecting the first display portion and the second display portion; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a calendar application on said first display portion; receiving a request to display information about a first event associated with the calendar application; and In response to receiving the request: Displaying event details of the first event on the first display portion, the event details including the start time and the end time of the event; and displaying an enable indication for navigating the calendar application on the second display portion, the enable indication indicating a time range including at least the start time and the end time.

2. The electronic device of claim 1, wherein the one or more programs include instructions for: displaying a first view on the second display portion while displaying a current view of the calendar application on the first display portion, wherein the first view is different from the current view; detecting, via the first display portion, an interaction with the calendar application; as well as In response to detecting the interaction, updating the display of the second display portion from the first view to a second view.

3. The electronic device of claim 2, wherein the instructions for updating the display of the second display portion from the first view to the second view further include instructions for: Different time units are displayed in the first view and the second view.

4. The electronic device of claim 2, wherein the instructions for updating the display of the second display portion from the first view to the second view include instructions for: Different options for editing events associated with the calendar application are displayed in the first view and the second view.

5. The electronic device of claim 2, wherein the instructions for updating the display of the second display portion from the first view to the second view include instructions for: An option for changing the location of the first event is displayed in the second view, wherein the option is not displayed until the interaction with the calendar application is detected.

6. The electronic device of claim 2, wherein the instructions for updating the display of the second display portion from the first view to the second view include instructions for: An option for changing the calendar is displayed in the second view, wherein the option is not displayed until the interaction with the calendar application is detected.

7. The electronic device of claim 1, wherein the one or more programs include instructions for: displaying an application user interface of the calendar application on the second display portion while displaying a first user interface of the calendar application on the first display portion; detecting, via the second display portion, an interaction with the application user interface displayed on the second display portion; as well as In response to detecting the interaction with the application user interface, updating the display of the first display portion from the first user interface to a second user interface different from the first user interface.

8. The electronic device of claim 1, wherein the one or more programs include instructions for: Displays a calendar and one or more system-level user interface elements simultaneously.

9. The electronic device of claim 1, wherein the one or more programs include instructions for: detecting, via the second display portion, an input at the affordance that modifies the time range; and In response to detecting the input: modify at least one of the start time and the end time of the first event according to the input; and A modified time range for the event is displayed on the first display portion based on the input.

10. An electronic device according to claim 9, wherein the input that modifies the time range is a press input, which maintains contact with the enable representation for more than a threshold amount of time and then moves at least a portion of the enable representation on the second display portion.

11. An electronic device according to claim 9, wherein the input for modifying the time range is a swipe gesture, which moves across the second display portion and causes the electronic device to select a new start time and a new end time for the first event, wherein the new start time and the new end time correspond to a time slot that is the same as the duration covered by the start time and the end time.

12. The electronic device of claim 1, wherein the second display portion is a touch-sensitive display.

13. A method comprising: On an electronic device having a first display portion, a second display portion, and a hinge connecting the first display portion and the second display portion: displaying a calendar application on the first display portion; receiving a request to display information about a first event associated with the calendar application; and In response to receiving the request: displaying event details of the first event on the first display portion, the event details including a start time and an end time of the first event; and An affordance for navigating the calendar application is displayed on the second display portion, the affordance indicating a time range including at least the start time and the end time.

14. The method of claim 13, wherein the electronic device comprises one or more processors and a memory, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs further comprising instructions for executing claims 2-12.

15. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device comprising a first display portion, a second display portion, and a hinge connecting the first display portion and the second display portion, causes the electronic device to: displaying a calendar application on the first display portion; receiving a request to display information about a first event associated with the calendar application; and In response to receiving the request: displaying event details of the first event on the first display portion, the event details including a start time and an end time of the first event; and An affordance for navigating the calendar application is displayed on the second display portion, the affordance indicating a time range including at least the start time and the end time.

16. The computer-readable storage medium of claim 15, the one or more programs further comprising instructions for executing claims 2-12.

17. An electronic device comprising: one or more processors; A first display portion; A second display portion; a hinge connecting the first display portion and the second display portion; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: receiving a request to search within content displayed on the first display portion; In response to receiving the request: displaying, on the first display portion, a plurality of search results responsive to the search, wherein a focus is on a first search result of the plurality of search results; displaying on the second display portion respective representations each corresponding to a respective search result from the plurality of search results; detecting, via the second display portion, a touch input selecting a representation of the corresponding representations, the representation corresponding to a second search result of the plurality of search results that is different from the first search result; as well as In response to detecting the touch input, changing focus on the first display portion to a second search result.

18. An electronic device comprising: one or more processors; A first display portion; A second display portion; a hinge connecting the first display portion and the second display portion; as well as A memory storing one or more programs configured to be executed by one or more processors, the one or more programs including instructions for: detecting a new connection between the computing system and an external device different from the computing system; as well as In response to detecting the new connection, a plurality of affordances corresponding to functions available through the external device are displayed on the second display portion.

19. An electronic device comprising: one or more processors; A first display portion; A second display portion; a hinge connecting the first display portion and the second display portion; as well as A memory for storing one or more programs configured to be executed by one or more processors, the one or more programs including instructions for: displaying, on the first display portion, a user interface for an application executed on the electronic device; detecting a first input at a particular location within the user interface; as well as In response to detecting the first input, a set of affordances, each corresponding to a different character, is displayed on the second display portion.

Images