Systems and methods for enabling low vision users to interact with a touch-sensitive secondary display

Abstract

The present invention is entitled "System and method for enabling low vision users to interact with a touch-sensitive secondary display." Disclosed herein is a system and method for enabling low vision users to interact with a touch-sensitive secondary display. An exemplary method includes: displaying a first user interface for an application on a primary display, and displaying a second user interface on a touch-sensitive secondary display, the second user interface including multiple application-specific enable representations that control functions of the application. Each corresponding enable representation is displayed at a first display size. The method also includes: detecting input contacting at least one application-specific enable representation via the secondary display. In response to detecting the input, and while it remains in contact with the secondary display, the method includes: (i) continuing to display the first user interface on the primary display, and (ii) displaying an enlarged representation of at least one application-specific enable representation on the primary display. The enlarged representation is displayed at a second display size that is larger than the first display size.

Description

Technical Field

[0001] The disclosed embodiments relate to touch-sensitive secondary display devices and, more particularly, to improved techniques for enabling low vision users to interact with a touch-sensitive secondary display. Background Art

[0002] Occlusion problems often prevent many users from understanding and using features available through a touch-sensitive input device that is also used to display affordances. For example, a user of a touch-sensitive secondary display that may be located above a physical keyboard may not be able to view certain affordances because their fingers occlude or cover the affordances when they are displayed at the secondary display. In addition, affordances displayed in such secondary displays are typically small. These problems are particularly severe for low vision users, who may have difficulty seeing certain affordances displayed at the secondary display, and these difficulties are exacerbated and amplified by the occlusion problems described above. Summary of the invention

[0003] The embodiments described herein address the above shortcomings by providing the following devices and methods: the devices and methods seamlessly provide an enlarged representation of the enabling representation at the primary display in response to user interaction at the touch-sensitive secondary display device, thereby enabling low vision users to interact with the touch-sensitive secondary display (i.e., these users are now able to view the enlarged representation at the primary display and then determine the appropriate input to be provided at the secondary display). Such devices and methods also reduce the amount of mode switching required by the user (e.g., moving hands between keyboard and mouse, and moving eyes from keyboard to display), thereby reducing the number of inputs required to activate the desired function (e.g., reducing the number of inputs required to select menu options, as explained in more detail below). Such devices and methods also make more information available on a limited screen (e.g., the touch-sensitive secondary display is used to provide more information to the user, and the information is effectively presented using limited screen space). Such devices and methods also provide an improved human-computer interface by providing an emphasis effect to make information more discernible on a touch-sensitive secondary display; providing continuous interaction so that continuous input from a user directed to a touch-sensitive secondary display or a primary display causes the device to provide an output, which is then used to facilitate further input from the user (e.g., an affordance is displayed at the touch-sensitive secondary display, which allows the user to quickly preview how the information will be reproduced on the primary display by providing input at the touch-sensitive secondary display, as described below); and requiring less interaction from the user to achieve the desired result. In some cases, the touch-sensitive secondary display is also referred to herein as a dynamic function row (and vice versa). For these reasons and those discussed below, the devices and methods described herein reduce power usage and improve battery life of electronic devices.

[0004] According to some embodiments, a method is performed at a computing system having one or more processors; a first housing including a main display; a memory; and a second housing (which is different from the first housing) that at least partially includes a touch-sensitive secondary display that is different from the main display (as described below, the second housing and the touch-sensitive secondary display can be components of any device that includes a display smaller than the main display, for example, the touch-sensitive secondary display is part of a wearable computing device such as a watch, or the touch-sensitive secondary display is located above a physical keyboard in the second housing). The method includes: displaying a first user interface for an application on the main display. The method also includes: displaying a second user interface on the touch-sensitive secondary display, the second user interface including multiple application-specific enable representations that control functions available within the application, wherein each of the multiple application-specific enable representations is displayed at a first display size. The method further includes: detecting input contacting at least one of the multiple application-specific enable representations via the touch-sensitive secondary display. In response to detecting an input, and while the input maintains contact with the touch-sensitive secondary display, the method includes: (i) continuing to display a first user interface for an application on the primary display, and (ii) displaying an enlarged representation of at least one application-specific enabled representation on the primary display, wherein the enlarged representation of the at least one application-specific enabled representation is displayed at a second display size that is larger than the first display size.

[0005] In some cases, users of a computing system (specifically, users with low vision) may not be able to accurately view icons or affordances displayed at a small display size (such as those displayed on a smartwatch). Populating a touch-sensitive secondary display with application-specific affordances, and then displaying an enlarged representation of one of these affordances on a larger primary display in response to a single input, provides these users with clear visual feedback indicating which affordance they can select. Providing this improved visual feedback to the user enhances the operability of the device and makes the human-computer interface more efficient (e.g., by helping the user provide correct inputs and reducing user errors when operating / interacting with affordances displayed on the secondary display). In addition, allowing these users to accurately view affordances displayed on a small screen enables continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the additional time and effort required to correct these errors.

[0006] According to some embodiments, a method is performed at a computing system having one or more processors; a first housing including a primary display; a memory; and a second housing (which is different from the first housing) that at least partially contains a touch-sensitive secondary display that is different from the primary display (as described below, the second housing and the touch-sensitive secondary display can be components of any device that includes a display that is smaller than the primary display, for example, the touch-sensitive secondary display is part of a wearable computing device such as a watch, or the touch-sensitive secondary display is located above a physical keyboard in the second housing). The method includes: operating the touch-sensitive secondary display in an accessible mode. When operating a touch-sensitive secondary display in an accessible mode, the method includes: displaying a first user interface for an application on the primary display; displaying a second user interface on the touch-sensitive secondary display, the second user interface including: (i) multiple application-specific enable representations that control functions available within the application, and (ii) at least one system-level enable representation that controls system-level functions, wherein each of the multiple application-specific enable representations and at least one system-level enable representation are displayed at a first display size; detecting input that contacts at least one of the multiple application-specific enable representations via the touch-sensitive secondary display; and in response to detecting the input, and while the input remains in contact with the touch-sensitive secondary display: (A) continue to display the first user interface for the application on the primary display, and (B) display an enlarged representation of at least one application-specific enable representation on the primary display, wherein the enlarged representation of at least one application-specific enable representation is displayed at a second display size that is larger than the first display size.

[0007] In some cases, low vision users of a computing system rely on memorizing key locations on a keyboard so that they can accurately provide input to the computing system. For computing systems that include a touch-sensitive secondary display with frequently changing affordances (e.g., affordances that change to provide functionality useful to the user based on what they are doing within a particular application), these users cannot rely solely on memory to provide accurate input. Displaying an enlarged representation of at least one affordance of the application-specific affordances improves the operability of the computing system because the low vision user is able to interact with controls available at the touch-sensitive secondary display, which may be too small for the low vision user (or may be obscured because the user's fingers are covering the displayed controls) to accurately view. In this way, the low vision user is able to take advantage of the improved human-computer interface by, for example, continuously interacting with the touch-sensitive secondary display (rather than constantly correcting erroneous inputs).

[0008] According to some embodiments, a method is performed at a computing system having one or more processors; a first housing including a main display; a memory; and a second housing (which is different from the first housing) that at least partially includes a touch-sensitive secondary display that is different from the main display (as described below, the second housing and the touch-sensitive secondary display can be components of any device that includes a display smaller than the main display, for example, the touch-sensitive secondary display is part of a wearable computing device such as a watch, or the touch-sensitive secondary display is located above a physical keyboard in the second housing). The method includes: displaying a first user interface for an application on the main display. The method also includes: displaying a second user interface on the touch-sensitive secondary display, the second user interface including: (i) multiple application-specific affordances that control functions available within the application, and (ii) at least one system-level affordance that controls system-level functions. The method further includes: detecting a first input on a first application-specific affordance among the multiple application-specific affordances via the touch-sensitive secondary display. While a first input maintains contact with a first application-specific enable representation, the method includes: (i) detecting a second input that is not over the first application-specific enable representation via a touch-sensitive secondary display, and (ii) activating the first application-specific enable representation in response to detecting the second input.

[0009] Allowing activation of an affordance that is in contact with an input in response to a tap gesture that is not over an affordance (a "split tap gesture") enhances the operability of the device and makes the human-computer interface more efficient (e.g., by allowing the user to place a first finger over a desired affordance and then use a different finger to perform a selection of the desired affordance, thereby ensuring that only desired affordances are activated and helping to minimize erroneous selections / activations). Additionally, allowing the user to freely move their first finger around the touch-sensitive secondary display (without selecting an affordance) enables the user to maintain continuous interaction with the touch-sensitive secondary display (by exploring which affordances are displayed at the touch-sensitive secondary display), which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental selection of affordances) and the additional time and effort required to correct these errors.

[0010] According to some embodiments, a computing system includes a first housing having a main display unit configured to display a user interface; a second housing (different from the first housing) at least partially containing a touch-sensitive secondary display different from the main display and configured to receive user input and display a user interface; and a processing unit in communication with the main display unit and the touch-sensitive secondary display unit. The processing unit is configured to: display a first user interface for an application on the main display; display a second user interface on the touch-sensitive secondary display, the second user interface including a plurality of application-specific affordances that control functions available within the application, and each of the plurality of application-specific affordances is displayed at a first display size; detect input contacting at least one of the plurality of application-specific affordances via the touch-sensitive secondary display; and in response to detecting the input, and while the input remains in contact with the touch-sensitive secondary display: (i) continue to display the first user interface for the application on the main display, and (ii) display an enlarged representation of at least one application-specific affordance on the main display, wherein the enlarged representation of at least one application-specific affordance is displayed at a second display size that is larger than the first display size.

[0011] According to some embodiments, a computing system includes a first housing having a main display unit configured to display a user interface; a second housing (which is different from the first housing) that at least partially contains a touch-sensitive secondary display that is different from the main display and is configured to receive user input and display a user interface; and a processing unit that communicates with the main display unit and the touch-sensitive secondary display unit. The processing unit is configured to: operate the touch-sensitive secondary display in an accessible mode; when operating the touch-sensitive secondary display in the accessible mode: display a first user interface for an application on the primary display; display a second user interface on the touch-sensitive secondary display, the second user interface comprising: (i) multiple application-specific enabling representations that control functions available within the application, and (ii) at least one system-level enabling representation that controls system-level functions, wherein each of the multiple application-specific enabling representations and at least one system-level enabling representation are displayed at a first display size; detect input contacting at least one of the multiple application-specific enabling representations via the touch-sensitive secondary display; and in response to detecting the input, and while the input remains in contact with the touch-sensitive secondary display: (A) continue to display the first user interface for the application on the primary display, and (B) display an enlarged representation of at least one application-specific enabling representation on the primary display, wherein the enlarged representation of at least one application-specific enabling representation is displayed at a second display size that is larger than the first display size.

[0012] According to some embodiments, a computing system includes a first housing having a main display unit configured to display a user interface; a second housing (different from the first housing) at least partially containing a touch-sensitive secondary display different from the main display and configured to receive user input and display a user interface; and a processing unit in communication with the main display unit and the touch-sensitive secondary display unit. The processing unit is configured to: display a first user interface for an application on the main display; display a second user interface on the touch-sensitive secondary display, the second user interface including: (i) multiple application-specific affordances that control functions available within the application, and (ii) at least one system-level affordance that controls system-level functions, detecting a first input above a first application-specific affordance in the multiple application-specific affordances via the touch-sensitive secondary display; and when the first input remains in contact with the first application-specific affordance: (i) detecting a second input that is not on the first application-specific affordance via the touch-sensitive secondary display, and (ii) activating the first application-specific affordance in response to detecting the second input.

[0013] According to some embodiments, a computing system includes one or more processors; a first housing having a primary display; a second housing at least partially containing a touch-sensitive secondary display different from the primary display and optionally containing one or more sensors for detecting the intensity of contact with a touch-sensitive secondary surface; and a memory storing one or more programs, the one or more programs being configured for execution by the one or more processors, and the one or more programs including instructions for performing or causing the execution of operations or of any of the methods described herein. According to some embodiments, a computer-readable storage medium stores therein instructions, which, when executed by the computing system, cause the computing system to perform or cause the execution of operations of any of the methods described herein. According to some embodiments, a graphical user interface on a primary display of a computing system is provided, and the graphical user interface includes one or more of the elements displayed in any of the methods described herein, the elements being updated in response to input, as described in any of the methods described herein. According to some embodiments, a computing system includes a device for performing or causing the execution of operations of any of the methods described herein. According to some embodiments, an information processing device used in a computing system includes a device for performing or causing the execution of operations of any of the methods described herein.

[0014] Thus, a computing system including a primary display and a touch-sensitive secondary display is provided with faster, more efficient and useful / user-friendly methods and interfaces to enable a low vision user to interact with an enabling representation displayed at the touch-sensitive secondary display, thereby improving the operability of the computing system by, for example, allowing the user to continuously interact with the touch-sensitive secondary display. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals refer to corresponding parts throughout the drawings.

[0016] Figure 1A is an illustrative diagram of a portable computing system (e.g., a laptop computer) according to some embodiments.

[0017] Figure 1B According to some embodiments Figure 1A An illustrative diagram of the main body of a portable computing system in FIG.

[0018] Figure 2A is an illustrative diagram of a first specific implementation of a desktop computing system according to some embodiments.

[0019] Figure 2B is an illustrative diagram of a second specific implementation of a desktop computing system according to some embodiments.

[0020] Figure 2C is an illustrative diagram of a third specific implementation of a desktop computing system according to some embodiments.

[0021] Figure 2D is an illustrative diagram of a fourth specific implementation of a desktop computing system according to some embodiments.

[0022] Figure 3A is a block diagram of an electronic device according to some embodiments.

[0023] Figure 3B According to some embodiments, Figure 3A A block diagram of the components of event processing.

[0024] Figure 3C-Figure 3E Examples of dynamic intensity thresholds are shown according to some embodiments.

[0025] Figure 4 is a block diagram of a peripheral electronic device according to some embodiments.

[0026] Figure 5A-Figure 5N is a schematic diagram of a primary display and a secondary display according to some embodiments, which is used to illustrate an exemplary user interface for enabling a low vision user to interact with a touch-sensitive secondary display.

[0027] Figure 6A-6C A flowchart is shown of a method for enabling a low vision user to interact with a touch-sensitive secondary display according to some embodiments.

[0028] Figure 7A-7CA flowchart is shown of a method for enabling a low vision user to interact with a touch-sensitive secondary display according to some embodiments.

[0029] Figure 8A-8C A flowchart is shown of a method for enabling a low vision user to interact with a touch-sensitive secondary display according to some embodiments.

[0030] Figure 9-11 A functional block diagram of a computing system according to some embodiments is shown. DETAILED DESCRIPTION

[0031] Figure 1A-1B , Figure 2A-2D , Figure 3A-3E and Figure 4 A description of an exemplary apparatus is provided. Figure 5A-Figure 5N is a schematic diagram of a display for illustrating an exemplary user interface for enabling a low vision user to interact with a touch-sensitive secondary display. Figure 6A-6C , Figure 7A-7C and Figure 8A-8C is a flow chart of a method of enabling a low vision user to interact with a touch-sensitive secondary display. Figure 5A-Figure 5N The user interface in Figure 6A-6C , Figure 7A-7C and Figure 8A-8C The method and / or process in.

[0032] Exemplary Devices and Systems

[0033] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be put into practice without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail to unnecessarily obscure the various aspects of the embodiments.

[0034] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish an element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact.

[0035] The terms used in the description of the various embodiments described herein are for the purpose of describing specific embodiments only and are not intended to be limiting. As used in the description of the various embodiments described and in the appended claims, the singular forms "one, "a", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the items listed in association. It will also be understood that the terms "include", "comprises", "contains", and / or "have" when used in this specification specify the presence of stated features, integers, steps, operations, elements, and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0036] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0037] Figure 1A is an illustrative diagram of a portable computing system 100 according to some embodiments. The portable computing system 100 can be, for example, a laptop computer such as device, or any other portable computing device. The portable computing system 100 includes: (A) a display portion 110 (also referred to herein as a first housing 110 or housing 110) having a main display 102; and (B) a main body portion 120 (also referred to herein as a second housing 120 or housing 120) having a dynamic function row 104; a set of physical (i.e., movably actuatable) keys 106; and a touch pad 108 partially contained within the same housing. The display portion 110 is typically mechanically, electrically, and communicatively coupled to the main body portion 120 of the portable computing system 100. For example, the portable computing system 100 may include a hinge to allow the display portion 110 to rotate relative to the main body portion 120. The portable computing system 100 includes one or more processors and a memory storing one or more programs for execution by the one or more processors to perform any of the embodiments described herein. In some embodiments, reference is made to Figure 1BThe dynamic function row 104 described in more detail is a touch screen display that detects user touch input and selection using resistive sensing, acoustic sensing, capacitive sensing, optical sensing, infrared sensing, etc. In some embodiments, the main display 102 of the display portion 110 is also a touch screen display.

[0038] Figure 1B is an illustrative diagram of a main body portion 120 of a portable computing system 100 according to some embodiments. The main body portion 120 includes a set of physical keys 106 (also referred to herein as "physical keys 106" and "keyboard 106"), a dynamic function row 104, and a touchpad 108 partially contained within the same housing. In some embodiments, the dynamic function row 104, which is a touch screen, replaces the function row of the set of physical keys 106, thereby allowing the space occupied by the set of physical keys 106 to be reduced, thereby allowing a smaller overall main body portion 120 or allowing other portions, such as the touchpad 108 to be larger. In some embodiments, the dynamic function row 104 is approximately 18 inches in length relative to the major dimension of the set of physical keys 106. Although referred to as a "row" for ease of explanation, in some other embodiments, the main body portion 120 includes Figure 1A The touch screen of the dynamic function row 104 in the embodiment may take any other form, such as a square, a circle, a plurality of rows, a column, a plurality of columns, a plurality of separate sectors, etc. Figure 1A-1B A dynamic function row 104 is shown replacing a function row of a set of physical keys 106 , but in some other embodiments, the dynamic function row 104 may additionally and / or alternatively replace a numeric keypad portion, an editing / function portion, etc. of the set of physical keys 106 .

[0039] Each physical key of a set of physical keys 106 has at least one associated input. The input may be a printable character, a non-printable character, a function, or other input. The input associated with a physical key may be displayed by letters, words, symbols, or other markings displayed (e.g., printed) on the surface of a key in Latin script, Arabic characters, Chinese characters, or any other script. For example, a particular physical key indicated at 138 is associated with the alphabetic character "z", as indicated by the letter z shown on the key. In another example, a physical key marked with the word "command" may be associated with a command function. For example, according to ISO / IEC 9995, ANSI-INCITS 154-1988, JIS X 6002-1980, or other similar standards, a set of physical keys 106 is associated with QWERTY, Dvorak, or other keyboard layouts (e.g., standard, extended, or compact) having alphanumeric, numeric, and / or editing / function portions.

[0040] When the key has been activated by the user, a signal corresponding to the input associated with the physical key may be transmitted by the portable computing system 100 (or Figure 2A-2DThe computing device 202 or Figure 2A-2B In the illustrated example, each key of the set of physical keys 106 includes two plates and a spring. A user may activate a key by pressing the key, which compresses the spring. When the spring is compressed, the two plates may contact, thereby allowing current to flow through the connected plates. In response to the flow of current through the connected plates, an input corresponding to the key may be provided to the processor. For example, in response to Figure 2C Upon activation of a set of keys 106 of the peripheral keyboard 206, input corresponding to the activated keys will be provided to the computing device 202. It should be appreciated that other systems for movably actuated keys may be used.

[0041] In some embodiments, the dynamic function row 104 is a touch screen display (the dynamic function row is also referred to herein as a touch-sensitive secondary display 104) that displays one or more user-selectable symbols 142 (also sometimes referred to herein as "user interface elements," "user interface components," "indicators," "buttons," or "soft keys"). For example, the dynamic function row 104 replaces the function row keys on a typical keyboard. A user can select a specific one of the one or more user-selectable symbols 142 by touching a location on the touch screen display corresponding to the specific one of the one or more user-selectable symbols 142. For example, a user can select a user-selectable symbol indicated by the magnifying glass symbol 144 by tapping the dynamic function row 104 so that the user's finger contacts the dynamic function row 104 at the location of the magnifying glass indicator 214. In some embodiments, the tap contact or tap gesture includes a downward touch of the contact and a lift-off of the contact within a predetermined amount of time (e.g., 250ms, etc.). In some embodiments, the touch screen display of the dynamic function row 104 is implemented using resistive sensing, sound sensing, capacitive sensing, optical sensing, infrared sensing, etc. to detect user input and selection.

[0042] When the user selects a specific one of the one or more user-selectable symbols 142, a signal corresponding to the specific one of the one or more user-selectable symbols 142 is generated by the dynamic function row 104. For example, when the user taps "esc" on the dynamic function row 104, the dynamic function row 104 transmits a signal indicating the user input corresponding to the exit function to the portable computing system 100 (or Figure 2A-2D A processor of a computing device 202 in the embodiment of the present invention.

[0043] In some embodiments, when a particular one of the one or more user-selectable symbols 142 is selected, the dynamic function row 104 transmits a signal corresponding to the location at which the particular one of the one or more user-selectable symbols 142 is displayed on the touch screen display to the portable computing system 100 (or Figure 2A-2DFor example, the dynamic function row 104 may transmit a signal including a position value (0 to 20) based on the selected position on the touch screen display of a particular one of the one or more user-selectable symbols 142. Figure 1B In the illustrative example of , the "esc" symbol may have a position value of 0, the magnifying glass symbol 144 may have a position value of 16, and so on. The portable computing system 100 (or Figure 2A-2D A processor of the computing device 202 of the embodiment of the present invention may receive a signal indicating the position value of the selected user-selectable symbol and interpret the position value using context information, such as displayed on the primary display 102 of the display portion 110 (or the peripheral display device 204, Figure 2A-2D ) that is currently active or has focus.

[0044] Each of the one or more user-selectable symbols 142 may include an indicator, such as a symbol (e.g., a magnifying glass symbol as shown at 144), an abbreviated word (e.g., "esc"), an unabbreviated word, a character, an image, an animated image, a video, etc. In some embodiments, a respective one of the one or more user-selectable symbols 142 is capable of receiving one or more user inputs.

[0045] An input may be associated with each of the one or more user-selectable symbols 142. The input may be a function, a character, a numeric value, etc. A respective one of the one or more user-selectable symbols 142 may include an indicator corresponding to the input for the respective one of the one or more user-selectable symbols 142. For example, in Figure 1B , a user-selectable symbol with the abbreviation word "esc" indicates to the user that an exit function is associated with the user-selectable symbol. When the user selects the user-selectable symbol, the function associated with one or more user-selectable symbols 142 can be activated. For example, when the user selects the user-selectable symbol with the indicator "esc", the exit function can be activated. The activation of the function can have different effects, depending on the portable computing system 100 (or Figure 2A-2D For example, when the main display 102 (or the peripheral display device 204, Figure 2A-2D ), activating the leave function on the dynamic function row 104 can close the dialog box. In another example, when the portable computing system 100 (or Figure 2A-2D When a processor of a computing device 202 in the processor is executing a gaming application, activating the leave function on the dynamic function row 104 can pause the game.

[0046] In some embodiments, a function may be associated with a combination of movably actuated keys and / or user-selectable symbols. For example, simultaneous actuation of a command key and a "c" key (i.e., command+c) may be associated with a "copy" function. In another example, simultaneous actuation of a command key and a selection of a user-selectable symbol with an indicator "esc" (i.e., command+esc) may activate a function to open a specific application, such as a media player application. In yet another example, simultaneous selection of two user-selectable symbols (e.g., a user-selectable symbol with an indicator "esc" and a user-selectable symbol 144 with a magnifying glass indicator) may result in activation of a function such as a professional search function.

[0047] In some embodiments, a first subset 146 of one or more user-selectable symbols 142 of a dynamic function row 104 may be associated with a set of functions, and a second subset 148 of one or more user-selectable symbols 142 of a dynamic function row 104 may be associated with a second set of functions. For example, the user-selectable symbols in the first subset 146 may be global functions (e.g., system-level functions or affordances), and the user-selectable symbols in the second subset 148 may be application-specific functions. Thus, when the focus shifts from a first element of a graphical user interface displayed on the primary display 102 (e.g., a first window corresponding to an internet browser application) to a second element of the graphical user interface (e.g., a second window corresponding to an email application), the user-selectable symbols in the second subset 148 change. Conversely, when the focus shifts from the first element of the graphical user interface to the second element of the graphical user interface, the user-selectable symbols in the first subset 146 are maintained.

[0048] In some embodiments, the user-selectable symbols in the second subgroup 148 are determined based on an active user interface element in focus displayed on the primary display 102. In some embodiments, the term "in focus" can refer to an active element of a user interface (e.g., a window associated with an application, a particular toolbar or menu associated with an application, or an operating system) that is currently in the foreground and actively running, or that can be controlled by input received from a user of the computing system, such as key presses, mouse clicks, voice commands, gesture movements, etc.

[0049] In some embodiments, a first subset 146 of one or more user-selectable symbols 142 corresponding to global user-selectable symbols occupies a first area of ​​the dynamic function row 104 (e.g., the left half of the dynamic function row 104), and a second subset 148 of one or more user-selectable symbols 142 occupies a second area of ​​the dynamic function row 104 (e.g., the right half of the dynamic function row 104). It should be appreciated that other proportions of the dynamic function row 104 may be allocated to the first subset 146 and the second subset 148. In some embodiments, when no application has focus, the second area of ​​the dynamic function row 104 may not include any user-selectable symbols. In some embodiments, the dynamic function row 104 includes three or more subsets of user-selectable symbols. In some embodiments, the dynamic function row 104 includes a single group of user-selectable symbols that are not divided into subsets. Although in Figure 1B A single row of user-selectable symbols is shown in the dynamic function row 104 in FIG, but it should be understood that the dynamic function row 104 may include multiple rows of user-selectable symbols.

[0050] In some embodiments, the change in focus changes the focus displayed on the main display 102 (or peripheral display device 204, Figure 2A-2D ) is active and which element will receive user input. User input may be received from a keyboard, mouse, touchpad, or other user input device. Additionally and / or alternatively, in some embodiments, a change in focus changes the focus displayed on the main display 102 (or peripheral display device 204, Figure 2A-2D ) is an element shown in the foreground of a graphical user interface.

[0051] In some embodiments, the change in focus is in response to user input, such as in response to a display displayed on the main display 102 (or peripheral display device 204, Figure 2A-2D ) or occurs in response to a user selection of an element of a graphical user interface (e.g., a different window) on the portable system 100 (or ), or in response to a user selection of a user-selectable symbol (e.g., one of the affordances / symbols displayed on the dynamic function row 104). The user selection may be a keystroke, a mouse click, a mouse hover, a command+tab input, etc. In some embodiments, the change in focus occurs in response to a user selection of an element of a graphical user interface (e.g., a different window) on the portable system 100 (or ). Figure 2A-2D The operation system may be configured to determine whether the application window has focus or not. For example, when the user closes the application window that has focus, the operating system may give focus to a different application, such as the application that had focus before the closed application window. In another example, when the user closes the application window that has focus, the operating system may give focus to a dialog box prompting the user to save changes made to the document via the application.

[0052] In some embodiments, the change in focus may be a change from an element associated with an application to another element associated with the same application (e.g., from an email composition window of an email application to an inbox list window of an email application, or from one tab of an internet browser application to another tab of an internet browser application). In some embodiments, the change in focus may be a change from an element associated with one application to an element associated with another application (e.g., from an internet browser window to an email application window). In addition, in some embodiments, the change in focus may be a change from an element associated with an application to an element associated with an operating system, such as a system dialog box, a system settings control (e.g., a volume control), a window associated with a file / folder navigation application (e.g., the FINDER application of Apple Inc.), etc. In addition, the focus may also point to a dialog box, a file directory, a settings control (e.g., a volume control), or any other element of a graphical user interface that may receive a user and / or user input to which information may be presented.

[0053] Figure 2A 2 is an illustrative diagram of a first specific implementation of a desktop computing system 200 according to some embodiments. The desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a main display 102, a peripheral keyboard 206, and a peripheral mouse 208. The computing device 202 includes one or more processors and a memory storing one or more programs for execution by the one or more processors. In some embodiments, the peripheral display device 204 can be connected to the computing device 202 such as Device integration. In some embodiments, the primary display 102 of the peripheral display device 204 is a touch screen display. Figure 2A In the embodiment, the peripheral display device 204 (also referred to herein as the first housing 204 or housing 204), the peripheral keyboard 206, and the peripheral mouse 208 are communicatively coupled to the computing device 202 via a wired connection, such as USB or PS / 2, or via a wireless communication link, using a communication protocol such as Bluetooth, Wi-Fi, etc. For example, the peripheral keyboard 206 (also referred to herein as the second housing 206 or housing 206) is no more than fifteen feet (e.g., about three feet away) from the computing device 202. Figure 2A In some embodiments, the peripheral keyboard 206 includes a dynamic function row 104 and a set of physical keys 106 at least partially contained within the same housing. Figure 1BThe dynamic function row 104 described in more detail is a touch screen display. In some embodiments, the peripheral keyboard 206 includes one or more processors and a memory storing one or more programs that can be executed by the one or more processors of the peripheral keyboard 206 to perform any of the embodiments described herein. In some embodiments, the peripheral keyboard 206 relays signals indicating user input (e.g., key strokes and selections represented by user-selectable symbols / affordances displayed by the dynamic function row 104) to the computing device 202.

[0054] Figure 2B is an illustrative diagram of a second specific implementation of a desktop computing system 200 according to some embodiments. Figure 2B In FIG. 1 , a desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a main display 102, and a peripheral keyboard 206. Figure 2B In the embodiment, the peripheral display device 204 and the peripheral keyboard 206 are communicatively coupled to the computing device 202 via a wired connection, such as USB or PS / 2, or via a wireless communication link, using a communication protocol such as Bluetooth, Wi-Fi, etc. Figure 2B In some embodiments, the peripheral keyboard 206 includes a dynamic function row 104, a set of physical keys 106, and a touchpad 108 at least partially contained within the same housing. Figure 1B The dynamic function row 104 described in more detail is a touch screen display. In some embodiments, the peripheral keyboard 206 includes one or more processors and a memory storing one or more programs that can be executed by the one or more processors of the peripheral keyboard 206 to perform any of the embodiments described herein. In some embodiments, the peripheral keyboard 206 relays signals indicating user input (e.g., key strokes, user interactions with the touchpad 108, and selections represented by user-selectable symbols / affordances displayed by the dynamic function row 104) to the computing device 202.

[0055] Figure 2C is an illustrative diagram of a third specific implementation of a desktop computing system 200 according to some embodiments. Figure 2C In FIG. 1 , the desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a main display 102, a peripheral keyboard 206, and a first peripheral input mechanism 212. Figure 2C In the embodiment, the peripheral display device 204, the peripheral keyboard 206 and the first peripheral input mechanism 212 are communicatively coupled to the computing device 202 via a wired connection, such as USB or PS / 2, or via a wireless communication link, using a communication protocol such as Bluetooth, Wi-Fi, etc. Figure 2CIn some embodiments, the peripheral keyboard 206 includes a set of physical keys 106, and the first peripheral input mechanism 212 includes a dynamic function row 104 and a touchpad 108 at least partially contained within the same housing. Figure 1B The dynamic function row 104 described in more detail is a touch screen display. In some embodiments, the first peripheral input mechanism 212 includes one or more processors and a memory storing one or more programs, which can be executed by the one or more processors of the first peripheral input mechanism 212 to perform any of the embodiments described herein. In some embodiments, the first peripheral input mechanism 212 relays signals indicating user input (e.g., user interaction with the touchpad 108, and user selection represented by user-selectable symbols / indications displayed by the dynamic function row 104) to the computing device 202.

[0056] Figure 2D is an illustrative diagram of a fourth specific implementation of a desktop computing system 200 according to some embodiments. Figure 2D In FIG. 1 , the desktop computing system 200 includes a computing device 202, a peripheral display device 204 having a primary display 102, a peripheral keyboard 206, a peripheral mouse 208, and a second peripheral input mechanism 222. Figure 2C In the embodiment, the peripheral display device 204, the peripheral keyboard 206, the peripheral mouse 208 and the second peripheral input mechanism 222 are communicatively coupled to the computing device 202 via a wired connection, such as USB or PS / 2, or via a wireless communication link, using a communication protocol such as Bluetooth, Wi-Fi, etc. Figure 2A In FIG. 1 , the peripheral keyboard 206 includes a dynamic function row 104 and a set of physical keys 106. Figure 2D In some embodiments, the peripheral keyboard 206 includes a set of physical keys 106, and the second peripheral input mechanism 222 includes a dynamic function row 104 at least partially contained within a housing of the second peripheral input mechanism 222. Figure 1B The dynamic function row 104 described in more detail is a touch screen display. In some embodiments, the second peripheral input mechanism 222 includes one or more processors and a memory storing one or more programs, which can be executed by the one or more processors of the second peripheral input mechanism 222 to perform any of the embodiments described herein. In some embodiments, the second peripheral input mechanism 222 relays a signal indicating a user input (e.g., a user selection represented by a user-selectable symbol / indicator displayed by the dynamic function row 104) to the computing device 202.

[0057] Figure 3A300 is a block diagram of an electronic device 300 according to some embodiments. In some embodiments, the electronic device 300 is a portable electronic device, such as a laptop (e.g., portable computing system 100, Figure 1A In some embodiments, the electronic device 300 is not a portable device, but a desktop computer (e.g., the computing device 202 of the desktop computing system 200, Figure 2A-2D ), which is connected to a peripheral display system (e.g., a peripheral display device 204, Figure 2A-2D ) and an optional peripheral touch-sensitive surface (e.g., touchpad 108, Figure 2B-2C , and / or a touch-sensitive display, such as a peripheral display device 204, Figure 2A-2D , and / or dynamic function row 104, Figure 2A-2D ) communicatively coupled.

[0058] The electronic device 300 typically supports a variety of applications, such as one or more of the following applications: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a video conferencing application, an email application, an instant messaging application, an image management application, a digital camera application, a digital camcorder application, a web browser application, and / or a media player application.

[0059] Various applications executed on the electronic device 300 optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed by the electronic device 300 are optionally adjusted and / or varied from one application to the next, and / or within an application. In this way, the common physical architecture of the electronic device 300 (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and clear to the user.

[0060] The electronic device 300 includes a memory 302 (which optionally includes one or more computer-readable storage media), a memory controller 322, one or more processing units (CPUs) 320, a peripheral device interface 318, an RF circuit 308, an audio circuit 310, a speaker 311, a microphone 313, an input / output (I / O) subsystem 306, other input or control devices 316, and an external port 324. The electronic device 300 optionally includes a display system 312 (e.g., a main display 102 of the display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B), which may be a touch-sensitive display (also sometimes referred to herein as a "touch screen" or "touch screen display"). The electronic device 300 optionally includes one or more optical sensors 364. The electronic device 300 optionally includes one or more intensity sensors 365 for detecting the intensity of contact on a touch-sensitive surface such as a touch-sensitive display or a touch pad. The electronic device 300 optionally includes one or more tactile output generators 367 for generating tactile sensations on a touch-sensitive surface such as a touch-sensitive display or a touch pad (e.g., touch pad 108, Figure 1A-1B ) to generate tactile output. These components optionally communicate via one or more communication buses or signal lines 303.

[0061] As used in this specification, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or to a surrogate (surrogate) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average) to determine an estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or changes thereto, the capacitance of the touch-sensitive surface near the contact and / or changes thereto, and / or the resistance of the touch-sensitive surface near the contact and / or changes thereto are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the substitute measurement of contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some implementations, the substitute measurement of contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).

[0062] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or a touch / trackpad) is optionally interpreted by the user as a "press click" or "release click" on a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, movement of a touch-sensitive surface may optionally be interpreted or sensed by a user as "roughness" of the touch-sensitive surface even when there is no change in the smoothness of the touch-sensitive surface. While such interpretation of touch by a user will be limited by the user's individualized sensory perceptions, there are many sensory perceptions of touch that are common to most users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., "release click," "press click," "roughness"), unless otherwise stated, the tactile output generated corresponds to a physical displacement of the device or a component thereof that would generate that sensory perception for a typical (or average) user.

[0063] It should be understood that the electronic device 300 is merely an example, and that the electronic device 300 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 3A The various components shown in the drawings are implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing circuits and / or application specific integrated circuits).

[0064] The memory 302 optionally includes a high-speed random access memory, and optionally also includes a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The access to the memory 302 by other components of the electronic device 300 (such as one or more CPUs 320 and peripheral device interfaces 318) is optionally controlled by a memory controller 322. The peripheral device interface 318 can be used to couple input peripherals and output peripherals to one or more CPUs 320 and the memory 302. One or more processing units 320 run or execute various software programs and / or instruction sets stored in the memory 302 to perform various functions of the electronic device 300 and process data. In some embodiments, the peripheral device interface 318, one or more CPUs 320, and the memory controller 322 are optionally implemented on a single chip such as a chip 304. In some other embodiments, they are optionally implemented on separate chips.

[0065] RF (radio frequency) circuit 308 receives and transmits RF signals, also called electromagnetic signals. RF circuit 308 converts electrical signals into / from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 308 optionally includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC chipsets, subscriber identity modules (SIM) cards, memory, etc. RF circuit 308 optionally communicates with networks and other devices through wireless communications, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs)). Wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilization Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communications protocol, including communications protocols that have not yet been developed as of the filing date of this document.

[0066] The audio circuit 310, the speaker 311 and the microphone 313 provide an audio interface between the user and the electronic device 300. The audio circuit 310 receives audio data from the peripheral device interface 318, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 311. The speaker 311 converts the electrical signal into a sound wave audible to humans. The audio circuit 310 also receives an electrical signal converted from a sound wave by the microphone 313. The audio circuit 310 converts the electrical signal into audio data, and transmits the audio data to the peripheral device interface 318 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 302 and / or the RF circuit 308 by the peripheral device interface 318. In some embodiments, the audio circuit 310 also includes an earphone jack. The earphone jack provides an interface between the audio circuit 310 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single earphone or a binaural headset) and input (e.g., a microphone).

[0067] The I / O subsystem 306 couples the input / output peripherals of the electronic device 300, such as a display system 312 and other input or control devices 316, to a peripheral device interface 318. The I / O subsystem 306 optionally includes a display controller 356, an optical sensor controller 358, an intensity sensor controller 359, a tactile feedback controller 361, and one or more other input controllers 360 for other input or control devices. The one or more other input controllers 360 receive / send electrical signals from / to other input or control devices 316. Other input or control devices 316 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, the one or more other input controllers 360 are optionally coupled to (or not coupled to) any of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more physical buttons may optionally include an up / down button for volume control of the speaker 311 and / or microphone 313 .

[0068] Display system 312 (e.g., main display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B) provides an output interface between the electronic device 300 and the user (and optionally, when the display system is a touch-sensitive display, provides an input interface). The display controller 356 receives electrical signals from the display system 312 and / or sends electrical signals to the display system 312. The display system 312 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects / elements.

[0069] In some embodiments, a display system 312 (e.g., a main display 102 of a display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ) is a touch-sensitive display having a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on tactile / haptic contact. Thus, the display system 312 and display controller 356 (together with any associated modules and / or instruction sets in memory 302) detect contact (and any movement or interruption of that contact) on the display system 312 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the display system 312. In an exemplary embodiment, the point of contact between the display system 312 and the user corresponds to the area under the user's finger.

[0070] Display system 312 (e.g., main display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ) optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, LED (light emitting diode) technology, or OLED (organic light emitting diode) technology, although other display technologies are used in other embodiments. In some embodiments, when the display system 312 is a touch-sensitive display, the display system 312 and the display controller 356 optionally use any of a variety of touch sensing technologies now known or later developed and other proximity sensor arrays or other elements for determining one or more points of contact with the display system 312 to detect contact and any movement or interruption thereof. The various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as in the Apple Inc. (Cupertino, California) and The technology that exists in .

[0071] Display system 312 (e.g., main display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ) optionally has a video resolution of more than 400 dpi (e.g., 500 dpi, 800 dpi, or greater). In some embodiments, the display system 312 is a touch-sensitive display, and the user optionally contacts the touch-sensitive display using a stylus, finger, etc. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures. In some embodiments, the electronic device 300 translates rough finger-based input into precise pointer / cursor positions or commands for performing actions desired by the user.

[0072] In some embodiments, in addition to the display system 312, the electronic device 300 optionally includes a touch panel (eg, touch panel 108, Figure 1A-1B ) for activating or deactivating a particular function. In some embodiments, the touchpad is a touch-sensitive area of ​​the electronic device 300 that is different from the display system 312 and does not display visual output. In some embodiments, when the display system 312 is a touch-sensitive display, the touchpad is optionally a touch-sensitive surface separate from the display system 312, or an extension of the touch-sensitive surface formed by the display system 312.

[0073] The electronic device 300 also includes a power system 362 for powering the various components. The power system 362 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), etc.), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0074] The electronic device 300 optionally also includes one or more optical sensors 364 coupled to the optical sensor controller 358 in the I / O subsystem 306. The optical sensor 364 optionally includes a charge coupled device (CCD), or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 364 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with the imaging module 343, the one or more optical sensors 364 optionally capture a still image or video. In some embodiments, the optical sensor is located on the front of the electronic device 300 so that the user can optionally obtain an image of the user for video conferencing while watching other video conference participants on the display system 312.

[0075] The electronic device 300 optionally also includes one or more contact force sensors 365 coupled to the force sensor controller 359 in the I / O subsystem 306. The one or more contact force sensors 365 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). The one or more contact force sensors 365 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is coupled to a touch-sensitive surface (e.g., touch pad 108, Figure 1A-1B , or when it is a touch-sensitive display, the display system 312) is arranged in parallel or adjacent.

[0076] The electronic device 300 optionally also includes one or more tactile output generators 367 coupled to the tactile feedback controller 361 in the I / O subsystem 306. The one or more tactile output generators 367 optionally include one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). One or more contact force sensors 365 receive tactile feedback generation instructions from the tactile feedback module 333 and generate tactile outputs that can be sensed by a user of the electronic device 300. In some embodiments, at least one tactile output generator is coupled to a touch-sensitive surface (e.g., touch pad 108, Figure 1A-1B , or when it is a touch-sensitive display, the display system 312) is arranged in juxtaposition or proximity, and optionally generates tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward from the surface of the electronic device 300) or laterally (e.g., backward and forward in the same plane as the surface of the electronic device 300).

[0077] The electronic device 300 optionally also includes one or more proximity sensors 366 coupled to the peripheral device interface 318. Alternatively, the one or more proximity sensors 366 are coupled to one or more other input controllers 360 in the I / O subsystem 306. The electronic device 300 optionally also includes one or more accelerometers 368 coupled to the peripheral device interface 318. Alternatively, the one or more accelerometers 368 are coupled to one or more other input controllers 360 in the I / O subsystem 306.

[0078] In some embodiments, the software components stored in memory 302 include an operating system 326, a communication module 328 (or an instruction set), a contact / motion module 330 (or an instruction set), a graphics module 332 (or an instruction set), an application 340 (or an instruction set), and a dynamic function row module 350 (or an instruction set). In addition, in some embodiments, memory 302 stores device / global internal state 357 (or an instruction set), such as Figure 3A Device / global internal state 357 includes one or more of the following: an active application state, which indicates which application (if any) is currently active and / or in focus; a display state, which indicates what application, view, or other information occupies display system 312 (e.g., primary display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ) and / or a peripheral display system (e.g., a primary display 102 of a peripheral display device 204, Figure 2A-2D , and / or dynamic function row 104, Figure 2A-2D ) of various areas; sensor status, which includes information obtained from various sensors and input or control devices 316 of the electronic device 300, as well as position information about the position and / or posture of the electronic device 300.

[0079] The operating system 326 (e.g., DARWIN, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VXWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitating communication between various hardware components and software components.

[0080] The communication module 328 facilitates communication with other devices (e.g., computing device 202, Figure 2A-2D ; Peripheral mouse 208, Figure 2A and Figure 2D ; Peripheral keyboard 206, Figure 2A-2B ; A first peripheral input mechanism 212, Figure 2C and / or a second peripheral input mechanism 222, Figure 2D ) and also includes various software components for sending / receiving data via RF circuit 308 and / or external port 324. External port 324 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, external port 324 is connected to A multi-pin (eg, 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used on the device.

[0081] The contact / motion module 330 optionally detects contact with the display system 312 (when it is a touch-sensitive display) (in conjunction with the display controller 356) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 330 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 330 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single-finger contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, the contact / motion module 330 also detects a touch pad (e.g., touch pad 108, Figure 1A-1B ) on the contact.

[0082] In some embodiments, the contact / motion module 330 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has selected or "clicked" an indication). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of the electronic device 300). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0083] The contact / motion module 330 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap contact includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event and then detecting one or more finger drag events, and in some embodiments, subsequently detecting a finger lift (lift-off) event.

[0084] The graphics module 332 includes a module for causing and resulting in the display of graphics on the primary display 102 (e.g., the primary display 102 of the display portion 110, Figure 1A or the main display 102 of the peripheral display device 204, Figure 2A-2D ) or other known software components on the display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc. In some embodiments, the graphics module 332 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 332 receives one or more codes specifying the graphics to be displayed from an application, etc., and also receives coordinate data and other graphics attribute data if necessary, and then generates screen image data for output to the display controller 356.

[0085] The tactile feedback module 333 includes various software components for generating instructions used by one or more tactile output generators 367 to produce tactile output at one or more locations on the electronic device 300 in response to user interaction with the electronic device 300.

[0086] The application 340 optionally includes the following modules (or instruction sets), or a subset or superset thereof:

[0087] An email client module 341 (also sometimes referred to herein as a "mail application" or "email application") for receiving, sending, composing, and viewing emails;

[0088] An imaging module 342 for capturing still and / or video images;

[0089] An image management module 343 (also sometimes referred to herein as a "photo application") for editing and viewing still and / or video images;

[0090] A media player module 344 (also sometimes referred to herein as a "media player application") for playing back audio and / or video; and

[0091] A web browsing module 345 (also sometimes referred to herein as a "web browser") for connecting to and browsing the Internet.

[0092] Examples of other applications 340 optionally stored in memory 302 include instant messaging and communication applications, word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption applications, digital rights management applications, voice recognition applications, and voice replication applications.

[0093] In combination with RF circuit 308, display system 312 (e.g., main display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ), display controller 356, and one or more of contact module 330 and graphics module 332, the email client module 341 includes executable instructions for creating, sending, receiving and managing emails in response to user instructions. In conjunction with the image management module 343, the email client module 341 makes it very easy to create and send emails with static images or video images captured by the imaging module 342.

[0094] In combination with a display system 312 (e.g., a main display 102 of the display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ), display controller 356, one or more optical sensors 364, optical sensor controller 358, contact module 330, graphics module 332 and image management module 343, imaging module 342 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 302, modifying the characteristics of still images or videos, or deleting still images or videos from memory 302.

[0095] In combination with a display system 312 (e.g., a main display 102 of the display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B), one or more of the display controller 356, the contact module 330, the graphics module 332, and the imaging module 342, the image management module 343 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.

[0096] In combination with a display system 312 (e.g., a main display 102 of the display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ), display controller 356, contact module 330, graphics module 332, audio circuit 310, speaker 311, RF circuit 308, and web browsing module 345, media player module 344 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), and to display, present, or otherwise play back video (e.g., on main display 102 of display portion 110, Figure 1A or on the primary display 102 of a peripheral display device 2014 connected via the external port 324, Figure 2A-2B ) executable instructions.

[0097] In combination with RF circuit 308, display system 312 (e.g., main display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ), one or more of the display controller 356, the contact module 330 and the graphics module 332, the web browsing module 345 includes executable instructions for browsing the Internet according to user instructions (including searching, linking to, receiving, and displaying web pages or parts thereof, as well as attachments and other files linked to web pages).

[0098] The dynamic function row (DFR) module 350 includes: a focus determination module 351, a DFR determination module 352 and a DFR rendering module 353. In some embodiments, the focus determination module 351 is configured to determine the focus of the display system 312 (e.g., the main display 102 of the display portion 110, Figure 1A ) or a peripheral display system (e.g., a peripheral display device 204, Figure 2A-2D ) in focus on a graphical user interface displayed by the display system 312 (e.g., dynamic function row 104, Figure 1A-1B) on the DFR determination module 352. The DFR presentation module 353 includes a module for causing and resulting in the graphic being displayed on the display system 312 (e.g., the dynamic function row 104, Figure 1A-1B ) on the display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc. In some embodiments, the DFR module 350 includes other modules for performing the following operations: adjusting the sensitivity of the dynamic function row 104; adjusting the auditory and / or tactile feedback provided by the dynamic function row 104; adjusting the settings of the enabling representations and information (e.g., size, brightness, font, language, etc.) displayed by the dynamic function row 104; adjusting the current power mode (e.g., normal and low power modes) of the dynamic function row 104; and so on.

[0099] In some embodiments, the dynamic feature row module 350 interacts with components that allow for the provision of predictive / proactive / suggested content items, including predicted recipients, suggested text completion strings, proactively suggested applications, etc. Proactively suggested content items are discussed in more detail in U.S. Application Serial No. 15 / 167,713, which is hereby incorporated by reference in its entirety.

[0100] Each module and application program in the above-mentioned identified modules and applications program corresponds to an executable instruction set for performing the above-mentioned one or more functions and the method described in this patent application (e.g., the computer-implemented method and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 302 optionally stores a subset of the above-mentioned modules and data structures. In addition, memory 302 optionally stores additional modules and data structures not described above.

[0101] Figure 3B According to some embodiments, Figure 3A In some embodiments, the memory 302 ( Figure 3A ) includes an event classifier 370 (e.g., in the operating system 326) and an application 340-1 (e.g., any one of the aforementioned applications 341, 342, 343, 344 or 345).

[0102] Event classifier 370 receives event information and determines the application 340-1 and application view 391 of application 340-1 to which the event information is to be delivered. Event classifier 370 includes event monitor 371 and event dispatcher module 374. In some embodiments, application 340-1 includes application internal state 392, which indicates when the application is active or displayed on display system 312 (e.g., primary display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B ) on one or more current application views. In some embodiments, device / global internal state 357 is used by event classifier 370 to determine which application(s) are currently active or in focus, and application internal state 392 is used by event classifier 370 to determine application view 391 to which event information is to be delivered.

[0103] In some embodiments, the application internal state 392 includes additional information, such as one or more of the following: resumption information to be used when the application 340-1 resumes execution, user interface state information indicating information being displayed by the application 340-1 or information ready to be displayed by the application 340-1, a state queue for enabling a user to return to a previous state or view of the application 340-1, and a repeat / undo queue of previous actions taken by the user.

[0104] Event monitor 371 receives event information from peripherals interface 318. Event information includes information about sub-events (e.g., user touches on display system 312 (when it is a touch-sensitive display) as part of a multi-touch gesture). Peripherals interface 318 transmits information it receives from I / O subsystem 306 or sensors (such as one or more proximity sensors 366, one or more accelerometers 368, and / or microphone 313 (through audio circuit 310)). The information that peripherals interface 318 receives from I / O subsystem 306 includes information from display system 312 (when it is a touch-sensitive display) or another touch-sensitive surface (e.g., touchpad 108, Figure 1A-1B ) information.

[0105] In some embodiments, event monitor 371 sends requests to peripheral device interface 318 at predetermined intervals. In response, peripheral device interface 318 transmits event information. In other embodiments, peripheral device interface 318 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0106] In some embodiments, the event classifier 370 also includes a hit view determination module 372 and / or an active event identifier determination module 373.

[0107] The hit view determination module 372 provides software processes for determining where a sub-event occurs within one or more views when the display system 312 displays more than one view, where a view consists of controls and other elements that a user can see on the display.

[0108] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are considered to be correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0109] The hit view determination module 372 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, the hit view determination module 372 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0110] Active event recognizer determination module 373 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 373 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 373 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, higher views in the hierarchy will still remain as actively participating views.

[0111] Event distributor module 374 distributes event information to event identifiers (e.g., event identifier 380). In embodiments including active event identifier determination module 373, event distributor module 374 delivers the event information to the event identifier determined by active event identifier determination module 373. In some embodiments, event distributor module 374 stores the event information in an event queue, which is retrieved by corresponding event receiver 382.

[0112] In some embodiments, operating system 326 includes event classifier 370. Alternatively, application 340-1 includes event classifier 370. In yet another embodiment, event classifier 370 is a standalone module or is part of another module stored in memory 302, such as contact / motion module 330.

[0113] In some embodiments, application 340-1 includes multiple event handlers 390 and one or more application views 391, each of which includes instructions for handling touch events occurring in a corresponding view of the user interface of the application. Each application view 391 of application 340-1 includes one or more event identifiers 380. Typically, application view 391 includes multiple event identifiers 380. In other embodiments, one or more event identifiers in event identifier 380 are part of an independent module, such as a user interface toolkit (not shown) or a higher-level object from which application 340-1 inherits methods and other properties. In some embodiments, the corresponding event handler 390 includes one or more of the following: data updater 376, object updater 377, GUI updater 378, and / or event data 379 received from event classifier 370. Event handler 390 optionally utilizes or calls data updater 376, object updater 377 or GUI updater 378 to update application internal state 392. Alternatively, one or more of the application views 391 include one or more corresponding event handlers 390. Additionally, in some embodiments, one or more of the data updater 376, object updater 377, and GUI updater 378 are included in the application view 391.

[0114] A corresponding event identifier 380 receives event information (e.g., event data 379) from event classifier 370 and identifies an event from the event information. Event identifier 380 includes an event receiver 382 and an event comparator 384. In some embodiments, event identifier 380 also includes metadata 383 and at least a subset of event delivery instructions 388 (which optionally include sub-event delivery instructions).

[0115] The event receiver 382 receives event information from the event classifier 370. The event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (for example, from a longitudinal orientation to a horizontal orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0116] Event comparator 384 compares event information with predefined event or sub-event definitions, and based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 384 includes event definition 386. Event definition 386 includes a definition of an event (e.g., a predefined sub-event sequence), such as event 1 (387-1), event 2 (387-2), and other events. In some embodiments, sub-events in event 387 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (387-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift-off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift-off (touch end) of a predetermined duration. In another example, the definition of event 2 (387-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the display system 312 (when it is a touch-sensitive display), and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 390.

[0117] In some embodiments, event definition 387 includes a definition of events for corresponding user interface objects. In some embodiments, event comparator 384 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view (where three user interface objects are displayed on display system 312), when a touch is detected on display system 312 (when it is a touch-sensitive display), event comparator 384 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 390, the event comparator uses the result of the hit test to determine which event handler 390 should be activated. For example, event comparator 384 selects an event handler associated with a sub-event and the object that triggered the hit test.

[0118] In some embodiments, the definition of the corresponding event 387 also includes a delay action that delays the delivery of the event information until it has been determined whether the sub-event sequence does or does not correspond to the event type of the event identifier.

[0119] When a corresponding event recognizer 380 determines that a series of sub-events does not match any event in event definition 386, the corresponding event recognizer 380 enters an event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0120] In some embodiments, the corresponding event identifier 380 includes metadata 383 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event identifiers. In some embodiments, metadata 383 includes configurable properties, flags, and / or lists that indicate how event identifiers interact or can interact with each other. In some embodiments, metadata 383 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0121] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 380 activates an event handler 390 associated with the event. In some embodiments, the corresponding event recognizer 380 delivers event information associated with the event to the event handler 390. Activating the event handler 390 is different from sending (and deferred sending) sub-events to the corresponding hit view. In some embodiments, the event recognizer 380 throws a tag associated with the identified event, and the event handler 390 associated with the tag obtains the tag and performs a predefined process.

[0122] In some embodiments, the event delivery instructions 388 include a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event series or to an actively participating view. The event handler associated with the sub-event series or with an actively participating view receives the event information and executes a predetermined process.

[0123] In some embodiments, data updater 376 creates and updates data used in application 340-1. For example, data updater 376 stores video files used by media player module 344. In some embodiments, object updater 377 creates and updates objects used by application 340-1. For example, object updater 376 creates a new user interface object or updates the position of a user interface object. GUI updater 378 updates the GUI. For example, GUI updater 378 prepares display information and sends it to graphics module 332 for display on display system 312 (e.g., main display 102 of display portion 110, Figure 1A , and / or dynamic function row 104, Figure 1A-1B )superior.

[0124] In some embodiments, one or more event handlers 390 include or have access to data updater 376, object updater 377, and GUI updater 378. In some embodiments, data updater 376, object updater 377, and GUI updater 378 are included in a single module of application 340-1 or application view 391. In other embodiments, they are included in two or more software modules.

[0125] It should be understood that the above discussion of event processing of user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the electronic device 300, and not all user inputs are initiated on the touch screen. For example, mouse movement and mouse button presses, optionally combined with single or multiple keyboard presses or holds; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to a sub-event defining an event to be recognized.

[0126] Figure 4 A block diagram of a peripheral electronic device 400 according to some embodiments is shown. In some embodiments, the peripheral electronic device 400 is a peripheral input and output device that at least partially combines a dynamic function row 104 and a physical input mechanism, such as a set of physical keys (e.g., a set of physical keys 106, Figure 2A-2B ) and / or a touch pad (e.g., touch pad 108, Figure 2B-2C ) are contained in the same housing. Examples of the peripheral electronic device 400 include: a peripheral keyboard (eg, the peripheral keyboard 206, Figure 2A-2B ), a peripheral touch-sensitive surface (eg, a first peripheral input mechanism 212, Figure 2C ) or other peripheral input mechanisms (e.g., the second peripheral input mechanism 222, Figure 2D ). The peripheral electronic device 400 and the computing device 202 ( Figure 2A-2D ) communicatively coupled. For example, the peripheral electronic device 400 is communicatively coupled to the computing device 202 via a wired connection, such as USB or PS / 2, or via a wireless communication link, using a communication protocol such as Bluetooth, Wi-Fi, etc. The peripheral electronic device 400 may rely on the electronic device 300 ( Figure 3A ) or some of the components or processes in the peripheral electronic device 400 instead of the electronic device 300, or some of these components or processes may be performed by the peripheral electronic device 400 instead of the electronic device 300, be located in the peripheral electronic device 400 instead of the electronic device 300, or be accommodated by the peripheral electronic device 400 instead of the electronic device 300.

[0127] In some embodiments, the peripheral electronic device 400 includes one or more of a memory 402 (which optionally includes one or more computer-readable storage media), a memory controller 422, one or more processing units (CPUs) 420, a peripheral device interface 418, an RF circuit 408, an audio circuit 410, a speaker 411, a microphone 413, an input / output (I / O) subsystem 406, other input or control devices 416, and an external port 424. The peripheral electronic device 400 includes a touch-sensitive display system 412 (e.g., a dynamic function row 104, Figure 2A-2D) (also sometimes referred to herein as a "touch-sensitive display," "touch screen," or "touch screen display").

[0128] The peripheral electronic device 400 optionally includes a touch-sensitive surface such as a touch-sensitive display system 412 or a touch pad (e.g., touch pad 108, Figure 2B-2C ) on a touch-sensitive surface such as a touch-sensitive display system 412 or a touch pad (e.g., touch pad 108, Figure 2B-2C ) to generate tactile output. These components optionally communicate via one or more communication buses or signal lines 403.

[0129] Memory 402 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The access of other components of peripheral electronic device 400 (such as one or more CPUs 420 and peripheral device interface 418) to memory 402 is optionally controlled by memory controller 422. Peripheral device interface 418 can be used to couple one or more CPUs 420 and memory 402 to I / O subsystem 406 and other circuits. One or more processing units 420 run or execute various software programs and / or instruction sets stored in memory 402 to perform various functions of peripheral electronic device 400 and process data. In some embodiments, peripheral device interface 418, one or more CPUs 420, and memory controller 422 are optionally implemented on a single chip such as chip 404. In some other embodiments, they are optionally implemented on independent chips.

[0130] RF (radio frequency) circuitry 408 receives and transmits RF signals, also referred to as electromagnetic signals. RF circuitry 408 converts electrical signals to / from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. RF circuitry 408 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and the like. Wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including but not limited to near field communication (NFC), Bluetooth, wireless fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Wi-MAX, or any other suitable communication protocol, including communication protocols that have not yet been developed as of the filing date of this document.

[0131] The optional audio circuit 410, speaker 411 and microphone 413 provide an audio interface between the user and the peripheral electronic device 400. The audio circuit 410 receives audio data from the peripheral device interface 418, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 411. The speaker 411 converts the electrical signal into a sound wave audible to humans. The audio circuit 410 also receives an electrical signal converted from a sound wave by the microphone 413. The audio circuit 410 converts the electrical signal into audio data and transmits the audio data to the peripheral device interface 418 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 402 and / or the RF circuit 408 by the peripheral device interface 418. In some embodiments, the audio circuit 410 also includes an earphone jack. The earphone jack provides an interface between the audio circuit 410 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single earphone or a dual earphone) and input (e.g., a microphone).

[0132] The I / O subsystem 406 connects the input / output peripherals of the peripheral electronic device 400, such as the touch-sensitive display system 412 (e.g., the dynamic function row 104, Figure 2A-2D) is coupled to peripherals interface 418. I / O subsystem 406 optionally includes display controller 456, intensity sensor controller 459, tactile feedback controller 461, and one or more input controllers 460 for other input or control devices 416. One or more other input controllers 460 receive / send electrical signals from / to other input or control devices 416. Other input or control devices 416 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, a set of physical keys, a touch pad, etc.

[0133] Touch-sensitive display system 412 (e.g., dynamic function row 104, Figure 2A-2D ) provides an input / output interface between the peripheral electronic device 400 and the user. The touch-sensitive display (TSD) controller 456 receives electrical signals from the touch-sensitive display system 412 and / or sends electrical signals to the touch-sensitive display system 412. The touch-sensitive display system 412 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects / elements.

[0134] Touch-sensitive display system 412 (e.g., dynamic function row 104, Figure 2A-2D ) includes a touch-sensitive surface, sensor, or group of sensors that accept input from a user based on tactile and / or haptic contact. Thus, the touch-sensitive display system 412 and TSD controller 456 (together with any associated modules and / or instruction sets in memory 402) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 412 and convert the detected contact into a signal for selecting or controlling a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 412. In an exemplary embodiment, a point of contact between the touch-sensitive display system 412 and the user corresponds to an area of ​​the touch-sensitive display system 412 that is in contact with the user's finger.

[0135] Touch-sensitive display system 412 (e.g., dynamic function row 104, Figure 2A-2D) optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, LED (light emitting diode) technology, or OLED (organic light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 412 and TSD controller 456 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 412 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as in the Apple Inc. (Cupertino, California) and The technology that exists in .

[0136] Touch-sensitive display system 412 (e.g., dynamic function row 104, Figure 2A-2D ) optionally has a video resolution of more than 400 dpi (e.g., 500 dpi, 800 dpi, or greater). In some embodiments, the user contacts the touch-sensitive display system 412 using a stylus, finger, etc. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures.

[0137] In some embodiments, in addition to the touch-sensitive display system 412, the peripheral electronic device 400 optionally includes a touch pad (e.g., touch pad 108, Figure 2B-2C ). In some embodiments, the touchpad is a touch-sensitive area of ​​the peripheral electronic device 400 that is different from the touch-sensitive display system 412 and does not display visual output. In some embodiments, the touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 412, or an extension of the touch-sensitive surface formed by the touch-sensitive display system 412.

[0138] The peripheral electronic device 400 also includes a power system 462 for powering the various components. The power system 462 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), etc.), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0139] The peripheral electronic device 400 optionally also includes one or more contact force sensors 465 coupled to the force sensor controller 459 in the I / O subsystem 406. The one or more contact force sensors 465 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). The contact force sensor 465 receives contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is coupled to a touch-sensitive surface (e.g., touch-sensitive display system 412 and / or touch pad 108, Figure 2B-2C ) are arranged in juxtaposition or proximity.

[0140] The peripheral electronic device 400 optionally also includes one or more tactile output generators 467 coupled to the tactile feedback controller 461 in the I / O subsystem 406. The one or more tactile output generators 467 optionally include one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). One or more contact force sensors 465 receive tactile feedback generation instructions from the tactile feedback module 433 and generate tactile outputs that can be sensed by a user of the peripheral electronic device 400. In some embodiments, at least one tactile output generator is coupled to a touch-sensitive surface (e.g., a touch-sensitive display system 412 and / or a touchpad 108, Figure 2B-2C ) are arranged in juxtaposition or proximity, and optionally generate tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward toward the surface of the peripheral electronic device 400) or laterally (e.g., backward and forward in the same plane as the surface of the peripheral electronic device 400).

[0141] In some embodiments, the software components stored in memory 402 include an operating system 426, a communication module 428 (or an instruction set), a contact / motion module 430 (or an instruction set), and a dynamic function row module 450 (or an instruction set). In addition, in some embodiments, memory 402 stores a device state 457, which includes a display state that indicates what view or other information occupies the touch-sensitive display system 412 (e.g., the dynamic function row 104, Figure 2A-2D ) areas.

[0142] The operating system 426 includes various software components and / or drivers for controlling and managing general system tasks (eg, memory management, storage device control, power management, etc.) and facilitating communications between various hardware and software components.

[0143] The communication module 428 facilitates communication with other devices (e.g., computing device 202, Figure 2A-2D ) for communication, and also includes various software components for sending / receiving data via RF circuit 408 and / or external port 424. External port 424 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.).

[0144] The contact / motion module 430 optionally detects contact with the touch-sensitive display system 412 and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 430 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 430 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, the contact / motion module 430 also detects a touch pad (e.g., touch pad 108, Figure 2B-2C ) on contact.

[0145] In some embodiments, the contact / motion module 430 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has selected or "clicked" an indication). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the electronic device 400). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of the touchpad or touchscreen can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0146] The contact / motion module 430 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap contact includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event and then detecting one or more finger drag events, and in some embodiments, subsequently detecting a finger lift (lift-off) event.

[0147] The tactile feedback module 433 includes various software components for generating instructions that are used by one or more tactile output generators 467 to produce tactile output at one or more locations on the peripheral electronic device 400 in response to user interaction with the peripheral electronic device 400.

[0148] The dynamic function row (DFR) module 450 includes: a focus acquisition module 451, a DFR determination module 452, and a DFR presentation module 453. In some embodiments, the focus acquisition module 451 is configured to obtain a DFR from the computing device 202 ( Figure 2A-2D ) obtains an indication of an active user interface element, the active user interface element being displayed on the peripheral display device 204 ( Figure 2A-2D ) of the graphical user interface on the primary display 102. In some embodiments, the DFR determination module 452 is configured to determine a graphic (e.g., a set of one or more affordances) based on the active user interface element in the current focus. Alternatively, the computing device 202 ( Figure 2A-2D) determines a graphic (e.g., a set of one or more affordances) based on the active user interface element in focus, and provides the graphic to the peripheral electronic device 400 or a component thereof (e.g., the DFR module 450) for display on the touch-sensitive display system 412 (e.g., the dynamic function row 104, Figure 2A-2D In some embodiments, the DFR rendering module 453 is configured to render the dynamic function row 104, Figure 2A-2D ) on the touch-sensitive display system 412 (or provided by the computing device 202). The DFR rendering module 453 includes various known software components for causing and resulting in graphics to be displayed on the touch-sensitive display system 412, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. In some embodiments, the DFR module 450 includes other modules for performing the following operations: adjusting the sensitivity of the dynamic function row 104; adjusting the auditory and / or tactile feedback provided by the dynamic function row 104; adjusting the settings of the enabling representations and information (e.g., size, brightness, font, language, etc.) displayed by the dynamic function row 104; adjusting the current power mode (e.g., normal and low power modes) of the dynamic function row 104; and so on.

[0149] In some embodiments, memory 402 includes event classifier 470 (e.g., in operating system 426). In some embodiments, event classifier 470 performs the same operations as event classifier 370 ( Figure 3B ) and includes event classifier 370 ( Figure 3B )'s modules, processes, and instructions. Therefore, for the sake of brevity, event classifier 470 will not be described.

[0150] It should be understood that the peripheral electronic device 400 is merely an example, and that the peripheral electronic device 400 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 4 The various components shown in the drawings are implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing circuits and / or application specific integrated circuits).

[0151] Each of the above-mentioned identified modules corresponds to an executable instruction set for performing the above-mentioned one or more functions and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 402 optionally stores a subset of the above-mentioned modules and data structures. In addition, memory 402 optionally stores additional modules and data structures not described above.

[0152] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., touchpad 355 or touchpad 356 in FIG. 3 ). Figure 4 In some implementations of a touch screen display that includes a touch screen display that enables direct interaction with user interface elements on the touch screen display, the contact detected on the touch screen acts as a "focus selector" so that when input is detected (e.g., a press input by a contact) at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted according to the detected input. In some implementations, the focus moves from one area of ​​the user interface to another area of ​​the user interface without corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user to convey the user's desired interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0153] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of the contact as an attribute of the user input allows the user to access additional device functions that the user cannot easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical controls / mechanical controls such as knobs or buttons).

[0154] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of the portable computing system 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0155] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples sampled during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact intensity increases and / or before or after contact intensity decreases). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of contact intensity, the mean value of contact intensity, the average value of contact intensity, the value at the top 10% of contact intensity, the half-maximum value of contact intensity, the 90% maximum value of contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the intensity of the contact over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this example, a contact whose feature strength does not exceed the first threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the second threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more strength thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0156] In some embodiments, a portion of a gesture is identified for determining a characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an end position (e.g., a drag gesture), where the strength of the contact increases. In this example, the characteristic strength of the contact at the end position may be based only on a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only the swipe contact portion at the end position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the strength of the swipe contact for the purpose of determining the characteristic strength.

[0157] In some embodiments, one or more predefined intensity thresholds are used to determine whether a particular input meets the intensity-based criteria. For example, the one or more predefined intensity thresholds include (i) a contact detection intensity threshold IT0, (ii) a light press intensity threshold IT1, and (iii) a touch detection intensity threshold IT2. L (iii) Deep compression intensity threshold IT D (e.g., it is at least initially above 1 L ), and / or (iv) one or more other intensity thresholds (e.g., below I L The intensity threshold I H ). In some embodiments, the light press strength threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or a touchpad. In some embodiments, the deep press strength threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or a touchpad. In some embodiments, when a contact having a characteristic intensity lower than the light press strength threshold (e.g., and higher than a nominal contact detection intensity threshold IT0, contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing an operation associated with the light press strength threshold or the deep press strength threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface illustrations.

[0158] In some embodiments, the response of the device to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, the strength of the contact exceeding the first strength threshold during the input triggers a first response. In some embodiments, the response of the device to the input detected by the device depends on a criterion including both the contact strength during the input and a time-based criterion. For example, for some "deep press" inputs, as long as a delay time passes between satisfying the first strength threshold and satisfying the second strength threshold, the strength of the contact exceeding the second strength threshold greater than the first strength threshold of the light press during the input triggers a second response. The duration of the delay time is typically less than 200ms (e.g., 40ms, 100ms, or 120ms, depending on the magnitude of the second strength threshold, where the delay time increases as the second strength threshold increases). The delay time helps avoid accidental deep press inputs. As another example, for some "deep press" inputs, a period of reduced sensitivity will occur after reaching the first strength threshold. During this period of reduced sensitivity, the second strength threshold is increased. This temporary increase in the second strength threshold also helps avoid accidental deep press inputs. For other deep press inputs, the response to detecting the deep press input is not dependent on time-based criteria.

[0159] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Exemplary factors are described in U.S. patent application serial numbers 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety.

[0160] For example, Figure 3C A dynamic intensity threshold 480 is shown that changes over time based in part on the intensity of touch input 476 over time. Dynamic intensity threshold 480 is the sum of two components: a first component 474 that decays over time after a predefined delay time p1 from when touch input 476 is initially detected, and a second component 478 that tracks the intensity of touch input 476 over time. The initial high intensity threshold of first component 474 reduces accidental triggering of a “deep press” response while still allowing an immediate “deep press” response if touch input 476 provides sufficient intensity. Second component 478 reduces unintentional triggering of a “deep press” response through gradual intensity fluctuations of the touch input. In some embodiments, when touch input 476 satisfies dynamic intensity threshold 480 (e.g., at Figure 3C ), triggering a "deep press" response.

[0161] Figure 3D Another dynamic intensity threshold 486 (eg, intensity threshold I D ). Figure 3D Two other intensity thresholds are also shown: a first intensity threshold I H and the second intensity threshold I L .exist Figure 3D Although the touch input 484 satisfies the first intensity threshold I before time p2, H and the second intensity threshold I L , but no response is provided until the delay time p2 elapses at time 482. In addition, Figure 3D In the example, the dynamic intensity threshold 486 decays over time, wherein the decay is from time 482 (when the trigger is connected to the second intensity threshold I L This type of dynamic intensity threshold reduction begins immediately after the trigger is triggered with a lower threshold intensity (such as the first intensity threshold I H Or the second intensity threshold I L ) is triggered after or simultaneously with the response associated with the dynamic intensity threshold I D The associated response.

[0162] Figure 3E Another dynamic intensity threshold 492 (eg, intensity threshold I D ).exist Figure 3E In the example, after a delay time p2 has passed since the touch input 490 was initially detected, the trigger is triggered with the intensity threshold I L At the same time, the dynamic intensity threshold 492 decays after a predefined delay time p1 has elapsed since the touch input 490 was initially detected. L The associated response then decreases the intensity of the touch input 490, then increases the intensity of the touch input 490 without releasing the trigger associated with the intensity threshold I D The touch input 490 associated with the response (e.g., at time 494) may be detected even when the intensity of the touch input 490 is below another intensity threshold, such as the intensity threshold I. L The same is true when.

[0163] The characteristic intensity of the contact is below the light press intensity threshold IT L The intensity increases to between the light pressure intensity threshold IT L and deep compression intensity threshold IT D The intensity between IT and the deep press intensity threshold is sometimes referred to as a "light press" input. D The intensity of the deep compression increases to above the deep compression intensity threshold IT D The intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. L A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact on the touch surface. A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact lifted from the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some illustrations, shaded circles or ellipses are used to represent the intensity of a contact on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0164] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the strength of the contact (or multiple contacts) increases to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting that the strength of the corresponding contact increases to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and the strength of the contact subsequently decreases to below the press input strength threshold, and the corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases to below the press input threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0165] In some embodiments, the device employs intensity hysteresis to avoid unexpected input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the strength of the contact increases from an strength equal to or below a hysteresis strength threshold to an strength equal to or above a press input strength threshold and, optionally, the strength of the contact subsequently decreases to an strength equal to or below the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (for example, depending on the circumstances, the strength of the contact increases or the strength of the contact decreases).

[0166] For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input are optionally triggered in response to detecting that the intensity of the contact increases above the press input intensity threshold, the intensity of the contact increases from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreases below the press input intensity threshold, or the intensity of the contact decreases below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting that the intensity of the contact decreases below the press input intensity threshold, the operations are optionally performed in response to detecting that the intensity of the contact decreases below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, triggering of these operations is also dependent on satisfying time-based criteria (e.g., a delay time has elapsed between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0167] User interface and associated processes

[0168] Attention is now directed to the portable computing system 100 ( Figure 1A ) or desktop computing system 200 ( Figure 2A-2D ) implements an embodiment of a user interface ("UI") and associated processes. In some embodiments, the main display 102 is on the portable computing system 100 ( Figure 1A ) is implemented in the display portion 110 of the device. Alternatively, in some embodiments, the main display 102 is implemented in the peripheral display device 204 ( Figure 2A-2D ) is implemented in. In some embodiments, the dynamic function row 104 is implemented in the portable computing system 100 ( Figure 1A-1B ) is implemented in the main body 120 of the device. Alternatively, in some embodiments, the dynamic function row 104 is implemented in the peripheral keyboard 206 ( Figure 2A-2B )、the first peripheral input mechanism 212 ( Figure 2C ) or the second peripheral input mechanism 222 ( Figure 2D ) implemented in a touch-sensitive secondary display.

[0169] Figure 5A-Figure 5N is a schematic diagram of a primary display and a secondary display according to some embodiments, which is used to illustrate an exemplary user interface for enabling a low vision user to interact with a touch-sensitive secondary display. The user interfaces in these figures are used to illustrate the methods and / or processes described below, including Figure 6A-6C , Figure 7A-7C and Figure 8A-8CThe method in. Those of ordinary skill in the art will appreciate that the following user interfaces are merely examples. In addition, those of ordinary skill in the art will appreciate that additional affordances and / or user interface elements, or fewer affordances and / or user interface elements, may be used in practice.

[0170] Figure 5A The main display 102 is shown displaying a user interface for a system preference application, wherein an accessibility option of "enable touch bar zoom" is presented. Figure 5A As shown, the affordances displayed at the touch-sensitive secondary display 104 are used to activate or control application-specific functions associated with the system preference application currently displayed at the primary display 102. For example, the touch-sensitive secondary display includes application-specific affordances for enabling touch bar zooming, navigating between menu options (an affordance immediately following the system-level "esc" affordance); and a "show all" affordance for causing the primary display to exit the accessibility menu instead of presenting all system preference menu options. In addition to these application-specific affordances, Figure 5A Touch-sensitive secondary display 104 is also shown to include system-level affordances 5085-5088 for controlling system-level functions (as described below with reference to method 600).

[0171] Figure 5A A user selection of the Enable Touch Bar Zoom check box is shown using cursor 504 (in some embodiments, the primary display is also touch sensitive, and thus, the user may use their finger or stylus to provide input rather than using cursor 504). In response to the user selection of the check box, the Touch Bar Zoom accessible mode is then enabled for the touch sensitive secondary display 104 (as shown in FIG. Figure 5B). In some embodiments, the accessible mode is an operating mode in which certain features are activated to enable users with vision, hearing, physical and motor skills, or learning and reading disabilities to interact with their electronic devices (i.e., to allow people with disabilities to drive the user interface in a non-traditional manner), and these specific features are not available (or may be disabled) during the normal operating mode of the electronic device. For example, as described herein, the touch bar zoom accessible mode is an operating mode for a touch-sensitive secondary display in which interactions at the touch-sensitive secondary display cause enlarged representations of the affordances and user interfaces displayed at the touch-sensitive secondary display to be presented at the primary display, thereby enabling low vision users of the touch-sensitive secondary display to view and interact with these affordances and user interfaces. In other words, activating the touch bar zoom accessible mode allows a person with visual impairments to drive the user interface presented at the touch-sensitive secondary display in a non-traditional manner, i.e., by viewing the enlarged representation on the primary display and providing selections and other inputs at the touch-sensitive secondary display. In some embodiments, the touch bar zoom accessible mode is disabled by default, and must be activated using, for example Figure 5A-Figure 5B to activate it by clicking on the user interface shown.

[0172] Figure 5B Also included is user selection of mail icon 506 from application tray 514 including affordances 506-515 for activating various applications using cursor 504. In response to the user selection of mail icon 506, the user interface of mail / email application 580 is then presented on primary display 102 and touch-sensitive secondary display 104 is updated to include new application-specific affordances (e.g., affordances 5080-5084) corresponding to the functionality available within mail application 580.

[0173] Figure 5C Also shown is a user selecting the text "Everyone" within the body of an email displayed at the user interface of email application 580 using cursor 504. Figure 5D As shown, in response to the user selecting the text, the text is then highlighted at the primary display 102 and the touch-sensitive secondary display is updated to include application-specific enable representations (e.g., enable representations 5102, 5106, and 5089-5091) for manipulating how the selected text is presented or displayed.

[0174] Figure 5DAlso shown is input 5102 at touch-sensitive secondary display 104 that contacts affordance 5104, and in response to input 5102, primary display is updated to present an enlarged representation (e.g., enlarged representation 5204) of affordance 5104 that contacts, which is shown within user interface 5202 that includes enlarged representations of other affordances displayed at secondary display 104. Additionally, focus indicator 5206 is presented at user interface 5202 that is positioned to correspond to the location of input 5102 at secondary display 104. By providing a larger affordance and focus indicator, low vision users are enabled to interact with (and continue to interact with) touch-sensitive secondary display 104 in response to only a single input at secondary display 104 (as explained in more detail below with reference to methods 600-800).

[0175] Figure 5D Also shown is when input 5102 begins to move laterally across secondary display 104, and in response, user interface 5202 shifts in accordance with the lateral movement and displays an additional enlarged representation of the affordance on the primary display (e.g., Figure 5E As shown, user interface 5202 is offset to the right to display an additional magnified indication). Figure 5E A second input 5103 is shown contacting the touch-sensitive secondary display 104 (in response, a second focus indicator is presented at the magnified user interface 5202, which is positioned based on the location of the second input 5103 at the secondary display 104). Figure 5E As shown, input 5102 and second input 5103 are then used to provide a de-pinch gesture, wherein the two inputs move in substantially opposite directions across the secondary display 104, and in response to the de-pinch gesture, the zoom level of the magnified user interface 5202 increases (e.g., Fig. 5F ). In some embodiments, before providing the de-pinch gesture, the user also presses a key, such as a "command" key, to modify the zoom level of the magnified user interface via the de-pinch gesture.

[0176] In some embodiments, the user can also reduce the zoom level by providing a pinch gesture (eg, Fig. 5F 5102 and 5103 in the input), and in response, the zoom level of the magnification user interface 5202 is reduced accordingly (as shown in FIG. Figure 5G shown). Figure 5G It is then shown moving sideways across the secondary display 104 and returning to its original position ( Figure 5H ) input 5102, which is in Figure 5D Shown in. Figure 5HIt is shown that the user can activate affordance 5104 by providing a split tap gesture, such as holding input 5102 at secondary display 104 and additionally providing a momentary tap 5106 at secondary display 104. In response, the appearance of focus indicator 5206 may change ( Figure 5K ) and then indicate that 5104 and 5204 are able to move across their respective associated sliders.

[0177] Alternatively, instead of providing a split tap gesture, the user may maintain contact with enable indication 5104 for more than a predetermined period of time (e.g., 0.5 seconds, 0.75 seconds, or 1 second), and in response, the representation of countdown timer 5208 is updated to count down until enable indication 5104 is activated (e.g., Figure 5I-5K As shown, where a countdown timer surrounding focus indicator 5206 moves in a clockwise direction until it expires and indicators 5104 and 5204 can then be moved).

[0178] like Figure 5L As shown, the user can then move affordances 5104 and 5204 across their respective sliders to manipulate the text size associated with the selected “Everyone” text. For example, as the user moves affordances 5104 / 5204 across the sliders, the text size of “Everyone” dynamically increases in accordance with the movement across the sliders ( Figure 5L ).

[0179] Now go to Figure 5M , provides an additional example of a split tap gesture in which input 5102 remains in contact with an affordance (e.g., affordance 5089 for bolding text in an email application), and a momentary tap 5107 that is not over the affordance is detected. In response to the split tap gesture, the affordance contacted by input 5102 (e.g., affordance 5089) is activated, and a function associated with the affordance (e.g., such as Figure 5N As shown, the selected text "Everyone" now appears bold). Figure 5N It is also shown that the input 5102 has been lifted off the touch-sensitive secondary display 104 , and in response, the magnified representation is no longer presented at the primary display 102 .

[0180] The following reference methods 600-800 provide information about Figure 5A-Figure 5N Additional description of .

[0181] Figure 6A-6C6 is a flow chart of a method 600 of enabling a low vision user to interact with a touch-sensitive secondary display according to some embodiments. The method 600 is performed (602) at a computing system that includes one or more processors, a memory, a first housing including a primary display, and a second housing that at least partially contains a touch-sensitive secondary display different from the primary display. Some operations in the method 600 are optionally combined, and / or the order of some operations is optionally changed.

[0182] In some embodiments, the computing system is a portable computing system 100 ( Figure 1A ) or desktop computing system 200 ( Figure 2A-2D In some embodiments, the primary display is on the portable computing system 100 ( Figure 1A ) is implemented in a display portion 110 (also referred to herein as a first housing 110, which includes a main display 102) Figure 1A ), and the second housing is a main body portion 120 of the portable computing system 100, and the second housing at least partially contains a touch-sensitive secondary display (e.g., the dynamic function row 104, Figure 1A-1B ) and a physical keyboard (e.g., a set of physical keys 106) (604).

[0183] In some embodiments, the second housing is not connected to the first housing (606), for example, because the first housing is part of the first device and the second housing is part of a device that is different from the first device (e.g., the second housing may be part of a mobile phone, a tablet device, or any other device that includes an affordance displayed on a smaller secondary display while the other device is connected to a computing system that includes a larger primary display). As a non-limiting example, in some embodiments, the second housing is a peripheral keyboard 206 ( Figure 2A-2B ), which at least partially includes a touch-sensitive secondary display (e.g., a dynamic function row 104, Figure 2A-2B ) and a physical keyboard (e.g., a set of physical keys 106, Figure 2A-2B As another example, in some embodiments, the second housing is the first peripheral input mechanism 212 ( Figure 2C ), which at least partially includes a touch-sensitive secondary display (e.g., a dynamic function row 104, Figure 2C ), and the second housing includes an input mechanism (e.g., a touch pad 108, Figure 2C ) without including a physical keyboard. As yet another example, in some embodiments, the second housing is part of a wearable computing device (608), such as a smart watch.

[0184] As described below, method 600 (and associated interface) enables a low vision user to interact with a touch-sensitive secondary display. Fig. 6A As shown, method 600 includes displaying (610) a first user interface for an application on a primary display (e.g., Figure 5C , a first user interface for an email application is shown as being displayed on primary display 102). Method 600 also includes displaying (612) a second user interface on the touch-sensitive secondary display, the second user interface including a plurality of application-specific affordances that control functionality available within the application, with each of the plurality of application-specific affordances being displayed at the first display size. For example, the plurality of application-specific affordances may be selected to control context-dependent functionality within the application, such as functionality that the user will need to access based on what they are currently doing in the application (such as Figure 5C The multiple application-specific affordances shown may be used to send emails, add emoticons to email messages, and select various auto-complete options). In some embodiments or situations, the multiple application-specific affordances also change based on user interaction with the application (e.g., Figure 5C The user is shown using cursor 504 to select text in an email message, and in response to the selection of the text, a plurality of application-specific affordances displayed in touch-sensitive secondary display 104 are updated to include text editing options, such as slider knob 5104 for editing the size of the text by dragging it along slider 5106, and affordances 5082-5084 for selecting bold, italic, and underline options, as shown. Figure 5D Additional details and numerous examples of how affordances may change at a touch-sensitive secondary display based on user interaction at a primary display are provided in commonly owned U.S. patent application Ser. No. 15 / 275,298, which is hereby incorporated by reference in its entirety.

[0185] In some embodiments, and as Figure 5C As shown, the touch-sensitive secondary display also includes one or more system-level enable indications for activating or controlling system-level functions (e.g., an enable indication 5079 for executing a leave function, an enable indication 5085 for controlling the brightness of the primary display 102, an enable indication 5086 for controlling the volume level of the computing system, an enable indication 5087 for muting the computing system, and an enable indication 5088 for controlling or activating a virtual personal assistant available via the computing system).

[0186] In some embodiments, each of the application-specific affordances is (only) selectable via one or more inputs at the touch-sensitive secondary display (614). In other words, the affordance (any of the multiple application-specific affordances and any system-level affordances) may be selected in any manner other than by providing an input at the secondary display. In some embodiments, the one or more inputs include a quick tap that includes a lift-off of the input shortly thereafter followed by a tap; a split tap (described below); and a selection event that occurs after a countdown timer expires (described below).

[0187] Method 600 also includes detecting (616) input contacting at least one of the plurality of application-specific affordances via the touch-sensitive secondary display (e.g., input 5102 at affordance 5104, Figure 5D In response to detecting the input and while the input remains in contact with the touch-sensitive secondary display, method 600 includes (618) continuing to display the first user interface for the application on the primary display (e.g., the primary display continues to display the first user interface for the email application), and displaying an enlarged representation of at least one application-specific affordance on the primary display (e.g., enlarged representation 5204). The enlarged representation of the at least one application-specific affordance is displayed at a second display size that is larger than the first display size (e.g., the primary display continues to display the first user interface for the email application). Figure 5D As shown, enable indication 5204 is shown at a display size larger than enable indication 5104).

[0188] In some cases, users of a computing system (specifically, users with low vision) are unable to accurately view icons or affordances displayed at a small display size (such as those displayed on a smartwatch). Populating a touch-sensitive secondary display with application-specific affordances, and then displaying an enlarged representation of one of these affordances on a larger primary display in response to a single input (as described above), provides these users with clear visual feedback indicating which affordance they can select. Providing this improved visual feedback to the user enhances the operability of the device and makes the human-computer interface more efficient (e.g., by helping the user provide correct inputs and reducing user errors when operating / interacting with affordances displayed on the secondary display). In addition, allowing these users to accurately view affordances displayed on a small screen enables continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the time required to waste correcting these errors.

[0189] In some embodiments, displaying the enlarged representation of at least one application-specific affordance includes (620) displaying the enlarged representation of at least one application-specific affordance within the enlarged representation of the second user interface (e.g., enlarged affordance 5204 is displayed within enlarged representation 5202 of the second user interface). In some embodiments, displaying the enlarged representation of the second user interface includes displaying an enlarged representation of the second application-specific affordance on the primary display adjacent to the application-specific affordance on the touch-sensitive secondary display.

[0190] Displaying an enlarged view of at least one application-specific enable representation within an enlarged representation of a second user interface in response to a single input provides clear visual feedback to the user, indicating which enable representation they may be selecting and indicating which enable representations are located nearby. Providing this improved visual feedback to the user enhances the operability of the device and makes the human-computer interface more efficient (e.g., by helping the user provide correct inputs and reducing user errors when operating / interacting with the enable representations and other adjacent enable representations displayed on the secondary display). In addition, allowing these users to accurately view the enable representations displayed on the small screen enables continuous interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the time required to correct these errors.

[0191] In some other embodiments, instead of (or in addition to) displaying an enlarged representation of at least one application-specific enabling representation within an enlarged representation of a second user interface, an enlarged representation of at least one application-specific enabling representation is displayed in a central area of ​​the primary display (e.g., overlaying a first user interface for an email application).

[0192] Now go to Figure 6B In some embodiments, displaying the enlarged representation of the second user interface includes (622) displaying a focus indicator (e.g., focus indicator 5206, including a representation of countdown timer 5208, within the enlarged representation of the second user interface) . Figure 5D), the focus indicator being positioned based at least in part on the location where the input on the touch-sensitive secondary display contacts the touch-sensitive secondary display (e.g., the location of the focus indicator 5206 within the magnified representation 5202 generally corresponds to the location of the input 5102 at the touch-sensitive secondary display 104). In some embodiments, the focus indicator is displayed in response to the input and provides low vision users with a clear visual indication of where their finger is located on the touch-sensitive secondary display to allow these users to accurately select displayed affordances from within the touch-sensitive secondary display. Displaying the focus indicator within the magnified representation of the second user interface provides users with clear visual feedback about the location of their finger on the touch-sensitive secondary display. In some cases, a user may not be able to see a small affordance displayed on the touch-sensitive secondary display due to vision problems or because their finger obscures an affordance located below. Therefore, providing a focus indicator within the magnified representation enhances the operability of the device and makes the human-computer interface more efficient (e.g., by helping the user provide correct input and reducing user errors when operating affordances displayed on the secondary display and other adjacent affordances / interacting with the affordances and other adjacent affordances). Additionally, allowing these users to accurately understand the location of their fingers on the touch-sensitive secondary display enables continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the need to waste time correcting these errors.

[0193] In some embodiments, the focus indicator includes (624) a representation of a countdown timer (e.g., representation 5208 of a countdown timer surrounding focus indicator 5206). In some embodiments, method 600 includes: based on determining that the input has remained in contact with at least one application-specific affordance for more than a predetermined amount of time (which determination may also include determining that the user has not moved the input more than a threshold distance (e.g., 5 px) within a threshold time period (e.g., 1 second), updating the representation of the countdown timer to indicate that the countdown timer is active. For example, Figure 5H-5KAs shown, the representation 5208 of the countdown timer surrounding the focus indicator 5206 moves in a clockwise direction to reflect that the timer is counting down. Based on determining that the countdown timer has expired (e.g., after a period of 1, 1.5, or 2 seconds), method 600 includes: activating at least one application-specific enabling representation. Displaying a focus indicator with a representation of a countdown timer provides users with clear visual feedback that they are about to select an enabling representation, and the countdown timer starts counting down after the input has been in contact with the enabling representation for more than a predetermined period of time. In some cases, users may not be able to see the small enabling representation displayed on the touch-sensitive secondary display due to visual problems or because their fingers cover the enabling representation located below, and therefore, these users may not understand when they activate / select various enabling representations. Therefore, providing a focus indicator with a countdown timer enhances the operability of the device and makes the human-computer interface more effective (e.g., by helping users provide correct inputs and reducing user errors when operating the enabling representation and other adjacent enabling representations displayed on the secondary display / interacting with the enabling representation and other adjacent enabling representations). Additionally, allowing these users to know exactly when they are about to activate an affordance enables continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the time required to correct these errors.

[0194] In some embodiments, activating an application-specific enable representation may include any of the following: (i) executing a function associated with at least one application-specific enable representation (such as a function available via an application); (ii) expanding the enable representation within the touch-sensitive secondary display and the primary display to display additional functionality (such as expanding the volume enable representation to display a slider that allows the volume setting to be modified); and (iii) allowing the user to move the enable representation along the slider. In some embodiments, and as explained below, activating at least one application-specific enable representation may be performed based on receiving other types of input. For example, activation of an enable representation may occur based on determining that a quick tap gesture has been received on at least one application-specific enable representation, or that a separate tap has been received that selects at least one application-specific enable representation (i.e., the first input is on the enable representation, and the second tap is not on the enable representation, but results in selection of the enable representation).

[0195] In some embodiments, at least one application-specific enable representation is associated with a slider (626) (e.g., enable representation 5104 is associated with slider 5106), and activating the at least one application-specific enable representation includes updating an enlarged representation of the at least one application-specific enable representation (e.g., as shown in FIG. 5A ) based on (or more generally, in order to allow) movement of the at least one application-specific enable representation along the slider. Figure 5KAs shown, in response to the countdown timer of focus indicator 5206 having expired, enable indication 5204 is now movable along the slider). In some embodiments, at least one application-specific enable indication is not slidable until it is activated after the countdown timer expires (or after it is activated in response to the quick tap and split tap inputs described above). In some embodiments, the touch-sensitive secondary display is also updated to allow the same movement of at least one application-specific enable indication (e.g., enable indication 5104 is now moving along slider 5106). In some embodiments, after the countdown timer expires, the visual characteristics used to present the focus indicator (or associated with the focus indicator) are modified to indicate that the focus indicator is now able to move at least one application-specific enable indication (e.g., as shown in FIG. 5B ). Figure 5K-5L As shown, the countdown timer representation maintains its expired state to indicate that the enabling representation located underneath is now moveable / slidable).

[0196] Activating the affordance associated with the slider after the countdown timer expires helps enhance the operability of the device and makes the human-machine interface more efficient (e.g., by reducing user errors when operating / interacting with the affordance and helping to avoid accidental modification of the slider). Additionally, allowing the user to accurately understand when they are able to manipulate the slider enables continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental modification of the slider) and the time required to correct these errors.

[0197] In some embodiments, based on determining that the input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time (e.g., 0.5 seconds, 0.75 seconds, or 1 second elapses while input 5012 remains in continuous contact with touch-sensitive secondary display 104 before the enlarged representation is presented at the primary display), an enlarged representation of at least one application-specific affordance is displayed (628) on the primary display. In some embodiments, if the input maintains such continuous contact with the touch-sensitive secondary display, the device determines that the input is not a selection or tap input, and based on this determination, the device then presents an enlarged representation of at least one application-specific affordance at the primary display (in some embodiments, and as described below, the device makes this determination only when the touch-sensitive secondary display is operating in an accessible mode).

[0198] In some embodiments, based on determining that the touch-sensitive secondary display is operating in an accessible mode, an enlarged representation of at least one application-specific affordance is displayed (630). (For example, by selecting a system preference to activate the accessible mode, such as Figure 5A-Figure 5B As shown, and explained in more detail below with reference to method 700).

[0199] Reference now Figure 6CIn some embodiments, method 600 includes: while the input maintains contact with at least one application-specific affordance at the touch-sensitive secondary display, detecting (632) a tap gesture (e.g., tap 5105, Figure 5H ), the touch-sensitive secondary display does not contact at least one application-specific enable representation. In some embodiments, the touch-sensitive secondary display includes a first area having at least one application-specific enable representation and a second area having other enable representations of a plurality of application-specific enable representations (i.e., these other enable representations do not include at least one application-specific enable representation), and the tap gesture is received at the second area and therefore does not contact at least one application-specific enable representation. In some embodiments, the "split tap gesture" is an alternative selection option (without having to wait for a countdown timer to expire). In response to detecting the tap gesture, method 600 includes: activating at least one application-specific enable representation (e.g., allowing a user to move enable representation 5204 / 5104 along a corresponding slider, as described above with reference to Figure 5K-5L Another example is in Figure 5M-Figure 5N , where input 5102 is maintained in contact with the touch-sensitive secondary display at enable representation 5090, and then a tap gesture 5107 is received on enable representation 5090, and in response, enable representation 5090 is activated, which causes the selected text "Everyone" at primary display 102 to be bolded. Allowing activation of an enable representation in contact with an input in response to a tap gesture that is not on an enable representation (a split tap gesture) enhances the operability of the device and makes the human-computer interface more efficient (e.g., by allowing the user to place a first finger on a desired enable representation and then use a different finger to perform a selection of the desired enable representation). In addition, allowing the user to freely move their first finger around the touch-sensitive secondary display enables the user to maintain continuous interaction with the touch-sensitive secondary display (by exploring which enable representations are displayed at the touch-sensitive secondary display), which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental selection of an enable representation) and the need to waste time correcting these errors.

[0200] In some embodiments, the user is also able to use a split tap gesture to activate various types of affordances. For example, in some embodiments, the user is able to manipulate the date or time range of a calendar event by first placing their finger on either end of the date range modification affordance (such as the examples shown in Figures 36J-36Q of co-owned patent application 15 / 275,298, incorporated by reference above). In some embodiments, the main display is then updated to include a focus indicator with a countdown timer at each end of the date range modification affordance, and the user is able to manipulate the date range when two of the countdown timers expire (such as Figure 5E-5FAs shown, focus indicators 5206 and 5212 are provided at the touch-sensitive secondary display with each input). In some embodiments, a user can modify one end of an affordance that includes a range of values ​​(e.g., to manipulate only one end of the affordance) by allowing a countdown timer for the focus indicator to expire at either end of the affordance.

[0201] In some embodiments, method 600 includes: detecting (634) a predefined gesture at the touch-sensitive secondary display (e.g., a user pressing a command button and then performing a gesture with inputs 5102 and 5103 at the touch-sensitive secondary display ( Fig. 5F ) or a pinching gesture with inputs 5102 and 5103 ( Figure 5E ), the predefined gesture manipulating a zoom level for displaying an enlarged representation of at least one application-specific affordance at the primary display. In response to detecting the predefined gesture (or in response to detecting each incremental manipulation of the zoom level during the predefined gesture), method 600 includes: updating the enlarged representation at the primary display while the zoom level is manipulated using the predefined gesture (e.g., in response to Figure 5E further zoom in with a de-pinch gesture, or in response to Fig. 5F Allowing users to manipulate the zoom level of a magnified representation using predefined gestures enhances the operability of the device and makes the human-computer interface more effective (e.g., by allowing users to quickly and easily adjust the zoom level to suit their personal preferences). Additionally, allowing users to manipulate the zoom level enables users to maintain continuous interaction with the touch-sensitive secondary display by ensuring that such users are able to adjust the zoom level so that they can accurately view affordances that can be displayed at different display sizes, which is important for low vision users of varying vision levels.

[0202] It should be understood that Figure 6A-6C The specific order in which the operations are described is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art would think of many ways to reorder the operations described herein.

[0203] Figure 7A-7C 700 is a flow chart of a method 700 of enabling a low vision user to interact with a touch-sensitive secondary display according to some embodiments. The method 700 is performed (702) at a computing system that includes one or more processors, a memory, a first housing including a primary display, and a second housing that at least partially contains a touch-sensitive secondary display different from the primary display. Some operations in the method 700 are optionally combined, and / or the order of some operations is optionally changed.

[0204] In some embodiments, the computing system is a portable computing system 100 ( Figure 1A ) or desktop computing system 200 ( Figure 2A-2D In some embodiments, the primary display is on the portable computing system 100 ( Figure 1A ) is implemented in a display portion 110 (also referred to herein as a first housing 110, which includes a main display 102) Figure 1A ), and the second housing is a main body portion 120 of the portable computing system 100, and the second housing at least partially contains a touch-sensitive secondary display (e.g., the dynamic function row 104, Figure 1A-1B ) and a physical keyboard (e.g., a set of physical keys 106) (704).

[0205] In some embodiments, the second housing is not connected to the first housing (706), for example, because the first housing is part of the first device and the second housing is part of a device that is different from the first device (e.g., the second housing may be part of a mobile phone, a tablet device, or any other device that includes an affordance displayed on a smaller secondary display while the other device is connected to a computing system that includes a larger primary display). As a non-limiting example, in some embodiments, the second housing is a peripheral keyboard 206 ( Figure 2A-2B ), which at least partially includes a touch-sensitive secondary display (e.g., a dynamic function row 104, Figure 2A-2B ) and a physical keyboard (e.g., a set of physical keys 106, Figure 2A-2B As another example, in some embodiments, the second housing is the first peripheral input mechanism 212 ( Figure 2C ), which at least partially includes a touch-sensitive secondary display (e.g., a dynamic function row 104, Figure 2C ), and the second housing includes an input mechanism (e.g., a touch pad 108, Figure 2C ) without including a physical keyboard. As yet another example, in some embodiments, the second housing is part of a wearable computing device (708), such as a smart watch.

[0206] As described below, method 700 (and associated interface) enables a low vision user to interact with a touch-sensitive secondary display. Compared to method 600, method 700 includes operating the touch-sensitive secondary display in an accessible mode before providing an enlarged representation at the primary display. The examples and descriptions provided above with reference to method 600 also apply to method 700, and for the sake of brevity, these examples and descriptions are generally not repeated here. Fig. 7A As shown, method 700 includes: operating (710) the touch-sensitive secondary display in an accessible mode. Figure 5A-Figure 5BAs shown, a system preference option is available that allows the user to enable an accessible mode for the touch-sensitive secondary display (e.g., by selecting a checkbox to "Enable touch bar zoom" using cursor 504). In some embodiments, a check mark option is also presented at the touch-sensitive secondary display 104, which allows the user to enable or disable the accessible mode by providing input at the touch-sensitive secondary display 104 (e.g., Figure 5A-Figure 5B shown).

[0207] The remaining operations of method 700 (operations 714-738) are each performed when the touch-sensitive secondary display is operated in an accessible mode (712). Method 700 includes displaying (714) a first user interface for an application on the primary display (e.g., in Figure 5C ). An exemplary user interface is shown for an email application in ). The method also includes displaying (716) a second user interface on the touch-sensitive secondary display, the second user interface comprising: (i) multiple application-specific enable representations that control functionality available within the application (e.g., enable representations 5080-5084, additional descriptions of which are provided above with reference to method 600), and (ii) at least one system-level enable representation that controls system-level functionality (e.g., enable representations 5079 and 5085-5088, additional descriptions of which are provided above with reference to method 600). Each of the multiple application-specific enable representations and at least one system-level enable representation are displayed at a first display size. In some embodiments, each respective enable representation is (only) selectable via one or more inputs at the touch-sensitive secondary display (718) (additional details regarding operation 718 are provided above with reference to operation 614).

[0208] Method 700 also includes detecting (720) via the touch-sensitive secondary display an input (e.g., input 5102, Figure 5D In response to detecting the input and while the input remains in contact with the touch-sensitive secondary display (722), the method includes: continuing to display the first user interface for the application on the primary display, and displaying an enlarged representation of at least one application-specific affordance on the primary display. The enlarged representation of the at least one application-specific affordance is displayed at a second display size that is larger than the first display size. Additional details and examples of operation 722 are provided above with reference to operation 618.

[0209] In some cases, low vision users of a computing system rely on memorizing key locations on a keyboard so that they can accurately provide input to the computing system. For computing systems that include a touch-sensitive secondary display with frequently changing affordances, these users are unable to rely solely on memory to provide accurate input. Displaying an enlarged representation of at least one affordance of application-specific affordances improves the operability of the computing system because the low vision user is able to interact with controls available at the touch-sensitive secondary display, which may be too small for the low vision user (or may be obscured because the user's fingers are covering the displayed controls) to accurately view. In this way, the low vision user is able to take advantage of the improved human-computer interface by, for example, continuously interacting with the touch-sensitive secondary display (rather than constantly correcting erroneous inputs).

[0210] Now go to Figure 7B , displaying an enlarged representation of at least one application-specific affordance includes (724) displaying an enlarged representation of at least one application-specific affordance within an enlarged representation of the second user interface. Additional details and examples regarding operation 724 are provided above with reference to operation 620. Displaying an enlarged view of at least one application-specific affordance within an enlarged representation of the second user interface in response to a single input provides the user with clear visual feedback indicating which affordance they may be selecting and which affordances are located near it. Providing this improved visual feedback to the user enhances the operability of the device and makes the human-computer interface more efficient (e.g., by helping the user provide correct input and reducing user errors when operating affordances and other adjacent affordances displayed on the secondary display / interacting with the affordances and other adjacent affordances). In addition, allowing these users to accurately view affordances displayed on the small screen enables continuous interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the need to waste time correcting these errors.

[0211] In some embodiments, displaying the enlarged representation of the second user interface includes (726) displaying a focus indicator within the enlarged representation of the second user interface, the focus indicator being positioned at least in part based on the location where the input on the touch-sensitive secondary display contacts the touch-sensitive secondary display. Additional details and examples of operation 726 are provided above with reference to operation 622. Displaying the focus indicator within the enlarged representation of the second user interface provides the user with clear visual feedback about the location of their finger on the touch-sensitive secondary display. In some cases, the user may not be able to see the small enable representation displayed on the touch-sensitive secondary display due to visual problems or because their finger obscures the enable representation located below. Therefore, providing a focus indicator within the enlarged representation enhances the operability of the device and makes the human-computer interface more effective (e.g., by helping the user provide correct input and reducing user errors when operating the enable representation and other adjacent enable representations displayed on the secondary display / interacting with the enable representation and other adjacent enable representations). In addition, allowing these users to accurately understand the location of their fingers on the touch-sensitive secondary display enables continuous interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the need to waste time correcting these errors.

[0212] In some embodiments, the focus indicator includes (728) a representation of a countdown timer, and method 700 includes: based on determining that the input has been in contact with at least one application-specific enable representation for more than a predetermined amount of time, method 700 includes updating the representation of the countdown timer to indicate that the countdown timer is active. Based on determining that the countdown timer has expired, method 700 includes activating at least one application-specific enable representation. Additional details and examples regarding operation 728 are provided above with reference to operation 624. Displaying a focus indicator with a representation of a countdown timer provides the user with clear visual feedback that they are about to select an enable representation, which begins counting down after the input has been in contact with the enable representation for more than a predetermined period of time. In some cases, users may not be able to see the small enable representation displayed on the touch-sensitive secondary display due to visual issues or because their fingers obscure the enable representation located below, and therefore, these users may not understand when they activate / select various enable representations. Thus, providing a focus indicator with a countdown timer enhances the operability of the device and makes the human-machine interface more efficient (e.g., by helping users provide correct input and reducing user errors when operating / interacting with affordances and other adjacent affordances displayed on the secondary display). Additionally, allowing such users to accurately understand when they are about to activate affordances enables continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the time required to correct these errors.

[0213] In some embodiments, at least one application-specific enable representation is associated with the slider (730), and activating the at least one application-specific enable representation includes updating an enlarged representation of the at least one application-specific enable representation based on (or more generally, in order to allow) movement of the at least one application-specific enable representation along the slider. Additional details and examples regarding operation 730 are provided above with reference to operation 626. Activating the enable representation associated with the slider after the countdown timer expires helps enhance the operability of the device and makes the human-computer interface more efficient (e.g., by reducing user errors when operating / interacting with the enable representation, and helping to avoid accidental modifications of the slider). In addition, allowing the user to accurately understand when they are able to manipulate the slider to achieve continuous interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental modifications of the slider) and the need to waste time correcting these errors.

[0214] In some embodiments, based on determining that the input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time, an enlarged representation of at least one application-specific affordance is displayed (732) on the primary display. Additional details and examples regarding operation 732 are provided above with reference to operation 628.

[0215] Now turn your attention to Figure 7C In some embodiments, method 700 further includes: while the input maintains contact with at least one application-specific affordance at the touch-sensitive secondary display, detecting (736) a tap gesture (e.g., tap 5107, Figure 5M , or tap 5105, Figure 5H), the touch-sensitive secondary display is not in contact with at least one application-specific enable representation. In some embodiments, the touch-sensitive secondary display includes a first area having at least one application-specific enable representation and a second area having other enable representations of a plurality of application-specific enable representations (i.e., these other enable representations do not include at least one application-specific enable representation), and the tap gesture is received at the second area and therefore does not contact at least one application-specific enable representation. In response to detecting the tap gesture, method 700 includes: activating at least one application-specific enable representation. Additional details and examples regarding operation 736 are provided above with reference to operation 632. Allowing activation of an enable representation in contact with an input in response to a tap gesture that is not on an enable representation (a "split tap gesture") enhances the operability of the device and makes the human-computer interface more efficient (e.g., by allowing a user to place a first finger on a desired enable representation and then use a different finger to perform a selection of the desired enable representation). Additionally, allowing the user to freely move their first finger around the touch-sensitive secondary display enables the user to maintain continuous interaction with the touch-sensitive secondary display (by exploring which affordances are displayed at the touch-sensitive secondary display), which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental selection of affordances) and the need to waste time correcting these errors.

[0216] In some embodiments, method 700 includes: detecting (738) a predefined gesture (such as those exemplary predefined gestures discussed above with reference to operation 634) at the touch-sensitive secondary display, the predefined gesture manipulating a zoom level for displaying an enlarged representation of at least one application-specific affordance at the primary display. In response to detecting the predefined gesture (or in response to detecting each incremental manipulation of the zoom level), method 700 includes: updating the enlarged representation at the primary display as the zoom level is manipulated using the predefined gesture. Additional details and examples regarding operation 738 are provided above with reference to operation 634. Allowing a user to manipulate the zoom level of the enlarged representation using predefined gestures enhances the operability of the device and makes the human-computer interface more effective (e.g., by allowing the user to quickly and easily adjust the zoom level to accommodate their personal preferences). In addition, allowing the user to manipulate the zoom level enables users to maintain continuous interaction with the touch-sensitive secondary display by ensuring that these users are able to adjust the zoom level so that they can accurately view affordances that can be displayed at different display sizes, which is important for low vision users of various vision levels.

[0217] It should be understood that Figure 7A-7C The specific order in which the operations are described is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art would think of many ways to reorder the operations described herein.

[0218] Figure 8A-8C 8 is a flow chart of a method 800 for enabling a low vision user to interact with a touch-sensitive secondary display according to some embodiments. The method 800 is performed (802) at a computing system that includes one or more processors, a memory, a first housing including a primary display, and a second housing that at least partially contains a touch-sensitive secondary display different from the primary display. Some operations in the method 800 are optionally combined, and / or the order of some operations is optionally changed.

[0219] In some embodiments, the computing system is a portable computing system 100 ( Figure 1A ) or desktop computing system 200 ( Figure 2A-2D In some embodiments, the primary display is on the portable computing system 100 ( Figure 1A ) is implemented in a display portion 110 (also referred to herein as a first housing 110, which includes a main display 102) Figure 1A ), and the second housing is a main body portion 120 of the portable computing system 100, and the second housing at least partially contains a touch-sensitive secondary display (e.g., the dynamic function row 104, Figure 1A-1B ) and a physical keyboard (e.g., a set of physical keys 106) (804).

[0220] In some embodiments, the second housing is not connected to the first housing (806), for example, because the first housing is part of the first device and the second housing is part of a device that is different from the first device (e.g., the second housing may be part of a mobile phone, a tablet device, or any other device that includes an affordance displayed on a smaller secondary display while the other device is connected to a computing system that includes a larger primary display). As a non-limiting example, in some embodiments, the second housing is a peripheral keyboard 206 ( Figure 2A-2B ), which at least partially includes a touch-sensitive secondary display (e.g., a dynamic function row 104, Figure 2A-2B ) and a physical keyboard (e.g., a set of physical keys 106, Figure 2A-2B As another example, in some embodiments, the second housing is the first peripheral input mechanism 212 ( Figure 2C ), which at least partially includes a touch-sensitive secondary display (e.g., a dynamic function row 104, Figure 2C ), and the second housing includes an input mechanism (e.g., a touch pad 108, Figure 2C ) without including a physical keyboard. As yet another example, in some embodiments, the second housing is part of a wearable computing device (808), such as a smart watch.

[0221] As described below, method 800 (and associated interface) enables a low vision user to interact with a touch-sensitive secondary display. The examples and descriptions provided above with reference to methods 600 and 700 also apply to method 800, and for the sake of brevity, these examples and descriptions are generally not repeated here. Fig. 8A As shown, method 800 includes displaying (802) a first user interface for an application (eg, a user interface for an email application) on a primary display. Figure 5C ). Method 800 also includes displaying (812) a second user interface on the touch-sensitive secondary display, the second user interface including: (i) multiple application-specific enable representations that control functionality available within the application (e.g., enable representations 5080-5084, additional descriptions of these enable representations are provided above with reference to method 600), and (ii) at least one system-level enable representation that controls system-level functionality (e.g., enable representations 5079 and 5085-5088, additional descriptions of these enable representations are provided above with reference to method 600). In some embodiments, each respective enable representation is (only) selectable via one or more inputs at the touch-sensitive secondary display (814) (additional details regarding operation 814 are provided above with reference to operation 614).

[0222] In some embodiments, the first application-specific enable representation is displayed at a first display size at the touch-sensitive secondary display, and method 800 also includes (818): in response to detecting the first input and while the first input maintains contact with the first application-specific enable representation: continuing to display the first user interface for the application on the primary display; and displaying an enlarged representation of the first application-specific enable representation on the primary display, wherein the enlarged representation is displayed at a second display size that is larger than the first display size. Additional details and examples regarding operation 818 are provided above with reference to operation 618.

[0223] Method 800 further includes: detecting, via the touch-sensitive secondary display, a first input on a first application-specific affordance of the plurality of application-specific affordances (eg, input 5102 on affordance 5089, Figure 5M ). While the first input maintains contact with the first application-specific enable representation, method 800 includes (820): detecting, via the touch-sensitive secondary display, a second input that is not on the first application-specific enable representation (e.g., input 5107 that is not on enable representation 5089, Figure 5M ), and in response to detecting the second input, activating the first application-specific affordance (e.g., bolding selected text on the primary display due to activation of affordance 5089, Figure 5N). Additional examples and descriptions of operation 820 are provided above with reference to operation 632. Allowing activation of an affordance that is in contact with an input in response to a tap gesture that is not on an affordance (a "split tap gesture") enhances the operability of the device and makes the human-computer interface more efficient (e.g., by allowing the user to place a first finger on a desired affordance and then use a different finger to perform a selection of the desired affordance, thereby ensuring that only desired affordances are activated and helping to minimize erroneous selections / activations). Additionally, allowing the user to freely move their first finger around the touch-sensitive secondary display (without selecting affordances) enables the user to maintain continuous interaction with the touch-sensitive secondary display (by exploring which affordances are displayed at the touch-sensitive secondary display), which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental selections of affordances) and the time required to correct these errors.

[0224] Now turn your attention to Figure 8B . In some embodiments, displaying an enlarged representation of at least one application-specific enable representation includes (822) displaying an enlarged representation of at least one application-specific enable representation within an enlarged representation of the second user interface. Additional details and examples regarding operation 822 are provided above with reference to operation 620. Displaying an enlarged view of at least one application-specific enable representation within an enlarged representation of the second user interface in response to a single input provides the user with clear visual feedback indicating which enable representation they may be selecting and which enable representations are located near it. Providing this improved visual feedback to the user enhances the operability of the device and makes the human-computer interface more efficient (e.g., by helping the user provide correct input and reducing user errors when operating the enable representation and other adjacent enable representations displayed on the secondary display / interacting with the enable representation and other adjacent enable representations). In addition, allowing these users to accurately view the enable representation displayed on the small screen enables continuous interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the need to waste time correcting these errors.

[0225] In some embodiments, displaying an enlarged representation of the second user interface includes (824) displaying a focus indicator within the enlarged representation of the second user interface, the focus indicator being positioned at least in part based on the location where the input on the touch-sensitive secondary display contacts the touch-sensitive secondary display. Additional details and examples of operation 824 are provided above with reference to operation 622. Displaying a focus indicator within the enlarged representation of the second user interface provides the user with clear visual feedback about the location of their finger on the touch-sensitive secondary display. In some cases, the user may not be able to see the small enable representation displayed on the touch-sensitive secondary display due to visual problems or because their finger obscures the enable representation located below. Therefore, providing a focus indicator within the enlarged representation enhances the operability of the device and makes the human-computer interface more effective (e.g., by helping the user provide correct input and reducing user errors when operating the enable representation and other adjacent enable representations displayed on the secondary display / interacting with the enable representation and other adjacent enable representations). In addition, allowing these users to accurately understand the location of their fingers on the touch-sensitive secondary display enables continuous interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect selections) and the need to waste time correcting these errors.

[0226] In some embodiments, at least one application-specific enable representation is associated with the slider (828), and activating the at least one application-specific enable representation includes updating an enlarged representation of the at least one application-specific enable representation based on (or more generally, in order to allow) movement of the at least one application-specific enable representation along the slider. Additional details and examples regarding operation 828 are provided above with reference to operation 626. Activating the enable representation associated with the slider after the countdown timer expires helps enhance the operability of the device and makes the human-computer interface more efficient (e.g., by reducing user errors when operating / interacting with the enable representation, and helping to avoid accidental modifications of the slider). In addition, allowing the user to accurately understand when they are able to manipulate the slider to achieve continued interaction with the touch-sensitive secondary display, which would otherwise be impossible due to frequent errors (e.g., incorrect or accidental modifications of the slider) and the need to waste time correcting these errors.

[0227] In some embodiments, based on determining that the input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time, an enlarged representation of at least one application-specific affordance is displayed (830) on the primary display. Additional details and examples regarding operation 830 are provided above with reference to operation 628.

[0228] In some embodiments, based on determining that the touch-sensitive secondary display is operating in an accessible mode, an enlarged representation of at least one application-specific affordance is displayed. Additional details and examples regarding operation 832 are provided above with reference to operations 630, 710, and 712.

[0229] Now go to Figure 8C In some embodiments, method 800 includes: detecting (834) a predefined gesture (e.g., an example is provided above with reference to operation 634) at the touch-sensitive secondary display, the predefined gesture manipulating a zoom level for displaying an enlarged representation of at least one application-specific affordance at the primary display; and in response to detecting the predefined gesture, updating the enlarged representation at the primary display when the zoom level is manipulated using the predefined gesture. Additional details and examples regarding operation 834 are provided above with reference to operation 634. Allowing a user to manipulate the zoom level of the enlarged representation using predefined gestures enhances the operability of the device and makes the human-computer interface more effective (e.g., by allowing the user to quickly and easily adjust the zoom level to accommodate their personal preferences). In addition, allowing the user to manipulate the zoom level enables users to maintain continuous interaction with the touch-sensitive secondary display by ensuring that these users are able to adjust the zoom level so that they can accurately view affordances that can be displayed at different display sizes, which is important for low vision users of various vision levels.

[0230] It should be understood that Figure 8A-8C The specific order in which the operations are described is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art would think of many ways to reorder the operations described herein.

[0231] According to some embodiments, Fig. 9 A functional block diagram of a computing system 900 (also referred to as an electronic device 900 according to some embodiments) configured according to the principles of the various described embodiments is shown. The functional blocks of the system are optionally implemented by hardware, software, firmware, or a combination thereof that implements the principles of the various described embodiments. It should be understood by those skilled in the art that Fig. 9 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein. For ease of discussion, the computing system 900 is implemented as a portable computing system 100 ( Figure 1A In some embodiments, the computing system 900 is based on Figure 1A-Figure 2D It can be implemented by any of the devices / systems shown in .

[0232] like Fig. 9As shown, the computing system 900 includes a primary display unit 902 (e.g., a touch-sensitive display system 112, also referred to as a primary touch screen, a primary touch-sensitive display, and a primary touch screen display) configured to display information. Figure 1A ), a touch-sensitive secondary display unit 904 configured to receive contacts, gestures, and other user inputs on the touch-sensitive secondary display, and a processing unit 910. The system 900 optionally includes a physical keyboard unit 905 configured to receive keyboard input, and in some embodiments, the optional physical keyboard unit includes the touch-sensitive secondary display unit 904 in a second housing that is different from the first housing that includes the main display unit 902. In some embodiments, the first housing and the second housing are rotatably connected (for example, for embodiments in which the computing system 900 is a laptop computer), while in other embodiments, the first housing and the second housing are not connected and can be part of a separate computing device forming the system 900 (for example, the first housing is part of a laptop computer and the second housing is part of a separate device such as a wearable computing device (like a smart watch)). The processing unit 910 is coupled to the main display unit 902, the touch-sensitive secondary display unit 904, and the optional physical keyboard unit 905. In some embodiments, the processing unit 910 includes a display unit (e.g., display unit 912), a detection unit 914 (e.g., detection unit 914), and an enable indication activation unit (e.g., enable indication activation unit 916).

[0233] The processing unit is configured to: display a first user interface for an application on a primary display (e.g., using display unit 912 in combination with primary display unit 902); display a second user interface on a touch-sensitive secondary display (e.g., using display unit 912 in combination with touch-sensitive secondary display unit 904), the second user interface including a plurality of application-specific affordances for controlling functions available within the application, and each of the plurality of application-specific affordances being displayed at a first display size; and detect (e.g., using detection unit 914 in combination with touch-sensitive secondary display unit 904) the second user interface with the plurality of application-specific affordances for controlling functions available within the application. and (ii) displaying on the primary display (e.g., using display unit 912 in combination with primary display unit 902) an enlarged representation of at least one application-specific enable representation, and displaying the enlarged representation of at least one application-specific enable representation at a second display size that is larger than the first display size.

[0234] According to some embodiments of computing system 900, displaying an enlarged representation of at least one application-specific affordance includes displaying an enlarged representation of at least one application-specific affordance within an enlarged representation of the second user interface.

[0235] According to some embodiments of computing system 900, displaying an enlarged representation of the second user interface includes displaying a focus indicator within the enlarged representation of the second user interface, the focus indicator being positioned based at least in part on a location where input on the touch-sensitive secondary display contacts the touch-sensitive secondary display.

[0236] According to some embodiments of computing system 900, the focus indicator includes a representation of a countdown timer, and the processing unit is further configured to: based on determining that the input has maintained contact with at least one application-specific enable representation for more than a predetermined amount of time, update the representation of the countdown timer to indicate that the countdown timer is active (e.g., using display unit 912 in combination with main display unit 902); and based on determining that the countdown timer has expired, activate at least one application-specific enable representation (e.g., using enable representation activation unit 916).

[0237] In accordance with some embodiments of computing system 900, at least one application-specific enable representation is associated with a slider, and activating the at least one application-specific enable representation includes updating an enlarged representation of the at least one application-specific enable representation based on (or, more generally, in order to allow) movement of the at least one application-specific enable representation along the slider.

[0238] According to some embodiments of the computing system 900, the processing unit is further configured to: detect a tap gesture at the touch-sensitive secondary display (e.g., using detection unit 914 in combination with touch-sensitive secondary display unit 904) while the input remains in contact with at least one application-specific enable representation at the touch-sensitive secondary display, which does not contact the at least one application-specific enable representation; and in response to detecting the tap gesture, activate at least one application-specific enable representation (e.g., using enable representation activation unit 916). In some embodiments, the touch-sensitive secondary display includes a first area having at least one application-specific enable representation and a second area having other enable representations of a plurality of application-specific enable representations (i.e., these other enable representations do not include at least one application-specific enable representation), and the tap gesture is received at the second area and therefore does not contact the at least one application-specific enable representation.

[0239] According to some embodiments of the computing system 900, the processing unit is further configured to: detect (e.g., using detection unit 914 in combination with touch-sensitive secondary display unit 904) a predefined gesture at a touch-sensitive secondary display, the predefined gesture manipulating a zoom level, the zoom level being used to display an enlarged representation of at least one application-specific enable representation at the primary display; and in response to detecting the predefined gesture, update the enlarged representation at the primary display (e.g., using display unit 912 in combination with primary display unit 902) when the zoom level is manipulated using the predefined gesture.

[0240] According to some embodiments of computing system 900, based on determining that an input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time, an enlarged representation of at least one application-specific affordance is displayed on the primary display.

[0241] According to some embodiments of computing system 900, based on determining that the touch-sensitive secondary display is operating in an accessible mode, an enlarged representation of at least one application-specific affordance is displayed.

[0242] According to some embodiments of computing system 900, each of a plurality of application-specific affordances is selectable via one or more inputs at a touch-sensitive secondary display.

[0243] According to some embodiments of the computing system 900, the second housing also at least partially includes a physical keyboard.

[0244] According to some embodiments of computing system 900, the second housing is not connected to the first housing.

[0245] According to some embodiments of the computing system 900, the second housing is part of a wearable computing device.

[0246] According to some embodiments, Fig.10 A functional block diagram of a computing system 1000 (also referred to as an electronic device 1000 according to some embodiments) configured according to the principles of the various described embodiments is shown. The functional blocks of the system are optionally implemented by hardware, software, firmware, or a combination thereof that implements the principles of the various described embodiments. It should be understood by those skilled in the art that Fig.10 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein. For ease of discussion, the computing system 1000 is implemented as a portable computing system 100 ( Figure 1A In some embodiments, the computing system 1000 is based on Figure 1A-Figure 2D It can be implemented by any of the devices / systems shown in .

[0247] like Fig.10 As shown, the computing system 1000 includes a main display unit 1002 (e.g., a touch-sensitive display system 112, also referred to as a main touch screen, a main touch-sensitive display, and a main touch screen display, configured to display information. Figure 1A ), a touch-sensitive secondary display unit 1004 configured to receive contacts, gestures, and other user inputs on the touch-sensitive secondary display, and a processing unit 1010. The system 1000 optionally includes a physical keyboard unit 1005 configured to receive keyboard input, and in some embodiments, the optional physical keyboard unit includes the touch-sensitive secondary display unit 1004 in a second housing that is different from the first housing that includes the main display unit 1002. In some embodiments, the first housing and the second housing are rotatably connected (e.g., for embodiments in which the computing system 1000 is a laptop computer), while in other embodiments, the first housing and the second housing are not connected and can be part of a separate computing device forming the system 1000 (e.g., the first housing is part of a laptop computer and the second housing is part of a separate device such as a wearable computing device (like a smart watch)). The processing unit 1010 is coupled to the main display unit 1002, the touch-sensitive secondary display unit 1004, and the optional physical keyboard unit 1005. In some embodiments, the processing unit 1010 includes a display unit (e.g., display unit 1012), a detection unit 1014 (e.g., detection unit 1014), an enable indication activation unit (e.g., enable indication activation unit 1016), and an accessible mode operation unit (e.g., accessible mode operation unit 1018).

[0248] The processing unit is configured to: operate the touch-sensitive secondary display in an accessible mode; when operating the touch-sensitive secondary display in the accessible mode, display a first user interface for an application on the main display (e.g., using display unit 1012 in combination with main display unit 1002); and display a second user interface on the touch-sensitive secondary display (e.g., using display unit 1012 in combination with touch-sensitive secondary display unit 1004), the second user interface including: (i) multiple application-specific enable representations that control functions available within the application, and (ii) at least one system-level enable representation that controls system-level functions, and each of the multiple application-specific enable representations and at least one system-level enable representation are displayed in a first display size; via touch The touch-sensitive secondary display detects (e.g., using detection unit 1014 in combination with touch-sensitive secondary display unit 1004) input that contacts at least one application-specific enabling representation of a plurality of application-specific enabling representations; and in response to detecting the input, and while the input maintains contact with the touch-sensitive secondary display: continue to display (e.g., using display unit 1012 in combination with main display unit 1002) a first user interface for an application on the primary display, and display (e.g., using display unit 1012 in combination with main display unit 1002) an enlarged representation of at least one application-specific enabling representation on the primary display, and the enlarged representation of at least one application-specific enabling representation is displayed at a second display size that is larger than the first display size.

[0249] According to some embodiments of computing system 1000, displaying an enlarged representation of at least one application-specific affordance includes displaying an enlarged representation of at least one application-specific affordance within an enlarged representation of the second user interface.

[0250] According to some embodiments of computing system 1000, displaying an enlarged representation of the second user interface includes displaying a focus indicator within the enlarged representation of the second user interface, the focus indicator being positioned based at least in part on a location where input on the touch-sensitive secondary display contacts the touch-sensitive secondary display.

[0251] According to some embodiments of computing system 1000, the focus indicator includes a representation of a countdown timer, and the processing unit is further configured to: based on determining that the input has maintained contact with at least one application-specific enable representation for more than a predetermined amount of time, update the representation of the countdown timer to indicate that the countdown timer is active (e.g., using display unit 1012 in combination with main display unit 1002); and based on determining that the countdown timer has expired, activate at least one application-specific enable representation (e.g., using enable representation activation unit 1016).

[0252] According to some embodiments of computing system 1000, at least one application-specific enable representation is associated with a slider, and activating the at least one application-specific enable representation includes updating an enlarged representation of the at least one application-specific enable representation based on (or, more generally, in order to allow) movement of the at least one application-specific enable representation along the slider.

[0253] According to some embodiments of computing system 1000, the processing unit is further configured to: detect a tap gesture at the touch-sensitive secondary display while the input remains in contact with at least one application-specific enable representation at the touch-sensitive secondary display, which does not contact the at least one application-specific enable representation (e.g., using detection unit 1012 in combination with touch-sensitive secondary display unit 1004); and in response to detecting the tap gesture, activate at least one application-specific enable representation (e.g., using enable representation activation unit 1016). In some embodiments, the touch-sensitive secondary display includes a first area having at least one application-specific enable representation and a second area having other enable representations of a plurality of application-specific enable representations (i.e., these other enable representations do not include at least one application-specific enable representation), and the tap gesture is received at the second area and therefore does not contact the at least one application-specific enable representation.

[0254] According to some embodiments of the computing system 1000, the processing unit is further configured to: detect (e.g., using detection unit 1012 in combination with touch-sensitive secondary display unit 1004) a predefined gesture at a touch-sensitive secondary display, the predefined gesture manipulating a zoom level, the zoom level being used to display an enlarged representation of at least one application-specific enable representation at the primary display; and in response to detecting the predefined gesture, update the enlarged representation at the primary display (e.g., using display unit 1012 in combination with primary display unit 1002) when the zoom level is manipulated using the predefined gesture.

[0255] According to some embodiments of computing system 1000, based on determining that an input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time, an enlarged representation of at least one application-specific affordance is displayed on the primary display.

[0256] According to some embodiments of computing system 1000, each of a plurality of application-specific affordances is selectable via one or more inputs at a touch-sensitive secondary display.

[0257] According to some embodiments of computing system 1000, the second housing also at least partially includes a physical keyboard.

[0258] According to some embodiments of computing system 1000, the second housing is not connected to the first housing.

[0259] According to some embodiments of computing system 1000, the second housing is part of a wearable computing device.

[0260] According to some embodiments, Fig.11 A functional block diagram of a computing system 1100 (also referred to as an electronic device 1100 according to some embodiments) configured according to the principles of the various described embodiments is shown. The functional blocks of the system are optionally implemented by hardware, software, firmware, or a combination thereof that implements the principles of the various described embodiments. It should be understood by those skilled in the art that Fig.11 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein. For ease of discussion, the computing system 1100 is implemented as a portable computing system 100 ( Figure 1A In some embodiments, the computing system 1100 is based on Figure 1A-Figure 2D It can be implemented by any of the devices / systems shown in .

[0261] like Fig.11 As shown, the computing system 1100 includes a main display unit 1102 (e.g., a touch-sensitive display system 112, also referred to as a main touch screen, a main touch-sensitive display, and a main touch screen display, configured to display information. Figure 1A ), a touch-sensitive secondary display unit 1104 configured to receive contacts, gestures, and other user inputs on the touch-sensitive secondary display, and a processing unit 1110. The system 1100 optionally includes a physical keyboard unit 1105 configured to receive keyboard input, and in some embodiments, the optional physical keyboard unit includes the touch-sensitive secondary display unit 1104 in a second housing that is different from the first housing that includes the main display unit 1102. In some embodiments, the first housing and the second housing are rotatably connected (e.g., for embodiments in which the computing system 1100 is a laptop computer), while in other embodiments, the first housing and the second housing are not connected and can be part of a separate computing device forming the system 1100 (e.g., the first housing is part of a laptop computer and the second housing is part of a separate device such as a wearable computing device (like a smart watch)). The processing unit 1110 is coupled to the main display unit 1102, the touch-sensitive secondary display unit 1104, and the optional physical keyboard unit 1105. In some embodiments, the processing unit 1110 includes a display unit (e.g., display unit 1112), a detection unit 1114 (e.g., detection unit 1114), and an enable indication activation unit (e.g., enable indication activation unit 1116).

[0262] The processing unit is configured to: display (e.g., using display unit 1112 in combination with main display unit 1102) a first user interface for an application on a primary display; display (e.g., using display unit 1112 in combination with touch-sensitive secondary display unit 1104) a second user interface on a touch-sensitive secondary display, the second user interface comprising: (i) a plurality of application-specific enable representations for controlling functions available within the application, and (ii) at least one system-level enable representation for controlling system-level functions; detect (e.g., using detection unit 1114 in combination with touch-sensitive secondary display unit 1104) a first input on a first application-specific enable representation among the plurality of application-specific enable representations via the touch-sensitive secondary display; and while the first input remains in contact with the first application-specific enable representation: detect (e.g., using detection unit 1114 in combination with touch-sensitive secondary display unit 1104) a second input that is not on the first application-specific enable representation via the touch-sensitive secondary display, and in response to detecting the second input, activate the first application-specific enable representation (e.g., using enable representation activation unit 1116).

[0263] According to some embodiments of computing system 1100, the processing unit is further configured to: in response to detecting a first input and while the first input maintains contact with the first application-specific enabling representation: continue to display (e.g., using display unit 1112 in combination with main display unit 1102) a first user interface for the application on the primary display; and display (e.g., using display unit 1112 in combination with main display unit 1102) an enlarged representation of the first application-specific enabling representation on the primary display, wherein the enlarged representation is displayed at a second display size that is larger than the first display size.

[0264] According to some embodiments of computing system 1100, displaying an enlarged representation of at least one application-specific enabling representation includes displaying an enlarged representation of at least one application-specific enabling representation within an enlarged representation of the second user interface.

[0265] According to some embodiments of computing system 1100, displaying an enlarged representation of the second user interface includes displaying a focus indicator within the enlarged representation of the second user interface, the focus indicator being positioned based at least in part on a location where input on the touch-sensitive secondary display contacts the touch-sensitive secondary display.

[0266] In accordance with some embodiments of computing system 1100, at least one application-specific enable representation is associated with a slider, and activating the at least one application-specific enable representation includes updating an enlarged representation of the at least one application-specific enable representation based on (or, more generally, in order to allow) movement of the at least one application-specific enable representation along the slider.

[0267] According to some embodiments of the computing system 1100, the processing unit is further configured to: detect (e.g., using detection unit 1114 in combination with touch-sensitive secondary display unit 1104) a predefined gesture at a touch-sensitive secondary display, the predefined gesture manipulating a zoom level, the zoom level being used to display an enlarged representation of at least one application-specific enable representation at the primary display; and in response to detecting the predefined gesture, update the enlarged representation at the primary display (e.g., using display unit 1112 in combination with primary display unit 1102) when the zoom level is manipulated using the predefined gesture.

[0268] According to some embodiments of computing system 1100, based on determining that an input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time, an enlarged representation of at least one application-specific affordance is displayed on the primary display.

[0269] According to some embodiments of computing system 1100, based on determining that the touch-sensitive secondary display is operating in an accessible mode, an enlarged representation of at least one application-specific affordance is displayed.

[0270] According to some embodiments of computing system 1100, each of multiple application-specific affordances and at least one system-level affordance can be selected via one or more inputs at a touch-sensitive secondary display.

[0271] According to some embodiments of the computing system 1100, the second housing also at least partially includes a physical keyboard.

[0272] According to some embodiments of computing system 1100, the second housing is not connected to the first housing.

[0273] According to some embodiments of the computing system 1100, the second housing is part of a wearable computing device.

[0274] For the purpose of explanation, the foregoing description is described with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the disclosed precise form. Based on the above teachings, many modifications and variations are possible. The embodiments are selected and described in order to best illustrate the principles of the invention and its practical application, so that other persons skilled in the art can best use the invention and the various described embodiments with various modifications suitable for the specific purposes contemplated.

Claims

1. A method for interacting with a computing system, comprising: At the computing system, the computing system includes one or more processors, a first housing including a primary display, a memory, and a second housing (602), the second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display: Displaying a first user interface for an application in a first area of ​​the main display (610); displaying a second user interface on the touch-sensitive secondary display, the second user interface being different from the first user interface and comprising a plurality of application-specific affordances that control functionality available within the application, wherein each of the plurality of application-specific affordances is displayed at a first display size (612), and wherein each of the plurality of application-specific affordances is associated with a different functionality for controlling the application; detecting, via the touch-sensitive secondary display, input contact with at least one application-specific affordance of the plurality of application-specific affordances (616); and In response to detecting the input, and while the input remains in contact with the touch-sensitive secondary display (618): continuing to display on the main display at least a portion of the first user interface for the application in the first area of ​​the main display; An enlarged representation of the at least one application-specific enabling representation is displayed in a second area of ​​the main display, wherein the first area and the second area are not the same, and further wherein the enlarged representation of the at least one application-specific enabling representation is displayed to cover the first user interface and have a second display size that is larger than the first display size.

2. The method of claim 1 , wherein displaying the enlarged representation of the at least one application-specific affordance comprises: The enlarged representation of the at least one application-specific affordance is displayed on the primary display within the enlarged representation of the second user interface (620).

3. The method of claim 2, wherein displaying the enlarged representation of the at least one application-specific affordance comprises: A focus indicator is displayed on the primary display within the enlarged representation of the second user interface, the focus indicator being positioned based at least in part on a location where the input on the touch-sensitive secondary display contacts the touch-sensitive secondary display (622).

4. The method of claim 3, wherein the focus indicator comprises a representation of a countdown timer, and the method comprises: Based on determining that the input remains in contact with the at least one application-specific affordance for more than a predetermined amount of time, updating a representation of the countdown timer to indicate that the countdown timer is active; and Based on determining that the countdown timer has expired, activating the at least one application-specific affordance (624).

5. The method according to claim 4, wherein: The at least one application-specific affordance is associated with the slider, and Activating the at least one application-specific affordance includes updating the enlarged representation of the at least one application-specific affordance based on movement of the at least one application-specific affordance along the slider (626).

6. The method of claim 2, wherein the touch-sensitive secondary display comprises a first area having the at least one application-specific affordance and a second area having other affordances of the plurality of application-specific affordances that are different from the at least one application-specific affordance, and the method further comprises: detecting a tap gesture at the second area of ​​the touch-sensitive secondary display while the input remains in contact with the at least one application-specific affordance at the first area of ​​the touch-sensitive secondary display; as well as In response to detecting the tap gesture, the at least one application-specific affordance is activated (632).

7. The method according to any one of the preceding claims, further comprising: detecting, at the touch-sensitive secondary display, a predefined gesture, the predefined gesture manipulating a zoom level for displaying the enlarged representation of the at least one application-specific affordance at the primary display; as well as In response to detecting the predefined gesture, updating the magnified representation at the primary display while the zoom level is manipulated using the predefined gesture (634).

8. A method according to any one of claims 1-6, wherein the enlarged representation of the at least one application-specific enabling representation is displayed on the primary display (628) based on determining that the input has maintained continuous contact with the touch-sensitive secondary display for more than a predetermined amount of time.

9. The method of any one of claims 1-6, wherein the enlarged representation of the at least one application-specific affordance is displayed based on determining that the touch-sensitive secondary display is operating in an accessible mode (630).

10. The method of any of claims 1-6, wherein each of the plurality of application-specific affordances is selectable via one or more inputs at the touch-sensitive secondary display (614).

11. The method according to any one of claims 1-6, wherein the second housing is not connected to the first housing (606).

12. The method of claim 11, wherein the second housing is part of a wearable computing device (608).

13. The method of any one of claims 1-6, wherein displaying the first user interface comprises displaying content of the application, and wherein the second user interface does not include the content of the application.

14. A non-transitory computer-readable storage medium storing executable instructions that, when executed by a computing system having one or more processors, a first housing including a primary display, and a second housing at least partially containing a touch-sensitive secondary display different from the primary display, causes the computing system to perform the method of any one of claims 1-13.

15. A computing system comprising: one or more processors; a first housing, the first housing comprising a primary display; a second housing at least partially containing a touch-sensitive secondary display distinct from the primary display; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1-13.

Images