DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR UPDATING A SESSION AREA - Patent application
The integration of tactile feedback and efficient user interface methods on touch-sensitive surfaces addresses inefficiencies in existing technologies, enhancing interaction speed and power conservation in electronic devices.
Patent Information
- Application Number
- JP2022151726
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-09-02
- Filing Date
- 2022-09-22
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2042-09-22
AI Technical Summary
Existing methods for interacting with user interface objects on touch-sensitive surfaces in electronic devices are cumbersome, inefficient, and energy-intensive, particularly in battery-operated devices, leading to increased cognitive burden and power consumption.
Implementing methods and interfaces that provide faster and more efficient access to application controls and state information through tactile feedback, reducing the number of user inputs required and enhancing power conservation by using touch-sensitive surfaces with integrated sensors and processors to manage user interactions.
Enhances user interface efficiency, reduces power consumption, and extends battery life by minimizing unnecessary inputs and providing real-time state updates within a persistent session area, improving user satisfaction and device performance.
Smart Images

Figure 0007591539000001 
Figure 0007591539000002 
Figure 0007591539000003
Abstract
Description
Related Applications
[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 403,681, filed September 2, 2022, U.S. Provisional Patent Application No. 63 / 348,437, filed June 2, 2022, and U.S. Provisional Patent Application No. 63 / 339,406, filed May 6, 2022, which are incorporated by reference in their entireties herein. [Technical field]
[0002] It relates generally to electronic devices with touch-sensitive surfaces, including, but not limited to, electronic devices with touch-sensitive surfaces that include a display area having one or more sensors and a session area. [Background technology]
[0003] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch-sensitive surfaces include touchpads and touchscreen displays. Such surfaces are widely used to manipulate user interfaces on the displays and objects therein. Exemplary user interface objects include digital images, video, text, icons, and control elements such as buttons and other graphics.
[0004] Exemplary operations include adjusting the position and / or size of one or more user interface objects, activating a button or opening a file / application represented by a user interface object, as well as associating metadata with one or more user interface objects or otherwise manipulating the user interface. Exemplary user interface objects include digital images, video, text, icons, and control elements such as buttons and other graphics.
[0005] However, methods of performing these operations are cumbersome and inefficient. For example, using a series of mouse-based inputs to select one or more user interface objects and perform one or more actions on the selected user interface objects is tedious and creates a significant cognitive burden for the user. In addition, the methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices. Summary of the Invention
[0006] Thus, there is a need for electronic devices with faster, more efficient methods and interfaces for viewing status information and accessing controls for controlling applications. Such methods and interfaces, optionally, complement or replace conventional methods for viewing status information and accessing controls for controlling applications. Such methods and interfaces reduce the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and extend the time between battery charges.
[0007] The above deficiencies and other problems associated with user interfaces for electronic devices having touch-sensitive surfaces are reduced or eliminated by the devices of the present disclosure. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device such as a watch). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or sets of instructions stored in the memory for performing a number of functions. In some embodiments, a user interacts with the GUI primarily through contacts and gestures of a stylus and / or finger on the touch-sensitive surface. In some embodiments, the functionality optionally includes image editing, drawing, presenting, word processing, spreadsheet creation, game playing, making phone calls, video conferencing, emailing, instant messaging, training support, digital photography, digital videography, web browsing, digital music playback, note taking, and / or digital video playback, and executable instructions to perform those functionality are optionally contained on a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.
[0008] According to some embodiments, a method includes, in a computer system in communication with one or more sensors and a display generation component having a display area, displaying in a first display region of the display area a first user interface object including status information provided by the respective software regarding a status of the computer system, the one or more sensors being disposed within one or more sensor regions surrounded by the display area, the display generation component being unable to display content within the one or more sensor regions, the first display region encompassing the one or more sensor regions. The method includes displaying in a second display region of the display generation component different from the first display region a user interface of an application different from the respective software, the second display region at least partially surrounding the first display region. The method further includes, when the status of the computer system changes, displaying in the first user interface object updated status information provided by the respective software regarding the changed status of the computer system, the updated status information including visual elements selected to avoid overlapping positions of the one or more sensors in the first display region.
[0009] According to some embodiments, a method includes, in a computer system in communication with a display generating component having a display area including a status area, displaying a first user interface in a display area outside the status area including a user interface of an individual application running on the computer system, and updating the user interface of the individual application according to one or more changes in a state of the individual application. The method includes detecting a user input corresponding to a request to terminate the first user interface, and ceasing to display the first user interface in the display area outside the status area including the user interface of the individual application in response to detecting the user input corresponding to the request to terminate the first user interface. The method includes displaying an indication of a current state of the individual application in the status area while the state of the individual application continues to change, and displaying a second user interface in the display area outside the status area associated with an individual software different from the individual application. The method further includes detecting a user input to navigate from a second user interface associated with a particular piece of software different from the particular application to a third user interface, and in response to detecting the user input to navigate from the second user interface, displaying the third user interface in a display area outside the status area while continuing to display an indication of a current state of the particular application in the status area.
[0010] According to some embodiments, a method includes, in a computer system in communication with a display generation component having a display area, displaying a first user interface of a first application in an application user interface region, the display area including a status region and an application user interface region separate from the status region, the application user interface region at least partially surrounding the status region. The method includes detecting a user input corresponding to a request to display a second user interface in the application user interface region, and in response to detecting the user input corresponding to the request to display the second user interface in the application user interface region, ceasing to display the first user interface of the first application in the application user interface region and displaying the second user interface in the application user interface region in accordance with a determination that the second user interface is associated with a separate software different from the first application. The method further includes, in accordance with a determination that the status area is associated with an active session of a second application, different from the first application, displaying a first indication of a current state of the first application in a first portion of the status area, the first indication including updating the displayed first indication as the state of the first application changes, and displaying a second indication of a current state of the second application in a second portion of the status area, different from the first portion, the second indication including updating the displayed second indication as the state of the second application changes, wherein the first portion of the status area and the second portion of the status area are displayed simultaneously with a second user interface.The method includes, pursuant to a determination that the status area is not associated with an active session of a second application different from the first application, displaying a first indication of a current state of the first application in the status area, without displaying an indication of the current state of the second application in the status area, where the status area is displayed simultaneously with the second user interface.
[0011] According to some embodiments, a method includes, in a computer system in communication with a display generation component having a display area that at least partially contains a status area, detecting a user input corresponding to the status area, and in response to detecting the user input corresponding to the status area, performing an operation associated with the respective application in accordance with a determination that the status area is associated with an active session of the respective application. The method includes providing feedback associated with the user input without displaying information about the respective application in the status area and without performing an operation associated with the respective application in accordance with a determination that the status area is not associated with an active session of the respective application.
[0012] According to some embodiments, a method includes, in a computer system in communication with a display generation component having a display area including a status area, displaying a first user interface in a display area outside the status area including a separate user interface element for initiating enrollment of a biometric characteristic of a user. The method includes detecting user input directed at the separate user interface element to initiate enrollment of a biometric characteristic of the user. The method further includes performing a biometric enrollment process including, in response to detecting the user input, displaying a biometric enrollment user interface in the status area while maintaining display of the first user interface in the display area outside the status area, and updating the biometric enrollment user interface in the status area during the biometric enrollment process while capturing biometric information related to the biometric characteristic of the user to indicate a current status of the biometric enrollment process.
[0013] According to some embodiments, the method includes detecting a user input that invokes a virtual assistant of the computer system in a computer system in communication with one or more sensors and a display generation component having a display area, the one or more sensors being disposed within one or more sensor regions surrounded by the display area, the display generation component being unable to display content within the one or more sensor regions, and a first display region of the display area encompassing the one or more sensor regions. The method includes displaying a visual indication that the virtual assistant is active in the first display region in response to detecting a user input that invokes the virtual assistant. The method further includes receiving a voice command directed to the virtual assistant, and in response to receiving the voice command directed to the virtual assistant, performing an operation in the computer system in response to the voice command and displaying a result of the operation performed in response to the voice command, and updating the first display region, the updating including displaying a visual element selected to avoid overlapping the positions of one or more sensors in the first display region.
[0014] According to some embodiments, a method includes, in a computer system in communication with a display generation component having a display area including a system user interface region, displaying a first user interface in a display area outside the system user interface region. The method includes, in accordance with a determination that the system user interface region is displayed in a first display state, displaying the system user interface region with a first visual edge treatment and displaying an interior background of the system user interface region with a first appearance. The method further includes, in accordance with a determination that the system user interface region is displayed in a second display state different from the first display state, displaying the system user interface region with a second visual edge treatment different from the first visual edge treatment and displaying the interior background of the system user interface region with the first appearance.
[0015] According to some embodiments, a method includes, in a computer system in communication with a display generation component having a display area including a status area, displaying first information in the status area including state information related to a first software. The status area is associated with an active session of the first software. The method includes detecting an occurrence of a first event corresponding to a transition to displaying second information different from the first information while displaying the first information in the status area. The method includes displaying a transition including ceasing display of the first information in the status area in response to detecting the occurrence of the first event, displaying second information in the status area, the second information including state information related to a second software different from the first software, the status area being associated with an active session of the second software, and reducing a size of the status area during the transition from displaying the first information in the status area to displaying the second information in the status area and expanding a size of the status area after reducing the size of the status area.
[0016] According to some embodiments, an electronic device includes a display, a touch-sensitive surface, optionally one or more sensors that detect intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, one or more processors, and a memory having stored thereon one or more programs, the one or more programs configured to be executed by the one or more processors, the one or more programs including instructions to perform or cause to be performed any of the operations of the methods described herein. According to some embodiments, a computer-readable storage medium has stored therein instructions that, when executed by an electronic device having a display, a touch-sensitive surface, optionally one or more sensors that detect intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators, cause the device to perform or cause to be performed any of the operations of the methods described herein. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, a memory, and one or more processors executing one or more programs stored in the memory includes one or more of the elements displayed in any of the methods described herein, and the elements are updated in response to input as described in any of the methods described herein. According to some embodiments, the electronic device includes a display, a touch-sensitive surface, optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, and means for performing or causing to be performed any of the operations of the methods described herein. According to some embodiments, an information processing apparatus for use in an electronic device having a display, a touch-sensitive surface, optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators includes means for performing or causing to be performed any of the operations of the methods described herein.
[0017] In this manner, electronic devices having a display, a touch-sensitive surface, optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, optionally one or more device orientation sensors, and optionally an audio system provide improved methods and interfaces for providing status updates, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may complement or replace conventional methods of providing status updates. [Brief description of the drawings]
[0018] For a better understanding of the various described embodiments, reference should be made to the following Detailed Description in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout:
[0019] [Figure 1A] 1 is a block diagram illustrating a portable multifunction device having a touch-sensitive display in accordance with some embodiments.
[0020] [Figure 1B] FIG. 2 is a block diagram illustrating example components for event processing according to some embodiments.
[0021] [Diagram 2] 1 illustrates a portable multifunction device with a touch screen in accordance with some embodiments.
[0022] [Diagram 3] 1 is a block diagram of an exemplary multifunction device having a display and a touch-sensitive surface in accordance with some embodiments.
[0023] [Figure 4A] 1 illustrates an exemplary user interface for a menu of applications on a portable multifunction device in accordance with some embodiments.
[0024] [Figure 4B] 1 illustrates an exemplary user interface for a multifunction device having a touch-sensitive surface separate from a display in accordance with some embodiments.
[0025] [Figure 5A] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5B] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5C] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5D] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5E] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5F] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5G] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5H] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5I] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5J] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5K] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5L] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5M] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5N] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5O] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5P] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5Q] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5R] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5S] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5T] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5U] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5V] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5W] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5X]1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5Y] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5Z] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AA] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AB] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AC] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AD] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AE] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AF] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5A-G] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AH] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AI] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AJ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AK] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AL] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AM] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AN] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AO] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AP] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AQ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AR1] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AR2] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AR3] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AS] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AT] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AU] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AV] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AW] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AX] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AY] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5AZ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BA1] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BA2] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BA3] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BB] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BC] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BD] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BE1] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BE2]1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BF] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BG] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BH] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BI] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BJ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BK] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BL] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BM1] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BM2] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BM3] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BM4] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BN] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BO] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BP] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BQ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BR] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS1] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS2] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS3] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS4] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS5] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS6] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BS7] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BT] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BU] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BV] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BW] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BX] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BY] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5BZ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CA] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CB] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CC] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5C-D] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CE] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CF] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CG] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CH]1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CI] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CJ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CK] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CL] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CM] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CN] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CO] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CP] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CQ] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CR] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CS] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Fig. 5CT] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CU] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CV] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CW] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CX] 1 illustrates an example user interface for updating state information in a session area according to some embodiments. [Figure 5CY] 1 illustrates an example user interface for updating state information in a session area according to some embodiments.
[0026] [Figure 6A] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6B] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6C] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6D] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6E1] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6E2] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6F] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6G] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6G1]1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6G2] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6H] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6I] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6J] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6K] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6L] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments. [Figure 6M] 1 illustrates an exemplary user interface for biometric enrollment according to some embodiments.
[0027] [Figure 7A] 1 illustrates an example user interface for updating a session area to use a stored payment method according to some embodiments. [Figure 7B1] 1 illustrates an example user interface for updating a session area to use a stored payment method according to some embodiments. [Figure 7B2] 1 illustrates an example user interface for updating a session area to use a stored payment method according to some embodiments. [Figure 7C] 1 illustrates an example user interface for updating a session area to use a stored payment method according to some embodiments. [Figure 7D]1 illustrates an example user interface for updating a session area to use a stored payment method according to some embodiments. [Figure 7E] 1 illustrates an example user interface for updating a session area to use a stored payment method according to some embodiments.
[0028] [Figure 8A] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments. [Figure 8B] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments. [Figure 8C] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments. [Figure 8D] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments. [Figure 8E] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments. [Figure 8F] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments. [Figure 8G] 1 illustrates an example user interface for updating a session area for a virtual assistant according to some embodiments.
[0029] [Figure 9A] FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments. [Figure 9B] FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments. [Figure 9C]FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments. [Figure 9D] FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments. [Figure 9E] FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments. [Figure 9F] FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments. [Figure 9G] FIG. 1 is a flow diagram of a process for updating state information in a region of a display area according to some embodiments.
[0030] [Figure 10A] FIG. 1 is a flow diagram of a process for continuing to display updated status information in a status area while switching between different user interfaces, according to some embodiments. [Figure 10B] FIG. 1 is a flow diagram of a process for continuing to display updated status information in a status area while switching between different user interfaces, according to some embodiments. [Figure 10C] FIG. 1 is a flow diagram of a process for continuing to display updated status information in a status area while switching between different user interfaces, according to some embodiments. [Figure 10D] FIG. 1 is a flow diagram of a process for continuing to display updated status information in a status area while switching between different user interfaces, according to some embodiments.
[0031] [Figure 11A] FIG. 2 is a flow diagram of a process for displaying updated status information for at least one application in a status area of a display according to some embodiments. [Figure 11B]FIG. 2 is a flow diagram of a process for displaying updated status information for at least one application in a status area of a display according to some embodiments. [Figure 11C] FIG. 2 is a flow diagram of a process for displaying updated status information for at least one application in a status area of a display according to some embodiments. [Figure 11D] FIG. 2 is a flow diagram of a process for displaying updated status information for at least one application in a status area of a display according to some embodiments.
[0032] [Figure 12A] FIG. 1 is a flow diagram of a process for interacting with a state area to perform an action according to some embodiments. [Figure 12B] FIG. 1 is a flow diagram of a process for interacting with a state area to perform an action according to some embodiments. [Figure 12C] FIG. 1 is a flow diagram of a process for interacting with a state area to perform an action according to some embodiments. [Figure 12D] FIG. 1 is a flow diagram of a process for interacting with a state area to perform an action according to some embodiments. [Figure 12E] FIG. 1 is a flow diagram of a process for interacting with a state area to perform an action according to some embodiments.
[0033] [Figure 13A] FIG. 13 is a flow diagram of a process for displaying biometric enrollment information in a status area according to some embodiments. [Figure 13B] FIG. 13 is a flow diagram of a process for displaying biometric enrollment information in a status area according to some embodiments. [Figure 13C] FIG. 13 is a flow diagram of a process for displaying biometric enrollment information in a status area according to some embodiments. [Figure 13D]FIG. 13 is a flow diagram of a process for displaying biometric enrollment information in a status area according to some embodiments. [Figure 13E] FIG. 13 is a flow diagram of a process for displaying biometric enrollment information in a status area according to some embodiments.
[0034] [Figure 14A] FIG. 1 is a flow diagram of a process for displaying updates for a virtual assistant in a sensor area according to some embodiments. [Figure 14B] FIG. 1 is a flow diagram of a process for displaying updates for a virtual assistant in a sensor area according to some embodiments. [Figure 14C] FIG. 1 is a flow diagram of a process for displaying updates for a virtual assistant in a sensor area according to some embodiments.
[0035] [Figure 15A] FIG. 1 is a flow diagram of a process for changing the appearance of a session area depending on the current state of a device, according to some embodiments. [Figure 15B] FIG. 1 is a flow diagram of a process for changing the appearance of a session area depending on the current state of a device, according to some embodiments. [Figure 15C] FIG. 1 is a flow diagram of a process for changing the appearance of a session area depending on the current state of a device, according to some embodiments. [Figure 15D] FIG. 1 is a flow diagram of a process for changing the appearance of a session area depending on the current state of a device, according to some embodiments. [Figure 15E] FIG. 1 is a flow diagram of a process for changing the appearance of a session area depending on the current state of a device, according to some embodiments.
[0036] [Figure 16A] FIG. 1 is a flow diagram of a process for switching the display of different types of information in the status area according to some embodiments. [Figure 16B] FIG. 1 is a flow diagram of a process for switching the display of different types of information in the status area according to some embodiments. [Figure 16C] FIG. 1 is a flow diagram of a process for switching the display of different types of information in the status area according to some embodiments. [Figure 16D] FIG. 1 is a flow diagram of a process for switching the display of different types of information in the status area according to some embodiments. [Figure 16E] FIG. 1 is a flow diagram of a process for switching the display of different types of information in the status area according to some embodiments. [Figure 16F] FIG. 1 is a flow diagram of a process for switching the display of different types of information in the status area according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0037] Many electronic devices have graphical user interfaces that allow users to navigate between application user interfaces and / or system user interfaces. Some methods for navigating between user interfaces allow for multitasking, such that individual applications continue to update in the background even after navigating away from the individual application user interface. For example, these methods may require the user to return to the individual application user interface to view the updates. In the embodiments described below, an improved method for providing status updates for multiple applications in a persistent session area is provided. This method streamlines the user's ability to view real-time status information for active sessions, thereby eliminating the need for an extra separate step of returning to the individual user interfaces of the individual applications to view status updates.
[0038] The methods, devices, and GUIs described herein use tactile feedback to improve user interface interaction in several ways: for example, they make it easier to indicate hidden thresholds and to indicate user interface components that represent selectable options.
[0039] The processes described below improve usability of the device and make the user device interface more efficient (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device) through various techniques including providing improved visual, audio, and / or haptic feedback to the user, reducing the number of inputs required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, performing an operation when a set of conditions is met without requiring further user input, improving privacy and / or security, reducing the amount of display area required to display notifications and / or status information and thus increasing the amount of display area available to other applications for displaying information, and / or additional techniques. These techniques also reduce power usage and improve the battery life of the device by allowing the user to use the device more quickly and efficiently. Saving battery power, and therefore weight, improves the ergonomics of the device.
[0040] In the following, FIGS. 1A, 1B, 2, and 3 provide a description of an exemplary device. FIGS. 4A-4B and 5A-5CY show exemplary user interfaces for updating status information in a session area. FIGS. 6A-6M show exemplary user interfaces for registering for biometric authentication. FIGS. 7A-7E show exemplary user interfaces for updating a session area to use a stored payment method, according to some embodiments. FIGS. 8A-8G show exemplary user interfaces for updating a session area for a virtual assistant, according to some embodiments. FIGS. 9A-9G are flow diagrams of a process for updating status information in a region of a display area, according to some embodiments. FIGS. 10A-10D are flow diagrams of a process for continuing to display updated status information in a status area while switching between different user interfaces, according to some embodiments. FIGS. 11A-11D are flow diagrams of a process for displaying updated status information for at least one application in a status area of a display, according to some embodiments. FIGS. 12A-12E are flow diagrams of a process for interacting with a status area to perform an operation, according to some embodiments. 13A-13E are flow diagrams of a process for displaying biometric enrollment information in the status area, according to some embodiments. FIGs. 14A-14C are flow diagrams of a process for displaying updates for a virtual assistant in the sensor area, according to some embodiments. FIGs. 15A-15E are flow diagrams of a process for changing the appearance of a session area depending on the current state of the device, according to some embodiments. FIGs. 16A-16F are flow diagrams of a process for switching the display of different types of information in the status area, according to some embodiments. The user interfaces of FIGs. 5A-5CY, 6A-6M, 7A-7E, and 8A-8G are used to explain the processes of FIGs. 9A-9G, 10A-10D, 11A-11D, 12A-12E, 13A-13E, 14A-14C, 15A-15E, and 16A-16F.
[0041] Exemplary Devices Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments being described. However, it will be apparent to those skilled in the art that the various embodiments described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
[0042] In this specification, terms such as first, second, etc. are used in some examples to describe various elements, but it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first contact can be referred to as a second contact, and similarly, a second contact can be referred to as a first contact, without departing from the scope of the various embodiments described. Although a first contact and a second contact are both contacts, they are not the same contact unless the context clearly indicates otherwise.
[0043] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. In the description of the various embodiments described and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that as used herein, the term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0044] As used herein, the term "if" is optionally interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.
[0045] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers having a touch-sensitive surface (e.g., a touchscreen display and / or a touchpad), are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or a touchpad).
[0046] In the following discussion, electronic devices are described that include a display and a touch-sensitive surface, however, it should be understood that the electronic device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse, and / or a joystick.
[0047] The device typically supports a variety of applications, such as one or more of note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disc authoring applications, spreadsheet applications, gaming applications, telephony applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.
[0048] The various applications running on the device optionally use at least one common physical user-interface device, such as a touch-sensitive surface. One or more features of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or changed for each application and / or within individual applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) optionally supports a variety of applications with user interfaces that are intuitive and transparent to the user.
[0049] Attention is now directed to embodiments of portable devices with touch-sensitive displays. FIG. 1A is a block diagram illustrating a portable multifunction device 100 having a touch-sensitive display system 112, according to some embodiments. Touch-sensitive display system 112 may be conveniently referred to as a "touch screen" or simply a touch-sensitive display. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), memory controller 122, one or more processing units (CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more light sensors 164. Device 100 optionally includes one or more intensity sensors 165 that detect the intensity of a contact on device 100 (e.g., a touch-sensitive surface, such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 that generate tactile output on device 100 (e.g., generate tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touchpad 355 of device 300). These components optionally communicate over one or more communication buses or signal lines 103.
[0050] As used herein and in the claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, that will be detected by a user with the user's sense of touch. For example, in a situation where a device or a component of a device is in contact with a touch-sensitive surface of a user (e.g., the fingers, palm, or other part of a user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical property of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "down-click" or "up-click" of a physical actuator button. In some cases, a user feels a tactile sensation such as a "down-click" or "up-click" even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's action. As another example, movement of a touch-sensitive surface is optionally interpreted or sensed by a user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. Such user interpretation of touch depends on the user's personal sensory perception, but there are many sensory perceptions of touch that are common to a majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an "up click," a "down click," "roughness"), unless otherwise noted, the generated tactile output corresponds to a physical displacement of the device, or a component of the device, that produces the described sensory perception for a typical (or average) user.Using tactile output to provide haptic feedback to the user improves usability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), as well as reducing power usage and improving the battery life of the device by enabling the user to use the device more quickly and efficiently.
[0051] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the movement waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.
[0052] When tactile outputs having different tactile output patterns are generated by the device (e.g., via one or more tactile output generators that move a movable mass to generate the tactile output), the tactile outputs can cause different tactile sensations when a user holds or touches the device. Although the user's sensations are based on the user's perception of the tactile output, most users are able to discern changes in waveform, frequency, and amplitude of the tactile output generated by the device. Thus, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different actions have been performed. Thus, in some situations, tactile output having a tactile output pattern designed, selected, and / or developed to simulate the properties (e.g., size, material, weight, stiffness, smoothness, etc.), behaviors (e.g., vibration, displacement, acceleration, rotation, expansion, etc.), and / or interactions (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment having virtual boundaries and virtual objects, an actual physical environment having physical boundaries and physical objects, and / or any combination of the above) provides a user with useful feedback that reduces input errors and increases the efficiency of a user's operation of the device. Additionally, the tactile output is optionally generated to correspond to feedback unrelated to simulated physical properties, such as input thresholds or object selection. In some situations, such tactile output provides a user with useful feedback that reduces input errors and increases the efficiency of a user's operation of the device.
[0053] In some embodiments, the tactile output with a suitable tactile output pattern serves as a cue to the appearance of an event of interest behind the scene in the user interface or device. Examples of events of interest include activation of an affordance (e.g., a real button, a virtual button, or a toggle switch) provided on the device or in the user interface, success or failure of a requested action, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detection or recognition of a type of input or gesture, etc. In some embodiments, the tactile output is provided to act as a warning or alert to an impending event or outcome that will occur unless a redirection or interrupt input is detected in a timely manner. Tactile output is also used in other contexts to improve the user experience, improve the accessibility of the device to users with visual or motor impairments or other accessibility needs, and / or improve the efficiency and functionality of the user interface and / or device. The tactile output, optionally accompanied by audio output and / or visible user interface changes, further enhances the user's experience when interacting with the user interface and / or device, facilitates better communication of information about the state of the user interface and / or device, reduces input errors, and increases the efficiency of the user's operation of the device.
[0054] It is understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of its components. The various components shown in FIG. 1A are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application specific integrated circuits.
[0055] Memory 102 optionally includes high-speed random access memory, and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100, such as CPU(s) 120 and peripherals interface 118, is optionally controlled by memory controller 122.
[0056] A peripheral interface 118 may be used to couple input and output peripherals of the device with the CPU(s) 120 and memory 102. The one or more processors 120 operate or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions and process data for the device 100.
[0057] In some embodiments, peripherals interface 118, CPU(s) 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.
[0058] The radio frequency (RF) circuitry 108 transmits and receives RF signals, also referred to as electromagnetic signals. The RF circuitry 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuitry 108 optionally includes well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, etc. The RF circuitry 108 optionally communicates with networks, such as the Internet, also referred to as the World Wide Web (WWW), an intranet, and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs), and with other devices, by wireless communication. The wireless communications options include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPADA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), and code division multiple access (CDMA).access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet message access protocol (IMAP) and / or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE)), instant messaging and presence services (e.g., IEEE 802.11b), instant messaging and presence services (e.g., IEEE 802.11c), instant messaging and presence services (e.g., IEEE 802.11g), instant messaging and presence services (e.g., IEEE 802.11n ... The present invention may use any of a number of communications standards, protocols, and technologies, including, but not limited to, Intermediate System (IS) Intermediate Message Service (IMPS), and / or Short Message Service (SMS), or any other suitable communications protocol, including communications protocols not yet developed as of the filing date of this document.
[0059] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripherals interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data, and transmits the audio data to the peripherals interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripherals interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., 212, FIG. 2). The headset jack provides an interface between audio circuitry 110 and a detachable audio input / output peripheral such as output-only headphones or a headset with both an output (e.g., one- or binaural headphones) and an input (e.g., a microphone).
[0060] I / O subsystem 106 couples input / output peripherals on device 100, such as touch-sensitive display system 112 and other input or control devices 116, with peripheral interface 118. I / O subsystem 106 optionally includes one or more input controllers 160 for display controller 156, light sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and other input or control devices. One or more input controllers 160 receive electrical signals from and send electrical signals to other input or control devices 116. Other input or control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, input controller(s) 160 are optionally coupled to any (or none) of a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. The one or more buttons (e.g., 208 in FIG. 2) optionally include up / down buttons (e.g., a single button that rocks in opposite directions, or separate up and down buttons) for volume control of speaker 111 and / or microphone 113. The one or more buttons optionally include push buttons (e.g., 206, FIG. 2).
[0061] Touch-sensitive display system 112 provides an input and output interface between the device and a user. Display controller 156 receives electrical signals from and / or transmits electrical signals to touch-sensitive display system 112. Touch-sensitive display system 112 displays visual output to the user. This visual output optionally includes graphics, text, icons, animation, and any combination thereof (collectively "graphics"). In some embodiments, some or all of the visual output corresponds to a user interface object. As used herein, the term "affordance" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object that is configured to respond to input directed toward the graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, a button, a slider, an icon, a selectable menu item, a switch, a hyperlink, or other user interface control.
[0062] Touch-sensitive display system 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic and / or tactile contact. Touch-sensitive display system 112 and display controller 156 (along with any associated modules and / or instruction sets in memory 102) detect contacts (and any movement or breaking of contact) on touch-sensitive display system 112 and translate the detected contacts into interactions with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touch-sensitive display system 112. In some embodiments, the point of contact between touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.
[0063] Touch-sensitive display system 112 optionally uses liquid crystal display (LCD), light emitting polymer display (LPD), or light emitting diode (LED) technology, although other display technologies are used in other embodiments. Touch-sensitive display system 112 and display controller 156 optionally detect contact and any movement or disruption thereof using any of a number of now known or later developed touch sensing technologies, including, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch-sensitive display system 112. In some embodiments, projected mutual capacitance sensing technology is used, such as that found in the iPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, California.
[0064] Touch-sensitive display system 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touchscreen video resolution is greater than 400 dpi (e.g., 500 dpi, 800 dpi, or higher). A user optionally contacts touch-sensitive display system 112 using any suitable object or appendage, such as a stylus, finger, or the like. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may be less precise than stylus-based inputs due to the larger contact area of a finger on a touchscreen than that of a stylus. In some embodiments, the device translates the rough finger input into precise pointer / cursor positions or commands to perform the actions desired by the user.
[0065] In some embodiments, in addition to a touchscreen, device 100 optionally includes a touchpad for activating or deactivating certain functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike a touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touch-sensitive display system 112 or an extension of the touch-sensitive surface formed by the touchscreen.
[0066] Device 100 also includes a power system 162 that provides power to the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power within a portable device.
[0067] Device 100 also optionally includes one or more light sensors 164 (e.g., as part of one or more cameras). FIG. 1A shows a light sensor coupled to a light sensor controller 158 in I / O subsystem 106. Light sensor(s) 164 optionally include a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. Light sensor(s) 164 receive light from the environment, projected through one or more lenses, and convert the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), light sensor(s) 164 optionally captures still images and / or video. In some embodiments, the light sensor is located on the back of device 100, opposite touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for still and / or video image acquisition. In some embodiments, another light sensor is placed on the front of the device so that an image of the user is captured (e.g., for a selfie, for a video conference while the user views other video conference participants on the touch screen, etc.).
[0068] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 in I / O subsystem 106. Contact intensity sensor(s) 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor(s) 165 receive contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is juxtaposed with or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite touchscreen display system 112, which is located on the front of device 100.
[0069] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I / O subsystem 106. In some embodiments, the proximity sensor turns off and disables touch-sensitive display system 112 when the multifunction device is placed near a user's ear (e.g., when the user is conducting a phone call).
[0070] Device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows a tactile output generator coupled to haptic feedback controller 161 in I / O subsystem 106. In some embodiments, tactile output generator(s) 167 include one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile output on the device). Tactile output generator(s) 167 receive tactile feedback generation instructions from haptic feedback module 133 and generate a tactile output on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is juxtaposed with or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112) and generates a tactile output, optionally by moving the touch-sensitive surface vertically (e.g., in / out of the surface of device 100) or horizontally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, the at least one tactile output generator sensor is located on the back of device 100, opposite touch-sensitive display system 112, which is located on the front of device 100.
[0071] Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows accelerometer 168 coupled to peripherals interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from the one or more accelerometers. In addition to accelerometer(s) 168, device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for obtaining information regarding the location and orientation (e.g., vertical or horizontal) of device 100.
[0072] In some embodiments, the software components stored in memory 102 include an operating system 126, a communications module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and applications (or instruction set) 136. Additionally, in some embodiments, as shown in Figures 1A and 3, memory 102 stores device / global internal state 157. Device / global internal state 157 includes one or more of: active application state, which indicates which applications, if any, are currently active; display state, which indicates what applications, views, or other information are occupying various areas of touch-sensitive display system 112; sensor state, which includes information obtained from the device's various sensors and other input or control devices 116; and position and / or location information regarding the device's position and / or orientation.
[0073] Operating system 126 (e.g., an embedded operating system such as iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks) includes various software components and / or drivers for controlling and managing overall system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.
[0074] The communications module 128 also includes various software components for facilitating communication with other devices via one or more external ports 124 and for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FIREWIRE, etc.) are adapted to couple directly to other devices or indirectly via a network (e.g., Internet, wireless LAN, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors identical to, similar to, and / or compatible with the 30-pin connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external ports are Lightning connectors identical to, similar to, and / or compatible with the Lightning connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external port is a USB Type-C connector that is identical to, similar to, and / or compatible with the USB Type-C connector used in several electronic devices from Apple Inc. of Cupertino, California.
[0075] Contact / motion module 130 optionally detects contact with touch-sensitive display system 112 (in cooperation with display controller 156), and with other touch-sensitive devices (e.g., a touchpad or physical click wheel). Contact / motion module 130 includes software components for performing various operations related to detecting contact (e.g., by a finger or stylus), such as determining if contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a proxy for the force or pressure of the contact), determining if there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining if the contact has stopped (e.g., detecting a finger up event or an interruption of the contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact, as represented by the set of contact data, optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact. These actions are optionally applied to a single contact (e.g., one finger contact or stylus contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, contact / motion module 130 and display controller 156 detect contacts on a touchpad.
[0076] Contact / motion module 130 optionally detects gesture input by a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timing, and / or strength of detected contacts). Thus, gestures are optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger down event, followed by a finger up (lift off) event at the same location (or substantially the same location) as the finger down event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger down event, followed by one or more finger drag events, followed by a finger up (lift off) event. Similarly, taps, swipes, drags, and other gestures are optionally detected by detecting a particular contact pattern with respect to the stylus with respect to the stylus.
[0077] In some embodiments, detecting a finger tap gesture depends on the length of time between detecting a finger down event and detecting a finger up event, but is not related to the strength of the finger contact between detecting a finger down event and detecting a finger up event. In some embodiments, a tap gesture is detected according to determining that the length of time between a finger down event and a finger up event is less than a predefined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the strength of the finger contact during the tap meets a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light or deep pressure intensity threshold. Thus, a finger tap gesture may satisfy certain input criteria that do not require that the characteristic intensity of the contact meet a given intensity threshold in order for the particular input criterion to be met. For clarity, a finger contact in a tap gesture generally needs to meet a nominal contact detection intensity threshold below which the contact is not detected in order to detect a finger down event. A similar analysis applies to detecting a tap gesture or other contact with a stylus. In cases where the device is capable of detecting contact of a finger or stylus hovering over the touch-sensitive surface, the nominal contact-detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.
[0078] In a similar manner, the same concepts apply to other types of gestures. For example, swipe gestures, pinch gestures, de-pinch gestures, and / or long press gestures are optionally detected based on meeting criteria that are either unrelated to the intensity of the contacts included in the gesture or do not require the contact(s) performing the gesture to reach an intensity threshold in order to be recognized. For example, swipe gestures are detected based on the amount of movement of one or more contacts. Pinch gestures are detected based on the movement of two or more contacts towards each other. De-pinch gestures are detected based on the movement of two or more contacts away from each other. Long press gestures are detected based on the duration of contact on the touch-sensitive surface with movement less than a threshold amount. Thus, a statement that a particular gesture recognition criterion does not require that the intensity of a contact(s) meet a corresponding intensity threshold in order for the particular gesture recognition criterion to be satisfied means that the particular gesture recognition criterion can be satisfied when the contact(s) in the gesture do not reach the corresponding intensity threshold, and can also be satisfied in situations where one or more of the contacts in the gesture reach or exceed the corresponding intensity threshold. In some embodiments, a tap gesture is detected based on a determination that a finger down event and a finger up event are detected within a predetermined time period, regardless of whether the contacts are above or below the respective intensity thresholds during the predetermined time period, and a swipe gesture is detected based on a determination that a movement of the contact is greater than a predetermined magnitude, even if the contacts exceed the respective intensity thresholds at the end of the movement of the contact. Even in implementations in which detection of gestures is affected by the strength of the contact performing the gesture (e.g., the device detects a long press more quickly when the intensity of the contact exceeds an intensity threshold, or the device is slow to detect a tap input when the intensity of the contact is higher), detection of those gestures does not require that the contact reach a particular intensity threshold, so long as the criteria for recognizing the gesture can be met in situations in which the contact does not reach a particular intensity threshold (e.g., even if the amount of time it takes to recognize the gesture varies).
[0079] The contact intensity threshold, duration threshold, and movement threshold are combined in various different combinations in some situations to create heuristics for distinguishing between two or more different gestures directed at the same input element or region, thereby enabling multiple different interactions with the same input element to provide a richer set of user interactions and responses. A statement that a particular set of gesture recognition criteria does not require the intensity of the contact(s) to meet the respective intensity threshold for that particular gesture recognition criterion to be satisfied does not preclude simultaneously evaluating other intensity-dependent gesture recognition criteria to identify other gestures having criteria that are satisfied when the gesture includes a contact having an intensity that exceeds the respective intensity threshold. For example, in some situations, a first gesture recognition criterion for a first gesture that does not require the intensity of the contact(s) to meet the corresponding intensity threshold for the first gesture recognition criterion to be satisfied is in competition with a second gesture recognition criterion for a second gesture that is dependent on the contact(s) reaching the corresponding intensity threshold. In such a competition, a gesture is optionally not recognized as satisfying the first gesture recognition criteria for the first gesture if the second gesture recognition criteria for the second gesture are satisfied first. For example, if the contact reaches the corresponding intensity threshold before it moves the predetermined amount of movement, a deep press gesture is detected rather than a swipe gesture. Conversely, if the contact moves the predetermined amount of movement before it reaches the corresponding intensity threshold, a swipe gesture is detected rather than a deep press gesture. Even in such a situation, the first gesture recognition criteria for the first gesture still do not require that the intensity of the contact(s) meet the corresponding intensity threshold for the first gesture recognition criteria to be satisfied, because if the contact remains below the corresponding intensity threshold until the end of the gesture (e.g., a swipe gesture with a contact that does not increase in intensity to above the corresponding intensity threshold), the gesture will have been recognized by the first gesture recognition criteria as a swipe gesture.In that way, a particular gesture recognition criterion that does not require that the intensity of the contact(s) meet a corresponding intensity threshold in order for the particular gesture recognition criterion to be satisfied may still be dependent on the intensity of the contact with respect to an intensity threshold in the sense that (A) in some circumstances it may ignore the intensity of the contact with respect to an intensity threshold (e.g., for a tap gesture) and / or (B) in some circumstances the particular gesture recognition criterion (e.g., for a long press gesture) will not function if a competing set of intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) recognizes an input as corresponding to an intensity-dependent gesture before the particular gesture recognition criterion recognizes a gesture corresponding to the input (e.g., for a long press gesture that competes with a deep press gesture for recognition).
[0080] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other display, including components for modifying the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, and animations.
[0081] In some embodiments, graphics module 132 stores data representing the graphics to be used. Each graphic is, optionally, assigned a corresponding code. Graphics module 132 receives one or more codes specifying the graphics to be displayed, such as from an application, along with coordinate data and other graphic characteristic data, as appropriate, and then generates screen image data for output to display controller 156.
[0082] The haptic feedback module 133 includes various software components that generate instructions (e.g., instructions used by the haptic feedback controller 161) to create tactile outputs at one or more locations on the device 100 using the tactile output generator(s) 167 in response to user interaction with the device 100.
[0083] Text input module 134 is optionally a component of graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).
[0084] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for use in location-based calling, to the camera 143 as photo / video metadata, and to applications that provide location-based services, such as a weather widget, a local yellow pages widget, and a maps / navigation widget).
[0085] Application 136 optionally includes the following modules (or sets of instructions), or a subset or superset thereof: ● a contacts module 137 (sometimes called an address book or contact list); ●Telephone module 138, ● Videoconferencing module 139, ● an email client module 140; ● Instant messaging (IM) module 141; ● Training support module 142, A camera module 143 for still and / or video images, ● Image management module 144, ● Browser module 147, ● Calendar module 148, A widget module 149, optionally including one or more of a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widgets 149-6; ● a widget creation module 150 for creating user-created widgets 149-6; ● Search module 151, ● a video and music player module 152, optionally consisting of a video player module and a music player module; ● Memo module 153, Map module 154, and / or ●Online video module 155.
[0086] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice duplication.
[0087] Contacts module 137, along with touch-sensitive display system 112, display controller 156, contacts module 130, graphics module 132, and text input module 134, includes executable instructions for managing an address book or contact list (e.g., stored in memory 102 or in application internal state 192 of contacts module 137 in memory 370), including adding names to the address book, deleting names from the address book, associating phone numbers, email addresses, physical addresses, or other information with names, associating pictures with names, categorizing and sorting names, providing phone numbers and / or email addresses to initiate and / or facilitate communication by phone 138, video conference 139, email 140, or IM 141, and the like.
[0088] In cooperation with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, telephone module 138 includes executable instructions for entering a sequence of characters corresponding to a telephone number, accessing one or more telephone numbers in address book 137, modifying an entered telephone number, dialing each telephone number, conducting a conversation, and disconnecting or hanging up when the conversation is completed. As discussed above, the wireless communication optionally uses any of a number of communication standards, protocols, and technologies.
[0089] In cooperation with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, videoconferencing module 139 contains executable instructions to initiate, conduct, and terminate a videoconference between a user and one or more other participants according to the user's commands.
[0090] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 contains executable instructions for creating, sending, receiving, and managing emails in response to user commands. In conjunction with image management module 144, email client module 140 greatly facilitates the creation and sending of emails with still or video images captured by camera module 143.
[0091] In cooperation with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for entering a series of characters corresponding to an instant message, modifying previously entered characters, sending each instant message (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, sent and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments as supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both telephone-based messaging (e.g., messages sent using SMS or MMS) and Internet-based messaging (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).
[0092] In cooperation with the RF circuitry 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the video and music player module 152, the training support module 142 includes executable instructions to create workouts (e.g., having time, distance, and / or calorie consumption goals), communicate with training sensors (in the sports device and the smart watch), receive training sensor data, calibrate sensors used to monitor the workouts, select and play music for the workouts, and display, store, and transmit training data.
[0093] Camera module 143, along with touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, and image management module 144, contain executable instructions to capture still images or videos (including video streams) and store them in memory 102, change characteristics of the still images or videos, and / or delete still images or videos from memory 102.
[0094] Image management module 144, along with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, contain executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.
[0095] Browser module 147, together with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, contains executable instructions for browsing the Internet according to a user's commands, including retrieving, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.
[0096] Calendar module 148, along with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, contain executable instructions to create, display, modify, and store calendars and data associated with calendars (e.g., calendar entries, to-do lists, etc.) according to a user's instructions.
[0097] Together with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally a mini-application downloaded and used by a user (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or a mini-application created by a user (e.g., user-created widget 149-6). In some embodiments, a widget includes a Hypertext Markup Language (HTML) file, a Cascading Style Sheets (CSS) file, and a JavaScript file. In some embodiments, a widget includes an Extensible Markup Language (XML) file and a JavaScript file (e.g., Yahoo! Widget).
[0098] In cooperation with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget creation module 150 contains executable instructions for creating widgets (e.g., turning user-specified portions of a web page into widgets).
[0099] In cooperation with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 contains executable instructions to search for text, music, sound, images, video, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with a user's commands.
[0100] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 contains executable instructions that enable a user to download and play recorded music or other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions to display, present, or otherwise play videos (e.g., on touch-sensitive display system 112 or on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (a trademark of Apple Inc.).
[0101] In cooperation with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, notes module 153 contains executable instructions for creating and managing notes, to-do lists, and the like according to user commands.
[0102] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contacts module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 contains executable instructions to receive, display, modify, and store maps and map-related data (e.g., driving directions, data about businesses and other points of interest at or near a particular location, and other location-based data) in accordance with a user's instructions.
[0103] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, online video module 155 contains executable instructions that enable a user to access, view, receive (e.g., by streaming and / or downloading), and play (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124) online videos in one or more file formats, such as H.264, and send and otherwise manage emails with links to particular online videos. In some embodiments, instant messaging module 141 is used rather than email client module 140 to send links to particular online videos.
[0104] Each of the above identified modules and applications corresponds to an executable instruction set that performs one or more of the functions described above, as well as the methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). The modules (e.g., instruction sets) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of the modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above identified modules and data structures. Furthermore, memory 102 optionally stores additional modules and data structures not described above.
[0105] In some embodiments, device 100 is a device in which operation of a given set of functions is performed exclusively via a touchscreen and / or touchpad. Using the touchscreen and / or touchpad as the primary input control device for operation of device 100 optionally reduces the number of physical input control devices (push buttons, dials, etc.) on device 100.
[0106] The set of predetermined functions performed solely through the touchscreen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 to a main menu, a home menu, or a root menu from any user interface displayed on device 100. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device rather than a touchpad.
[0107] 1B is a block diagram illustrating example components for event processing, according to some embodiments. In some embodiments, memory 102 (in FIG. 1A) or 370 (in FIG. 3) includes an event sorter 170 (e.g., in operating system 126) and a respective application 136-1 (e.g., any of applications 136, 137-155, 380-390 described above).
[0108] Event sorter 170 receives the event information and determines which application 136-1 to deliver the event information to and application view 191 of application 136-1. Event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192 that indicates the current application view(s) that are displayed on touch-sensitive display system 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event sorter 170 to determine which application(s) are currently active, and application internal state 192 is used by event sorter 170 to determine which application view 191 the event information is delivered to.
[0109] In some embodiments, application internal state 192 includes additional information such as one or more of resume information to be used when application 136-1 resumes execution, user interface state information indicating or ready to display information being displayed by application 136-1, state cues that allow a user to return to a previous state or view of application 136-1, and redo / undo cues of previous actions taken by the user.
[0110] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user's touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer(s) 168, and / or microphone 113 (via audio circuitry 110). Information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.
[0111] In some embodiments, event monitor 171 sends a request to peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, peripherals interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or for longer than a predetermined period of time).
[0112] In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173 .
[0113] Hit view determination module 172 provides software procedures for determining where in one or more views a sub-event occurred when touch-sensitive display system 112 displays more than one view. A view consists of controls and other elements that a user can see on the display.
[0114] Another aspect of a user interface associated with an application is the set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of a particular application) in which a touch is detected optionally correspond to programmatic levels within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are recognized as suitable inputs is optionally determined based at least in part on the hit view of the initial touch that initiates the touch gesture.
[0115] Hit view determination module 172 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should process the sub-events. In most situations, the hit view is the lowest level view in which an initiating sub-event occurs (e.g., the first sub-event in a sub-event sequence that forms an event or potential event). Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.
[0116] Active event recognizer determination module 173 determines which view(s) in the view hierarchy should receive the particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive the particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that contain the physical location of the sub-event are actively participating views, and therefore all actively participating views should receive the particular sequence of sub-events. In other embodiments, even if the touch sub-event is completely confined to the area associated with one particular view, views higher in the hierarchy still remain actively participating views.
[0117] Event dispatcher module 174 dispatches the event information to event recognizers (e.g., event recognizer 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information obtained by each event receiver module 182 in an event queue.
[0118] In some embodiments, operating system 126 includes event sorter 170. Alternatively, application 136-1 includes event sorter 170. In yet other embodiments, event sorter 170 is a stand-alone module or is part of another module stored in memory 102, such as contact / motion module 130.
[0119] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events that occur within a separate view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a separate application view 191 includes multiple event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface kit or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a separate event handler 190 includes one or more of a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from event sorter 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Instead, one or more of the application views 191 include one or more respective event handlers 190. Also, in some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in individual application views 191.
[0120] A separate event recognizer 180 receives event information (e.g., event data 179) from event sorter 170 and identifies an event from the event information. Event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188 (optionally including sub-event delivery instructions).
[0121] The event receiver 182 receives event information from the event sorter 170. The event information includes information about a sub-event, e.g., a touch or a movement of a touch. Depending on the sub-event, the event information also includes additional information, such as a position of the sub-event. When the sub-event involves a movement of a touch, the event information also optionally includes a speed and a direction of the sub-event. In some embodiments, the event includes a rotation of the device from one orientation to another (e.g., from portrait to landscape or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device's attitude).
[0122] The event comparator 184 compares the event information to a predefined event or sub-event definition and determines the event or sub-event or determines or updates the state of the event or sub-event based on the comparison. In some embodiments, the event comparator 184 includes an event definition 186. The event definition 186 includes a definition of an event (e.g., a sequence of predefined sub-events), such as event 1 (187-1) and event 2 (187-2). In some embodiments, the sub-events in event 187 include, for example, touch start, touch cancel, touch move, touch stop, and multiple touches. In one example, the definition for event 1 (187-1) is a double tap on a displayed object. The double tap includes, for example, a first touch (start of touch) for a predefined stage on the displayed object, a first lift-off (end of touch) for a predefined stage, a second touch (start of touch) for a predefined stage on the displayed object, and a second lift-off (end of touch) for a predefined stage. In another example, a definition of event 2 (187-2) is a drag on a displayed object. Drag includes, for example, a touch (or contact) of a predetermined degree on a displayed object, a movement of the touch across the touch-sensitive display system 112, and a lift-off of the touch (end of the touch). In some embodiments, the event also includes information regarding one or more associated event handlers 190.
[0123] In some embodiments, event definition 187 includes definitions of events for individual user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view in which three user interface objects are displayed on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a separate event handler 190, event comparator 184 uses the results of the hit test to determine which event handler 190 to activate. For example, event comparator 184 selects the event handler associated with the sub-event and object that triggers the hit test.
[0124] In some embodiments, each event 187 definition also includes a delay action that delays delivery of the event information until it is determined whether a set of sub-events corresponds to the event type of the event recognizer.
[0125] If the individual event recognizer 180 determines that the sequence of sub-events does not match any of the events in the event definition 186, the individual event recognizer 180 enters an event disabled, event failed, or event finished state and thereafter ignores the next sub-event of the touch-based gesture. In this situation, any other event recognizers that remain active for the hit view continue to track and process sub-events of the ongoing touch gesture.
[0126] In some embodiments, individual event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate to actively participating event recognizers how the event delivery system should perform sub-event delivery. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact with each other or are enabled to interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are delivered to various levels in a view or programmatic hierarchy.
[0127] In some embodiments, the individual event recognizer 180 activates an event handler 190 associated with an event when one or more particular sub-events of the event are recognized. In some embodiments, the individual event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is separate from sending (and postponing sending) the sub-events to the individual hit view. In some embodiments, the event recognizer 180 pops a flag associated with the recognized event, and the event handler 190 associated with the flag catches the flag and performs a predefined process.
[0128] In some embodiments, the event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with a set of sub-events or to an actively participating view. The event handler associated with the set of sub-events or the actively participating view receives the event information and performs a predetermined process.
[0129] In some embodiments, data updater 176 creates and updates data used by application 136-1. For example, data updater 176 updates phone numbers used by contacts module 137 or stores video files used by video and music player module 152. In some embodiments, object updater 177 creates and updates objects used by application 136-1. For example, object updater 177 creates new user interface objects or updates the positions of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on the touch-sensitive display.
[0130] In some embodiments, event handler(s) 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of an individual application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.
[0131] It should be understood that the foregoing description of event processing of a user's touch on a touch-sensitive display also applies to other forms of user input for operating multifunction device 100 using input devices, not all of which are initiated on a touch screen. For example, mouse movements and mouse button presses, contact movements such as tapping, dragging, scrolling on a touchpad, optionally coordinated with single or multiple presses or holds of a keyboard, pen stylus input, device movement, verbal commands, detected eye movements, biometric input, and / or any combination thereof, are optionally utilized as inputs corresponding to sub-events that define the event to be recognized.
[0132] FIG. 2 illustrates portable multifunction device 100 having a touch screen (e.g., touch-sensitive display system 112, FIG. 1A ), according to some embodiments. The touch screen optionally displays one or more graphics in user interface (UI) 200. In these embodiments, as well as those described below, a user is enabled to select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers 202 (not drawn to scale) or one or more styluses 203 (not drawn to scale). In some embodiments, selection of the one or more graphics is performed when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, upwards and / or downwards), and / or rolling of a finger in contact with device 100 (right to left, left to right, upwards and / or downwards). In some implementations or situations, accidental contact with a graphic does not select that graphic: for example, a swipe gesture sweeping over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.
[0133] Device 100 also optionally includes one or more physical buttons, such as a "home" or menu button 204. As mentioned above, menu button 204 is optionally used to navigate to any application 136 in a set of applications optionally running on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display, or as a system gesture such as an upward edge swipe.
[0134] In some embodiments, device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a home button 204), push buttons 206 for powering the device on / off and locking the device, volume control button(s) 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and / or an external docking / charging port 124. Push button 206 is optionally used to power the device on / off by pressing and holding the button down for a predetermined period of time, to lock the device by pressing and releasing the button before the predetermined time has elapsed, and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts verbal input through microphone 113 to activate or deactivate some features. Device 100 also optionally includes one or more contact intensity sensors 165 that detect the intensity of a contact on touch-sensitive display system 112 and / or one or more tactile output generators 167 that generate a tactile output to a user of device 100.
[0135] FIG. 3 is a block diagram of an exemplary multifunction device having a display and a touch-sensitive surface, according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a child's learning toy), a gaming system, or a control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects and controls communication between system components. Device 300 includes input / output (I / O) interfaces 330, including a display 340, which is typically a touchscreen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350, as well as a touchpad 355, a tactile output generator 357 for generating tactile output on device 300 (e.g., similar to tactile output generator(s) 167 described above with reference to FIG. 1A), sensors 359 (e.g., light sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and / or contact intensity sensors similar to contact intensity sensor(s) 165 described above with reference to FIG. 1A). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or a subset of, the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (FIG. 1A). Additionally, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, while memory 102 of portable multifunction device 100 (FIG. 1A) optionally does not store those modules.
[0136] Each of the elements in FIG. 3 identified above is optionally stored in one or more of the memory devices mentioned above. Each of the modules identified above corresponds to an instruction set that performs the functions described above. The modules or programs (i.e., instruction sets) identified above need not be implemented as separate software programs, procedures, or modules, and thus various subsets of those modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. Additionally, memory 370 optionally stores additional modules and data structures not described above.
[0137] Attention is now directed to embodiments of a user interface (“UI”) that is optionally implemented on portable multifunction device 100.
[0138] 4A shows an exemplary user interface for a menu of applications on portable multifunction device 100, according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof: signal strength indicator(s) for wireless communication(s), such as cellular and Wi-Fi signals; ●Time, ●Bluetooth (registered trademark) indicator, ● Battery status indicator, Tray 408 with icons of frequently used applications, such as: ○ An icon 416 for the phone module 138, labeled “Phone”, optionally including an indicator 414 of the number of missed calls or voicemail messages; ○ An icon 418 of the email client module 140, labeled “Mail”, optionally including an indicator 410 of the number of unread emails; ○ Icon 420 of the Browser module 147, labeled "Browser"; ○ An icon 422 of the video and music player module 152 labeled "Music", ● Icons of other applications, such as: ○ Icon 424 of IM module 141, labeled "Messages"; ○ Icon 426 of the Calendar module 148, labeled "Calendar", ○ Icon 428 of the image management module 144, labeled "Photos"; ○ An icon 430 of the camera module 143, labeled “camera”; ○ Icon 432 of the online video module 155, labeled "Online Video"; Icon 434 of Stock Price Widget 149-2, labeled "Stock Price" ○ Icon 436 of the map module 154, labeled "Map"; ○ Icon 438 of weather widget 149-1, labeled "Weather"; ○ Icon 440 of alarm clock widget 149-4, labeled "Clock"; ○ Icon 442 of Training Support Module 142, labeled "Training Support"; ○ Icon 444 of the Notes module 153, labeled "Notes"; ○ An icon 446 for a settings application or module that provides access to settings related to the device 100 and its various applications 136.
[0139] 4A are merely examples. For example, other labels are optionally used for the various application icons. In some embodiments, the labels for individual application icons include the name of the application that corresponds to the individual application icon. In some embodiments, the label of a particular application icon is different from the name of the application that corresponds to that particular application icon.
[0140] FIG. 4B illustrates an exemplary user interface on a device (e.g., device 300, FIG. 3) having a touch-sensitive surface 451 (e.g., tablet or touchpad 355, FIG. 3) separate from a display 450. Although many of the following examples are given with reference to input on touchscreen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface separate from the display, as shown in FIG. 4B. In some embodiments, the touch-sensitive surface (e.g., 451 in FIG. 4B) has a major axis (e.g., 452 in FIG. 4B) that corresponds to a major axis (e.g., 453 in FIG. 4B) on the display (e.g., 450). According to those embodiments, the device detects contact with touch-sensitive surface 451 (e.g., 460 and 462 in FIG. 4B) at locations that correspond to respective locations on the display (e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470). In this manner, when the touch-sensitive surface is separate from the display, user input (e.g., contacts 460 and 462, and their movement) detected by the device on the touch-sensitive surface (e.g., 451 in FIG. 4B ) is used by the device to operate a user interface on the display (e.g., 450 in FIG. 4B ) of the multifunction device. It is to be understood that similar methods are optionally used for the other user interfaces described herein.
[0141] In addition, while the following examples are given primarily with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture, etc.), it should be understood that in some embodiments, one or more of those finger inputs are replaced with input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of a touch) followed by movement of a cursor along the path of the swipe (e.g., instead of moving a touch). As another example, a tap gesture is optionally replaced by a mouse click (e.g., instead of detecting a touch and then stopping detection of the touch) while the cursor is located over the location of the tap gesture. Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice are optionally used simultaneously, or mice and finger touches are optionally used simultaneously.
[0142] The term "focus selector" as used herein refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations involving a cursor or other location marker, the cursor functions as a "focus selector" such that when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B) while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element is adjusted according to the detected input. In some implementations involving a touchscreen display (e.g., touch-sensitive display system 112 in FIG. 1A or touchscreen in FIG. 4A) that enables direct interaction with user interface elements on the touchscreen display, a contact detected on the touchscreen functions as a "focus selector" such that when an input (e.g., a press input by contact) is detected on the touchscreen display at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one region of the user interface to another region of the user interface (e.g., by using the tab or arrow keys to move focus from one button to another) without a corresponding cursor movement or contact movement on the touchscreen display. In these implementations, the focus selector moves to follow the movement of focus between various regions of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally a user interface element (or a contact on a touchscreen display) that is controlled by the user to communicate the user's intended interaction with the user interface (e.g., by indicating to the device which element of the user interface the user intends to interact with).For example, positioning of a focus selector (e.g., a cursor, contact, or selection box) over an individual button while a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that individual button (and not other user interface elements shown on the device's display).
[0143] User Interface and Related Processes Attention is now directed to embodiments of user interfaces ("UI") and associated processes that may be implemented on an electronic device, such as portable multifunction device 100 or device 300, that includes a display and a touch-sensitive surface, (optionally) one or more tactile output generators for generating tactile output, and (optionally) one or more sensors for detecting the intensity of contact with the touch-sensitive surface.
[0144] 5A-5CY show exemplary user interfaces for updating state information in a session area, according to some embodiments. FIGS. 6A-6M show exemplary user interfaces for enrolling in biometric authentication, according to some embodiments. FIGS. 7A-7E show exemplary user interfaces for updating a session area to use a stored payment method, according to some embodiments. FIGS. 8A-8G show exemplary user interfaces for updating a session area for a virtual assistant, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in FIGS. 9A-9G, 10A-10D, 11A-11D, 12A-12E, 13A-13E, 14A-14C, 15A-15E, and 16A-16F. For ease of explanation, some of the embodiments are discussed with reference to operations performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally a respective finger or stylus contact, a representative point corresponding to a finger or stylus contact (e.g., the face centroid of a respective contact or a point associated with a respective contact), or the face centroid of two or more contacts detected on touch-sensitive display system 112. However, a similar operation is optionally performed on a device having display 450 and separate touch-sensitive surface 451 in response to detecting a contact on touch-sensitive surface 451 while displaying the user interface shown in the figure on display 450, along with a focus selector.
[0145] 5A-5CY illustrate exemplary user interfaces for updating state information in a session area according to some embodiments.
[0146] FIG. 5A illustrates an exemplary user interface of home screen (also referred to as home user interface) 501 of portable multifunction device 100. In some embodiments, portable multifunction device 100 is a computer system, a handheld mobile device, a tablet, or other client device. In some embodiments, the home screen user interface includes icons for navigating to multiple applications executed or executable by device 100. In some embodiments, a user can interact with device 100 using one or more gestures, including touch input. For example, a tap input on an individual application icon launches the individual application on a display area of device 100 or otherwise opens the individual application's user interface. In some embodiments, multiple views (also referred to as pages) of the home screen user interface are available. For example, a user can swipe or otherwise navigate between multiple views of the home screen user interface, and in some embodiments, each view includes a different application icon for a different application. In some embodiments, the application icons are different sizes, such as application widgets that display information for the individual applications, and the application widgets are larger than the application icons.
[0147] In some embodiments, the device 100 includes a session area 502-1 that includes one or more sensors (e.g., speaker 111 and / or one or more light sensors 164). In some embodiments, the one or more sensors are disposed in one or more cutouts (also referred to as sensor areas) in the display of the device 100. In some embodiments, the session area 502-1 encompasses one or more sensor cutouts for one or more sensors. In some embodiments, additional sensors are disposed in the session area 502-1, and the cutouts shown in FIG. 5A include one or more sensors in the cutout (e.g., one or more additional sensors are disposed in the same cutout as the speaker 111 and / or one or more additional sensors are disposed in the same cutout as the light sensor(s) 164, such as a structured light emitter or projector). It will be understood that in some embodiments, alternative shapes and / or numbers of cutouts (e.g., more than two or less than two) and numbers of sensors in individual cutouts are implemented. In some embodiments, the cutouts are not visible from the surface of the device 100. In some embodiments, the device displays an overview of the session area 502-1. For example, the device displays a black session area 502-1 that encompasses cutouts for the speaker 111 and optional sensor 164. In some embodiments, the device displays the session area 502-1 in a color that matches or otherwise blends with the color of the sensor located within the cutout.
[0148] In some embodiments, the area between the two sensor cutouts remains the same color as the color of the sensor. For example, the area between two of the sensor cutouts includes a display that displays a color selected to match the color of the sensor hardware. In some embodiments, at least one of the sensor cutouts includes a camera as the sensor within the sensor cutout. In some embodiments, the area between two of the sensor cutouts displays content (e.g., a privacy indicator and / or a lock indicator). Illustrative examples of the area between the sensor cutouts are provided in the figures and accompanying description herein.
[0149] In some embodiments, the privacy indicator 518 is optionally displayed in an area between two of the sensor cutouts. In the example of FIG. 5A, the privacy indicator 518 is displayed in the session area 502-1 in an area between the cutouts for the speaker 111 and the cutouts for the sensor(s) 164. As described in more detail herein, the privacy indicator 518 typically indicates current or recent use of the camera, microphone, and / or location sensor to capture information about the device, the device's environment, or other user-specific and / or user-identifying information, such as by an application running on the device 100. In some embodiments, displaying the privacy indicator to indicate use of the camera, microphone, and / or location sensor is optional for applications whose sessions are not displayed in the session area. In some embodiments, the privacy indicator is displayed with increased brightness in some situations, such as when the device 100 is in a bright environment (e.g., in sunlight), to make the privacy indicator more noticeable to the user. In some embodiments, the privacy indicator has different colors for different types of data being captured (e.g., when location data is being collected, the privacy indicator is a first color, such as blue or purple; when visual data is being collected, the privacy indicator is a second color, such as green, white, or yellow; and / or when audio data is being collected, the privacy indicator is a third color, such as orange, red, or brown). In some embodiments, multiple privacy indicators are displayed simultaneously if multiple types of data are being collected simultaneously. In some embodiments, a single privacy indicator is displayed when more than one type of data is being collected (e.g., when both visual and audio data are being collected, a privacy indicator indicating that visual data is being collected is displayed, or when both audio and location data are being collected, a privacy indicator indicating that audio data is being collected is displayed).
[0150] In some embodiments, session region 502-1 is displayed with a shape that includes a curvature that matches the curvature of the display (e.g., the corners of the display) of device 100, such as the shape of the corners of touch screen 112 in Figure 5A. In some embodiments, the session region maintains a curvature that matches the curvature of the display even when the session region is expanded to show additional information and / or control of the session, as described below.
[0151] In some embodiments, session region 502-1 displayed without an active session (e.g., without state information) and / or session regions described herein displayed with at least one active session (e.g., with state information) are displayed in predetermined locations on the display as the user navigates between different user interfaces. For example, the session regions are displayed in the same area of the display and device 100 displays application user interfaces, a home screen user interface, and optionally a wake screen user interface (e.g., on top of touch screen 112, as shown throughout the figures).
[0152] FIG. 5B illustrates the home screen user interface described with reference to FIG. 5A. FIG. 5B further illustrates a session area 502-2 that does not include one or more sensors within the session area 502-2. For example, in some embodiments, the one or more sensors are not disposed within a display cutout of the device 100. In some embodiments, the session area 502-2 that does not include one or more sensors is enabled to perform all of the functions described herein (e.g., any of the same functions described herein as for the session area 502-1). While most examples described herein show one or more sensors within the session area, in some embodiments, the session area is displayed regardless of whether one or more sensors are included in the session area.
[0153] FIG. 5A illustrates a user input 504-1, such as a tap input to select an icon 424 for a messaging application. User input 504-2 in FIG. 5B is similar to user input 504-1 in FIG. 5A. In response to user input 504-1 (or user input 504-2), device 100 displays user interface 500 for the messaging application, as shown in FIG. 5C. As shown in FIG. 5C, device 100 maintains display of session area 502-1 (or, similarly, session area 502-2) while the user navigates between user interfaces on device 100 (e.g., from a home screen user interface to a user interface for a messaging application). In some embodiments, while device 100 is displaying user interface 500, device 100 receives a request to initiate a communication session. For example, device 100 receives an incoming phone call (e.g., from "Max"). In some embodiments, in response to receiving an incoming telephone call, device 100 updates session area 502-1 to expanded session area 502-3, as shown in FIG. 5D. In some embodiments, expanded session area 502-3 includes information about the incoming request and / or one or more controls for interacting with the incoming request. For example, session area 502-3 includes contact information associated with the incoming call (e.g., information indicating that the call is from “Max”) and / or a stored contact photo or icon associated with the contact. In some embodiments, session area 502-3 further includes a number of control options, including an option 510 to accept the call and an option 508 to reject the call, as shown in the example of FIG. 5D.
[0154] In some embodiments, the session region 502-3 is displayed with visual effects on at least one side of the session region 502-3. For example, the session region 502-3 is displayed with a shadow 514-1. In some embodiments, the size and / or darkness of the shadow is selected according to the current size of the session region. While a session region is displayed at a minimum size (e.g., its minimum size) (e.g., session region 502-1), such as when there is no active session associated with the session region, the session region is displayed without a shadow or other visual effects. In some embodiments, while a session region is minimized, the session region is considered to be an empty session region (e.g., even if some status information is displayed in the status region, if the displayed status region fits into the minimized status region, such as between sensor regions, such as privacy indicator information or device lock / unlock status information). In some embodiments, the session area is displayed at a first size larger than its minimum size (e.g., a condensed size, such as the size of session area 502-3, where a first set of information and / or interaction options of the active session are displayed), but the session area is displayed by a shadow 514-1, where the shadow 514-1 has a first size and a first darkness level (or collectively, a first intensity). In some embodiments, the session area is displayed at a second size larger than the first size (e.g., an expanded size, such as the size of session area 502-5 of FIG. 5I, where a larger second set of information and / or interaction options of the active session are displayed), but the session area is displayed by a shadow 514-2 having a second size and a second darkness level (or collectively, a second intensity), where the second size and / or second darkness level are larger than the first size and / or first darkness level (e.g., the second intensity is larger than the first intensity).
[0155] 5D further illustrates that while the session region 502-3 is displayed in an expanded size, the expanded size including one or more controls and / or additional information of the current session (e.g., in the case of a phone call) associated with the session region, a visual effect is applied to a portion of the display area that is outside of the session region. For example, a blur visual effect 512 is applied to a portion of the user interface 500 to highlight the session region 502-3 in that the session region 502-3 appears to be displayed above a visually unemphasized portion of the user interface 500. In some embodiments, the blur visual effect 512 is applied to the top third or top half, or other proportion or percentage, of the user interface 500. In some embodiments, the blur visual effect 512 has a predetermined size (e.g., maintains size) regardless of the current size of the session region, including whether the session region is expanded or collapsed (or minimized). In some embodiments, the blur visual effect 512 varies in intensity based on the area of the user interface 500 to which it is applied. For example, in some embodiments, the blur visual effect 512 is a gradient effect that creates more blur in the user interface closer to the session area and less blur in the user interface farther from the session area. For example, the blur visual effect 512 gradually decreases the amount of blur in the user interface 500 as the distance from the session area increases. In some embodiments, the gradient of the blur visual effect 512 is applied (e.g., progresses) in one direction (e.g., downward) along the user interface.
[0156] In some embodiments, as the size of the session area increases, the blur visual effect 512 is displayed gradually (e.g., as it moves outward and / or downward from the session area, possibly by progressing through multiple intermediate states over time, and / or as it increases in intensity over time). For example, while the session area is collapsed or minimized, optionally the blur visual effect 512 is not applied to the user interface surrounding the session area. In some embodiments, as the session area gradually expands, the blur visual effect 512 gradually shifts onto the display (optionally at the same or proportional speed as the expansion of the session area), optionally animating outward from the collapsed session area to the expanded session area. In some embodiments, the blur visual effect 512 gradually shifts downward from the top edge of the display. In some embodiments, as the session area collapses from its expanded size to its condensed size or to its minimum size, the blur visual effect 512 gradually shifts off the display, optionally by sliding up or down the top edge of the display and / or by reducing in intensity to fade out.
[0157] In some embodiments, device 100 detects user input 516, such as a tap input or other gesture, that selects option 510 to answer the call. In response to user input 516, the device establishes a communication session (e.g., between calling user "Max" and the user of device 100) and automatically reduces session area 502-3 to a condensed active session size, as shown as session area 502-4 in FIG. 5E, where the condensed active session size is a non-expanded view of the active session. However, the non-expanded view of the active session is expanded relative to a minimized (e.g., minimum) session area size (e.g., the size of session area 502-1). For example, in FIG. 5C , no sessions are active and thus session area 502-1 is displayed at a minimized size that does not include information about the active session, whereas in FIG. 5E , there is an active session of a communication session (e.g., the “Max” call) and the session area is expanded (e.g., to the left and / or right, and / or upward, and / or downward) relative to its minimized size to include information about the ongoing session (e.g., less expanded than in FIGS. 5D and 5I ).
[0158] In some embodiments, as shown in FIG. 5C, one or more status indicators (e.g., the current device time labeled "Time", battery indicator 506, cellular network indicator 503, and / or Wi-Fi indicator 505) are displayed while the session area is minimized. In some embodiments, when the status area increases in size from a minimal size (FIG. 5C) to a condensed or expanded size (FIG. 5D), one or more status indicators are repositioned on the display (e.g., to correspond to the session area increasing in size). For example, when session area 502-1 (FIG. 5C) is updated to session area 502-3 (FIG. 5D), the "Time" indicator is optionally moved to the left and battery indicator 506, cellular network indicator 503, and / or Wi-Fi indicator 505 are optionally moved to the right. In some embodiments, the battery indicator 506, the cellular network indicator 503, and / or the Wi-Fi indicator 505 are optionally compressed and displayed closer to one another when the display of one or more of the battery indicator 506, the cellular network indicator 503, and / or the Wi-Fi indicator 505 is stopped.
[0159] In some embodiments, while the session area is expanded, one or more status indicators (e.g., battery indicator 506, cellular network indicator 503, and / or Wi-Fi indicator 505) are stopped from being displayed. For example, one or more status indicators are removed from the display based on the current size of the session area. In some embodiments, for example, while the session area is expanded to cover the entire portion of the display in which one or more status indicators are displayed, all of the status indicators are no longer displayed (e.g., as shown in FIG. 5D). In some embodiments, the one or more status indicators to be removed are selected based on the current state of device 100, particularly in situations in which less than all of the one or more status indicators are to be removed. For example, the device may be capable of switching between a mode using cellular data and a mode using Wi-Fi. In some embodiments, indicators consistent with the current mode of device 100 continue to be displayed, and indicators of modes of device 100 that are not currently active are not displayed (e.g., are removed). For example, in Fig. 5E, cellular network indicator 503 is displayed while Wi-Fi indicator 505 is hidden, optionally pursuant to a determination that device 100 is using a cellular network and is not connected to or actively using Wi-Fi (e.g., in the case of an ongoing phone call). In Fig. 5F, cellular network indicator 503 is hidden while Wi-Fi indicator 505 is displayed pursuant to a determination that device 100 is connected to a cellular network and / or is using Wi-Fi and is not actively using the cellular network (e.g., in the case of an ongoing phone call). In some embodiments, when the device changes from being connected to Wi-Fi and / or actively using Wi-Fi to being connected to a cellular network and / or actively using the cellular network (or vice versa), the currently displayed status indicator is dynamically updated to reflect the current connection type.
[0160] In some embodiments, the number and spacing of the status indicators change dynamically as the session area changes size (e.g., in FIG. 5C, time is shown to the left of the session area, and a cellular signal indicator, a Wi-Fi indicator, and a battery indicator are displayed to the right of the session area), and the number of status indicators changes as the session area expands (or additional session areas are added) (e.g., removing the cellular signal indicator as shown in FIGS. 6E and 6F). In some embodiments, as the session area size changes, one or more status indicators move gradually to create more space (e.g., if the size of the session area is increasing) or to fill empty space (e.g., if the size of the session area is decreasing). In some embodiments, the movement of the status indicators changes the spacing between the status indicators (e.g., increasing or decreasing the space between adjacent status indicators). In some embodiments, if only a single status indicator is present, the status indicator moves left or right as the session area gets larger or smaller to keep the status indicator approximately centered in the area between the edge of the display and the edge of the session area closest to the status indicator. In some embodiments, the change in size of the session area is gradual, and the corresponding change in the number and / or spacing of the status bar indicators is also gradual. Although specific examples of status indicators are described above, it should be understood that the device may display a variety of status indicators in the status area, including indicators for notifications, network activity, wireless connection, Bluetooth connection, headphone connection, location usage, virtual private network connection status, etc.
[0161] In some embodiments, in the condensed active session size shown in FIG. 5E, the session area 502-4 includes information about the current session. For example, the session area 502-4 displays information about an ongoing communication session. In some embodiments, the information includes an indication of the application or software currently being used for the active session. For example, a phone icon is displayed and the duration of the ongoing call is displayed. In some embodiments, an audio waveform is displayed to indicate incoming and / or outgoing audio information (e.g., that is part of a call). In some embodiments, different portions of the waveform (e.g., along one axis, such as horizontal) represent different audio frequencies, and in some such embodiments, the height of the individual portions (e.g., different portions) of the waveform represent the amplitude of the audio signal at the frequency or frequency band corresponding to the individual portions of the waveform.
[0162] In some embodiments, the audio waveform includes at least two portions including a first portion (e.g., a left-most or left-aligned portion) and a second portion (e.g., a right-most or right-aligned portion). In some embodiments, the first portion displays incoming audio (e.g., audio received during a call) and the second portion displays outgoing audio (e.g., audio detected by one or more microphones of device 100 that is transmitted to one or more other call participants during a call), or vice versa. In some embodiments, the first portion of the audio waveform is displayed in a first color, such as blue or green (or a first color range, such as a blue-green spectrum of colors), and the second portion of the audio waveform is displayed in a second color (or a second color range, such as a red-orange spectrum of colors) that is different from the first color, such as red or orange. Thus, the audio waveform visually distinguishes between different audio sources (e.g., incoming audio data and outgoing audio data). In some embodiments, the audio waveforms at least partially overlap. In some embodiments, each audio waveform (e.g., for incoming and outgoing audio) is assigned a different range of the audio waveform such that when displayed simultaneously, both audio waveforms at least partially overlap and do not overlap, e.g., incoming audio is displayed only or primarily in the left-justified portion of the audio waveform and outgoing audio is displayed only or primarily in the right-justified portion of the audio waveform.
[0163] In some embodiments, the session region 502-4 optionally includes a privacy indicator 518 (e.g., as described herein with reference to FIG. 5A ) that is displayed in a display area between the sensor cutouts (e.g., for the speaker 111 and / or the sensor 164) and indicates current or recent use of a camera, microphone, and / or position sensor to capture information about the device or the device's environment. For example, the session region 502-4 may surround each of the sensor cutouts and display one or more icons or other visual elements between the sensor cutouts (e.g., the area between the sensors is part of the display of the device 100). In some embodiments, while the session region displays an active session, the privacy indicator 518 is displayed outside the session region, for example, in a corner of the display (e.g., next to or above the battery indicator 506). In some embodiments, initially displaying the privacy indicators 518, e.g., in response to the device 100 starting to use one or more sensors that triggered the display of the privacy indicators 518, includes gradually changing (e.g., reducing) the blurriness of the privacy indicators 518 (e.g., progressing through multiple intermediate display states). In some embodiments, ceasing to display the privacy indicators 518, e.g., in response to the device 100 stopping to use one or more sensors that triggered the display of the privacy indicators 518, is performed without gradually changing the blurriness of the privacy indicators 518 (e.g., the blurriness of the privacy indicators 518 is maintained regardless of whether other visual characteristics, such as opacity and / or brightness of the privacy indicators 518 are gradually changed as the display of the privacy indicators 518 is stopped).
[0164] In some embodiments, session area 502-4 updates the displayed information while the session remains active. For example, while the communication session continues, the length of time the call is continuing is updated (e.g., from 0:08 in FIG. 5E to 0:09 in FIG. 5F) and the audio waveform is updated according to the current audio data. For example, if "Max" is providing audio data, the portion of the audio waveform for the incoming audio is animated according to the audio data, but if the user of device 100 is not providing audio data (e.g., the user of device 100 is not speaking), the portion of the audio waveform for the outgoing audio is displayed as flat (e.g., no waveform or a waveform with zero volume).
[0165] In some embodiments, in response to a user input 520 (e.g., a swipe input, a drag gesture, a button press, or other user input), device 100 stops displaying user interface 500 and displays home user interface 501. In some embodiments, the display of session area 502-4 is maintained as the user navigates between an application user interface, such as messaging application user interface 500, and other application user interfaces, such as home screen user interface 501, and / or other system user interfaces. Thus, the user can view up-to-date information about ongoing sessions as the user interacts with other applications on device 100.
[0166] 5G-5H show session area 502-5 displayed on device 100 in an embodiment in which device 100 is configured without one or more sensors in session area 502-5. For example, the features and functionality described above with reference to FIGS. 5E-5F are also features and functionality that may be enabled for devices as shown in FIGS. 5G-5H. Thus, session area 502-5 displays the same information about an active session regardless of whether one or more sensor notches and / or one or more sensors are located in the sensor area. For example, session area 502-5 in FIG. 5G has the same appearance as session area 502-4 in FIG. 5E, and session area 502-5 in FIG. 5H has the same appearance as session area 502-4 in FIG. 5F.
[0167] In some embodiments, in response to detecting a user input 522 as shown in FIG. 5F (or a similar user input 522-2 as shown in FIG. 5H), such as a tap input, session region 502-4 (or session region 502-5) expands to display additional information about the active session in session region 502-6, as shown in FIG. 5I. In some embodiments, in addition to expanding the display of the session region to session region 502-6, device 100 provides non-visual output, such as audio and / or haptic output. As described above, in some embodiments, the size and / or darkness of shadow 514-2 is based on the size of session region 502-6. For example, a larger session region corresponds to a larger size and / or a darker shadow (e.g., as shown for session region 502-6) than the shadow of a smaller region (e.g., as shown for session region 502-3). In some embodiments, one or more session areas are displayed without shadows, such as in a minimized session area and / or a condensed session area associated with a particular application (e.g., session areas 502-1 and 502-4).
[0168] In some embodiments, a visual effect, such as blur visual effect 512, is displayed on the same portion of currently displayed user interface 500 of FIG. 5I as the portion of user interface 501 described with reference to FIG. 5D.
[0169] FIG. 5I illustrates an expanded session area 502-6 that includes multiple control options for controlling the operation of software and / or applications associated with a current session. For example, during an ongoing communication session, in response to a user providing a first type of input (e.g., tap input 522) on the session area 502-4, multiple control options are displayed, including a control 530-1 for ending the communication session, a control 530-2 for initiating a video communication session, and / or a control 530-3 for muting / unmuting the microphone of the device 100. In some embodiments, additional and / or alternative control options are also displayed, such as a keypad control for entering numbers and / or a speaker control for switching the speakerphone on or off for the communication session. It will be appreciated that different applications and / or software associated with individual sessions provide different sets of controls for the individual sessions. For example, controls for controlling a communication session are provided in FIG. 5I. In another example, controls are provided for stopping and / or pausing a timer during a timer session. In yet another example, controls for media playback are provided for a media player session. In some embodiments, in response to a user interacting with a particular control in session area 502-6, device 100 performs an action corresponding to the control. For example, in response to user-selected control 530-3, device 100 mutes (if currently muted) or unmutes (if currently unmuted) the microphone and stops providing outgoing voice data to the communication session with “Max,” as described with reference to FIGS. 5K-5M.
[0170] In some embodiments, the privacy indicator 518 is optionally displayed in a different position while the session area 502-6 is expanded than while the session area is condensed or minimized. For example, the privacy indicator 518 is optionally displayed at an edge of the expanded session area 502-6 or outside the session area 502-6 (e.g., in the background next to a battery status indicator, or in a top corner of the display) and not between the sensor cutouts. For example, FIG. 5I shows a privacy indicator 518-a that is optionally and alternatively displayed outside the expanded session area 502-6, in a corner of the display area, as opposed to the privacy indicator 518 that is displayed at the edge of the expanded session area 502-6. In some embodiments, the session area is expanded to a size that leaves a small area near the edge of the display unoccupied (e.g., as shown in FIG. 5D, FIG. 5AK, FIG. 5AM, and FIG. 5AY, in which case the privacy indicator is displayed at the top, left, or right side in FIG. 5D or FIG. 5AY, or between an edge of the session area including the top edge in FIG. 5AK or FIG. 5AM and an edge of the display close to the session area). In some embodiments, the privacy indicator 518 is optionally displayed in a different position while the session area is condensed than while the session area is minimized (and in yet another different position while the session area is expanded). For example, in FIG. 5J, the privacy indicator 518-b is instead displayed outside the condensed session area 502-5, optionally next to (e.g., to the right or left of) the session area 502-5, rather than within (e.g., between) the sensor area.
[0171] In some embodiments, while the session area 502-6 is expanded to display multiple control options, the user cannot interact with user interface elements displayed in the user interface 501 that are outside of the session area 502-6. For example, user input 526 on an application icon 438 optionally does not launch the application associated with the icon 438. Instead, in some embodiments, user input 526 detected outside the session area 502-6 simply causes the session area 502-6 to shrink or otherwise decrease in size back to its condensed size as the session area 502-5, and optionally ceases to display any corresponding visual effects, such as shadows and / or blur effects. In some embodiments, the session area continues to update information related to the ongoing session, such as updating the length of time the communication session has been in progress, even as the size of the session area changes. In some embodiments, while session region 502-6 is expanded, at least a portion of the content displayed in the user interface outside of session region 502-6 is at least partially obscured by a visual effect (e.g., a shadow and / or blur effect around session region 502-6) and / or is completely obscured by session region 502-6. Thus, FIG. 5J illustrates session region 502-6 reduced to its condensed size as session region 502-5 in response to user input 526, and with shadow 514-2 and blur effect 512 stopped from being displayed.
[0172] FIG. 5J illustrates detection of a user input 528 that at least partially overlaps with the session area 502-5. In some embodiments where one or more sensors are disposed within the session area, the device 100 is capable of detecting a user input occurring across one or more sensors and treats such user input as an input directed to the session area. For example, when a user taps a portion of the device 100 that corresponds to at least one sensor within the session area, the device 100 interprets the user input on the sensor as an input directed to the session area. In some embodiments, in response to a user input 528 that is at least partially directed to the session area 502-5, and in particular to a sensor area within the session area 502-5, the session area 502-5 is expanded into the session area 502-6 (e.g., of FIG. 5K). In some embodiments, different actions are performed based on the type of user input 528 directed to the session area 502-5. For example, upon detection of user input directed at session area 502-5, if the user input is a tap user input, device 100 displays an application user interface corresponding to the active session displayed in session area 502-5 (e.g., a call user interface displayed outside the session area instead of home user interface 501), and if the user input is a long press user input, device 100 displays an expanded session area 502-6 (e.g., while maintaining display of home user interface 501, without displaying an application user interface).
[0173] 5K-5M show examples of a user selecting one of the control options displayed in the expanded session area 502-6. In some embodiments, the user maintains the user input 528 shown in FIG. 5J by continuing to contact the display of the device 100. For example, the user may initiate contact at the location of the user input 528 (FIG. 5J) and, while maintaining contact with the display, move the contact to perform a drag or swipe gesture. In some embodiments, the session area 502-5 is expanded into the expanded session area 502-6 in response to a first portion (e.g., a long press portion) of the user input 528, and the movement of the contact (e.g., a drag portion) occurs during a second continuous portion of the user input 528 that immediately follows the first portion. In some embodiments, a location on the display where the user notes the contact (also referred to herein as a lift-off of the contact) is selected. For example, in FIG. 5K, the user drags the user input 528 to a location on the display corresponding to the control 530-3 before the user releases the contact 528-1 from on the control 530-3. In response to the user lifting contact 528-1 off control 530-3, device 100 mutes the microphone and ceases providing audio output to the communication session, thereby performing the action associated with control 530-3, and optionally provides audio and / or haptic feedback in response to the lift-off of contact 528-1.
[0174] In some embodiments, as a user moves a contact around the display area, audio and / or tactile output is optionally provided by device 100 in response to the contact overlapping a control option to indicate that the user input is enabled for selecting the control option, and to indicate that the control option is currently selected for activation if user input 528 is lifted off of its current position. For example, in the process of moving to control 530-3 in FIG. 5K, contact 528-1 moves over keypad control 528-4, and in response, device 100 generates audio and / or tactile feedback indicating that contact 528-1 is over an activatable control option. However, in a scenario in which contact 528-1 is not lifted over keypad control 528-4, the keypad is not displayed in response to user input 528. For example, as a contact from user input 528 moves through session region 502-6, device 100 may provide non-visual output (e.g., audio and / or haptic output) to alert the user when the contact moves through any of the control options (e.g., each time the contact moves over an individual control option). For example, a first non-visual output may be generated in response to user input 528 moving over a first control option, followed by a second non-visual output generated in response to user input 528 instead moving away from the first control option and over a second control option (or, in some embodiments, moving away from and then back over the same first control option).
[0175] FIG. 5L illustrates an example of a user lifting off an initiated contact 528-2 as a user input 528 to a control option 530-2 for initiating a video communication session. In response to the user lifting off contact 528-2 from over control option 530-2, device 100 initiates a video call. Similarly, in FIG. 5M, in response to the user releasing contact 528-3 while in an overlapping position with control option 530-1 for terminating the communication session, device 100 terminates the ongoing communication session (e.g., hangs up on "Max").
[0176] In some embodiments, following a determination that the ongoing session has ended, such as by hanging up the call in FIG. 5M, session area 502-1 is re-displayed as shown in FIG. 5N. In some embodiments, session area 502-1 is maintained on the display while device 100 is on and active (e.g., not in a sleep or low power state). Thus, even after a user has ended a communication session that was displayed in session area 502-6, device 100 reduces session area 502-1 to a minimized state with no active session and continues to display session area 502-1.
[0177] FIG. 5N illustrates detection of user input 532 on session region 502-1. In some embodiments, in response to a determination that an active session is not currently associated with session region 502-1, session region 502-1 expands in response to user input 532, and expanded session region 502-1b, shown in FIG. 5O, optionally expands the background color (e.g., black) of the session region in all directions. In some embodiments, session region 502-1 expands in one direction more than the other, or in one direction but not the other. For example, the session region expands horizontally but not vertically (e.g., while device 100 is in portrait mode as shown in FIG. 5O), or more commonly, lengthens (e.g., in the longer dimension) with less (or no) change in width (e.g., in the shorter dimension). In some embodiments, the session region is expanded into session region 502-1b for a predetermined period of time (e.g., 0.2 seconds, 0.5 seconds, 1 second, 5 seconds, 10 seconds, 30 seconds, or 60 seconds). In some embodiments, after expanding into session region 502-1b, the session region automatically shrinks back to its minimized size as session region 502-1 of FIG. 5P (e.g., upon expiration of the predetermined period of time).
[0178] In some embodiments, session region 502-1 expands to session region 502-1b directly in response to an initial detection of user input 532. In some embodiments, session region 502-1 expands to session region 502-1b pursuant to a determination that user input 532 is ongoing (e.g., maintained) for a threshold time (e.g., user input 532 is an input such as a press-and-hold input maintained for at least 1 second, 2 seconds, 3 seconds, or 5 seconds). In some embodiments, session region 502-1b optionally shrinks or reduces in size pursuant to termination of user input 532 (e.g., pursuant to lift-off of user input 532). In some embodiments, session area 502-1b optionally shrinks regardless of the state of user input 532, e.g., session area 502-1b shrinks after a predetermined period of time has elapsed (e.g., after initial detection of user input 532 or after session area 502-1 is expanded into session area 502-1b) even if user input 532 is still in progress. In some embodiments, in response to user input 532, and optionally following a determination that user input 532 has continued for a threshold time, device 100 optionally outputs non-visual feedback, such as a tactile output and / or an audio output (e.g., as shown by output 533 of FIG. 5O). In some embodiments, if there is an active session associated with the session area and user input is detected in the session area, device 100 optionally provides different non-visual feedback (e.g., a different tactile output and / or a different audio output) or does not provide a tactile and / or audio output while providing additional information about the active session, as described with reference to FIGS. 5AL-5AM.
[0179] FIG. 5P illustrates a user input 534, such as a tap input or a press-and-hold gesture, detected on an icon 440 for a clock application. In some embodiments, in response to detecting the user input 534, the device 100 displays a user interface 536-1 for the clock application, as shown in FIG. 5Q. In some embodiments, the user interface 536-1 provides the user with an option to set a timer on the device 100 and select an audio or other notification to alert the user when the timer is completed. FIG. 5Q illustrates the user starting a 15-minute timer by selecting a start button on the user interface 536-1, via user input 538. In some embodiments, the session area 502-1 remains displayed while the user interacts with the user interface of the clock application, as shown in FIG. 5Q. In FIGS. 5Q-5R, the session area 502-1 is not associated with a currently active session and therefore appears as a solid background (e.g., black or another solid color).
[0180] FIG. 5R illustrates a second user interface 536-2 for a clock application that includes a display of a timer countdown. In some embodiments, the device 100 detects a user input to switch from a display of the second user interface 536-2 for the clock application to another user interface. For example, a user may navigate from an application user interface for a first application to an application user interface for a second application and / or a system user interface (e.g., a home screen and / or a wake screen). In some embodiments, a swipe gesture such as input 538 on a particular portion of the display (e.g., a bottom edge in a horizontal direction (e.g., left or right)) allows a user to navigate between application user interfaces of applications currently open or otherwise running on the device 100. For example, as shown in FIGS. 5R-5S, a user switches from the second user interface 536-2 for the clock application to a user interface 500 for a messaging application. In some embodiments, the application user interface displayed in response to application switch input 538 is the user interface of the most recent previously opened application or the user interface of the next (or previous) application in the series of open or recently opened applications after the clock application (e.g., a messaging application or another recently opened application).
[0181] FIG. 5S further illustrates that in response to a user input 538 that causes device 100 to stop displaying second user interface 536-2 (e.g., because that information is no longer visible in the main application area), information about the ongoing timer is displayed in session area 502-7 while user interface 500 is displayed in the remainder of the display. In some embodiments, the timer is one of multiple event types associated with an application displayed in session area 502-7. For example, as described in more detail below with respect to FIG. 5BL, the session area displays continuous updates of multiple session types while a session is active and / or ongoing, including an active timer session. In some embodiments, session area 502-7 displays an application icon (e.g., a timer icon to the left of the sensor) that identifies the application associated with the active session, and displays (e.g., to the right of the sensor) a first set of information about the active session, such as a countdown of the remaining time on the timer (e.g., "14:49" in FIG. 5S). In some embodiments, as described above, the session region is large enough to display information about currently active sessions, but one or more status indicators are removed (e.g., the cellular connection status indicator is not displayed in FIG. 5S), particularly when the session region is expanded to occupy at least a portion of the display area in which the status indicators are displayed (e.g., the top right area of the display in FIG. 5S). In some embodiments, all status indicators are optionally removed (e.g., pursuant to a determination that the session region is expanded horizontally to overlap all status indicators, such as by overlapping at least a portion of all status indicators).
[0182] In some embodiments, the session area 502-7 continues to be displayed as the user navigates among other application user interfaces. For example, in FIG. 5S, a user input 540 (e.g., a left-to-right edge swipe gesture) is detected, and in response to the user input 540, the device 100 displays a user interface 542 of another application (e.g., a web browser), as shown in FIG. 5T, optionally an application that was recently opened on the device 100 or is the next (or previous) application in the series of open applications after a messaging application. As shown in FIG. 5T, the session area 502-7 continues to update by changing the remaining time on the timer (e.g., "14:48" in FIG. 5T).
[0183] In some embodiments, in response to a user input 544 (e.g., a swipe up gesture initiated at the bottom or bottom edge of the display or other system navigation input or home navigation input) to navigate to a system user interface (e.g., a home screen user interface), as shown in FIG. 5T, the device displays the home screen user interface 501 and continues to update the active timer session in the session area 502-7, as shown in FIG. 5U, which indicates that the remaining time on the timer has further changed (e.g., to "14:47"). In some embodiments, the user can navigate to another system user interface, such as another view of the home screen user interface 501-2 via a left swipe input 546, as shown in FIG. 5V. As the user navigates between system user interface views, the session area 502-7 is maintained and updated with the time remaining in the active timer session (e.g., "14:46" in FIG. 5V). As shown throughout the figures, the remaining time on the active timer session continues to count down until the timer ends (e.g., as shown in FIG. 5AY) or is stopped.
[0184] In some embodiments, device 100 detects user input 548 to enter an application switching view that displays multiple representations of applications corresponding to applications currently running, e.g., in the background, and / or recently opened applications on device 100. In some embodiments, user input 548 is a swipe gesture in a first direction (e.g., upward, downward, left-to-right, or right-to-left) optionally initiated at an edge of the display. In some embodiments, in response to user input 548, application switching user interface 550 is displayed, as shown in FIG. 5W. In some embodiments, in application switching user interface 550, the background of application switching user interface 550 is hidden (e.g., blurred, darkened, or otherwise not visually enhanced).
[0185] As shown in FIG. 5W, in some embodiments, the session region 502-7 is optionally maintained while displaying the application switching user interface 550, and continues to update as indicated by the amount of time it remains dropped (e.g., at "14:38" in FIG. 5W). In some embodiments, the session region 502-7 is not displayed while displaying the application switching user interface. In some embodiments, the application switching user interface allows the user to select an application to display (e.g., by tapping on a representation of an individual application) and / or navigate among other open applications (e.g., by swiping across an open representation of an application to view additional representations of the application). In some embodiments, a border 552 is displayed around the edge of the session region 502-7 (sometimes referred to as a key line) while the background of the area behind the session region 502-7 is visually de-emphasized. In some embodiments, the border 552 is displayed in a color selected according to the application associated with the current session. For example, for a timer session, the border 552 is orange (e.g., for another application session such as a phone call, the border may be green, and / or for a map navigation session, the border 552 may be blue). In some embodiments, the color of the border 552 is selected according to the content being updated in the session area. For example, for a media playback session, the color of the border 552 is selected based on the color associated with the currently playing (or currently selected, if playback is paused or has not yet started) media item. In some embodiments, the color associated with the currently playing media item includes a color sampled from the cover art of the media item (e.g., from the album cover of the current song or from the book cover of the current audiobook).Thus, for the same application having an active session represented in the session area, the boundary 552 of the session area optionally changes according to the content currently being presented or updated by the application within the session area.
[0186] In some embodiments, the border 552 is present in all views of the session region while the session region is associated with an active session. For example, in some situations, the border 552 is present but displayed in a color similar to the color of the user interface background surrounding the session region. Thus, in some user interfaces, the border 552 is not prominently displayed to the user (e.g., the border blends into the background).
[0187] In some embodiments, the border 552 enhances the display of the session region 502-7 such that the session region 502-7 is depicted as distinct from the background (e.g., even if the fill color of the session region is the same as the fill color of the surrounding user interface immediately outside the session region). In some embodiments, the border 552 is displayed to distinguish the session region while the session region is expanded (e.g., includes state information for one or more applications) and is not displayed while the session region is minimized (e.g., such that the session region optionally blends into the background). For example, as described in more detail below, in dark mode and / or while displaying a user interface with a dark background having a color similar to that of the session region, the session region is displayed with a more prominent key line.
[0188] In some embodiments, device 100 detects user input 554 exiting application switching user interface 550, such as by a tap input (or other selection input directed) on a background portion of application switching user interface 550 that is not associated with a representation of an application. In response to user input 554, device 100 updates the display to return to a system user interface (e.g., a home screen user interface, optionally a second page 501-2 of the same home screen as in FIG. 5V, immediately prior to navigating to application switching user interface 550), as shown in FIG. 5X. In some embodiments, session area 502-7 remains displayed with an updated remaining time for the active timer session (e.g., further reduced to "14:34" in FIG. 5X). In some embodiments, the user interacts with the home screen user interface 501-2, for example via user input 556 (e.g., a swipe or other navigation input) of FIG. 5X to navigate (e.g., from left to right) to a first view of the home screen user interface 501 as shown in FIG. 5Y, and the session region 502-7 continues to update as indicated by the time remaining further decreasing in FIG. 5Y (e.g., to “14:28”).
[0189] 5Y illustrates multiple separate inputs that can be directed to different user interface objects displayed in the home screen user interface 501. In some embodiments, each of the inputs corresponds to a request to open an application associated with a respective application icon selected by the input. For example, user input 558 on icon 424 for a messaging application opens a user interface for the messaging application (e.g., user interface 500), user input 560 on icon 426 for a calendar application opens a user interface for the calendar application, user input 562 on icon 446 for a settings application opens a user interface having a menu for changing multiple settings of device 100, and user input 564 on (or directed at) icon 422 for a music application opens a user interface for the music application. In some embodiments, as the user navigates to different user interfaces for different applications, session area 502-7 continues to be displayed and updated with the amount of time remaining in the current timer session.
[0190] In some embodiments, as described in more detail below, a user can initiate additional sessions by interacting with individual applications, and the sessions area is updated to reflect two or more currently active sessions simultaneously. For example, as described above, user input 564 on icon 422 for the music application causes device 100 to display user interface 566 for the music application, as shown in FIG. 5Z.
[0191] FIG. 5Z illustrates a user input 568, such as a tap input, requesting to begin playing a media content item (e.g., by selecting a “Listen Now” button). In some embodiments, the music application user interface 566 updates to user interface 570 to display information about the currently playing media content item, as shown in FIG. 5AA. In some embodiments, the currently playing media content item (e.g., a song, an audiobook, and / or a video), or general media content playback from the music application, corresponds to an active session, and thus, when the user navigates away from viewing the active media playback session in the music application user interface (e.g., navigates away from user interface 570), the session information about the active session moves to session area 502-7. In some embodiments, the request to navigate away from user interface 570 is a request, via user input 572, to display another application user interface, such as a mail application user interface 574. In some embodiments, user input 572 is a swipe gesture along a discrete portion (e.g., a bottom edge or a side) of the display.
[0192] FIG. 5AB illustrates that the session area displays information for at least one active session while displaying a user interface for an application, such as a mail application user interface 574, that does not correspond to an active session or is distinct from at least one other application that has an active session. In the example of FIG. 5AB, the session area displays information about two currently active sessions, including an active timer session (initiated as shown in FIGS. 5R-5S) and an active media playback session (initiated, for example, while displaying a music application as shown in FIGS. 5Z-5AB). In some embodiments, while two or more sessions are active simultaneously, the session area 502-7 separates into two portions, a first portion for the timer session area 502-8 and a second portion for the media playback session area 502-9, optionally with each portion being a different size and / or at least one portion including one or more sensor cutouts. For example, to fit information for both sessions simultaneously on the display, the timer session information is shifted to the right of the sensor and the media playback session information is in a separate user interface element to the right of the sensor. In some embodiments, at least one of the portions of the session area includes one or more sensors. For example, in FIG. 5AB, a first portion of session area 502-8 encompasses one or more sensor cutouts. In some embodiments, a second portion of session area 502-9 optionally encompasses one or more sensor cutouts. In some embodiments, one sensor cutout is encompassed by the first portion of session area 502-8 and the other sensor cutout is encompassed by the second portion of session area 502-9. In some embodiments, for two active sessions, the session area is divided into three portions, and information about the currently active session is displayed in separate areas for each active session, one to the right and one to the left of the third portion of the session area that encompasses one or more sensor cutouts.For example, a third portion of the session area that encompasses one or more sensor cutouts (eg, a central portion of the session area between the left and right active session portions) does not display active session information.
[0193] In some embodiments, while a session is active in the session area, the session area is displayed with an animation that periodically expands and contracts the size of the session area. In some embodiments, the session area optionally periodically expands and contracts while the session area is displayed as a condensed version of the session area and / or while the session area is displayed as an expanded version of the session area (e.g., one or more activatable control options displayed within an expanded session area, such as session area 502-3 and session area 502-6). In some embodiments, as shown in FIG. 5AB, the dotted lines indicate the outer limits of the animation in which the size of the individual corresponding portions of the session area increases before shrinking back to the size of the individual portions as shown in FIG. 5AB. In some embodiments, the animation of expanding and contracting (also referred to herein as vibrating) the individual portions of the session area is repeated periodically (e.g., every 0.01, 0.05, 0.1, 0.2, 0.5, 1, 2, or 5 seconds) while the session is active. In some embodiments, the periodic vibration animation expands the background (e.g., background color) of a separate portion of the session area without displaying additional information in the expansion area, and optionally without maintaining the information the same size while the background expands. In some embodiments, there are more than one portion of the session area, but the session area portions optionally expand and contract at different times, at different rates, and / or according to different vibration patterns. For example, as shown in FIG. 5AC, a first portion of session area 502-8 expands to an expanded size (e.g., the size shown by the dashed line in FIG. 5AB). In some embodiments, while the first portion of session area 502-8 is expanded, a second portion of session area 502-9, which corresponds to a different session (e.g., a media playback session), is not expanded. In some embodiments, when the first portion of session area 502-8 shrinks to its size shown in FIG. 5AB, the second portion of session area 502-9 expands to its expanded size shown by the dashed line in FIG. 5AB-FIG. 5AC.In some embodiments, the first portion of session area 502-8 and the second portion of session area 502-9, and optionally additional portions of the session area, periodically expand and contract at different times. For example, the session areas resize in a time-staggered manner, as shown in FIGS. 5AE-5AF, where the first portion of session area 502-8 is expanded when the second portion of session area 502-9 is not expanded, and vice versa. In some embodiments, each portion of the session area is animated according to the same or different periods. For example, the first portion of the session area completes an expanding and contracting animation every 0.5 seconds, and the second portion of the session area completes an expanding and contracting animation every 0.6 seconds. In some embodiments, the animations of different portions of the session area have the same phase or are out of phase (e.g., for two animations of the same or approximately the same period, one may start earlier than the other).
[0194] In some embodiments, the vibration rate of the session area is based on the individual application with which the session area is associated. In some embodiments, the session area is associated with two or more different applications represented in separate portions of the session area, but the vibration rate of the individual portions is based on the individual applications associated with the portions. For example, the vibration rate is based on the speed of animation of user interface elements displayed in the session area of the individual applications. In some embodiments, the vibration rate is based on the urgency of the session. For example, for alerts that are time sensitive or otherwise considered urgent, such as an alert for an incoming communication session or an alert for navigation instructions for a map session, the vibration rate is optionally faster than the vibration rate for non-urgent alerts, such as an ongoing voice recording session. In some embodiments, a first type of session associated with a first application causes the session area to vibrate (e.g., ongoing communication sessions, timers, and / or mapping sessions) and a second type of session associated with a second application does not cause the session area to vibrate (e.g., low battery alerts and / or silent mode alerts).
[0195] In some embodiments, the vibration speed while the session area is expanded is different from the vibration speed while the session area is condensed for an active session. For example, in some embodiments, both the condensed session area and the expanded session area vibrate. In some embodiments, the expanded session area has a slower vibration speed than the condensed session area. In some embodiments, the expanded session area optionally does not vibrate, while the condensed session area vibrates. In some embodiments, the session area does not vibrate pursuant to a determination that the session area is minimized (e.g., as shown in FIG. 5A, there is no active session displayed in the session area or the status information displayed in the session area fits into a minimized status area, such as between the sensor areas). In some embodiments, the session area vibrates if the session area is minimized but includes status information that fits into the minimized status area (e.g., a privacy indicator or a lock indicator displayed between the sensor areas).
[0196] In some embodiments, in response to a user input 576 (FIG. 5AC), such as a tap input or long press gesture, detected in the first portion of session area 502-8, the first portion of session area 502-8 expands to a session area 502-10 that includes additional information about the active session, including one or more controls of the active session, as shown in FIG. 5AD. For example, session area 502-10 provides a control option for pausing a timer and a control option for aborting a timer that ends the active timer session. In some embodiments, in response to a user input on a respective control (e.g., a tap input or other selection input), an action corresponding to the respective control is performed. In some embodiments, an animation is provided to display session area 502-10 gradually expanding from the first portion of session area 502-8 through multiple intermediate display states between displaying session area 502-8 and displaying session area 502-10. In some embodiments, the size of the expanded session area 502-10 covers the portion of the display where the second portion of session area 502-9 is displayed. In some embodiments, the second portion of session area 502-9 ceases to display until the optionally expanded session area 502-10 shrinks to the first portion of session area 502-8. In some embodiments, as shown in Figures 5AD-5AE, if no user input is detected in session area 502-10, session area 502-10 automatically reverts to the size of the first portion of session area 502-8 after a predetermined period of time (e.g., 1, 3, 5, 15, 30, or 60 seconds).In some embodiments, the session area 502-10 reverts to the size of the first portion of the session area 502-8 in response to detecting a user input outside the session area 502-10 (which in some embodiments is treated not only as a user input to exit the expanded session area 502-10 and return to the condensed session area 502-8, but also as a user input that interacts with a corresponding portion of a user interface displayed outside the session area (e.g., the mail application user interface 574 in FIG. 5AD)).
[0197] 5AD further illustrates user input 578 corresponding to a request to display a user interface of another application. In some embodiments, user input 578 is a drag gesture, a swipe gesture, a tap input, or another user input along a bottom edge to navigate between user interfaces on device 100. In response to user input 578, device 100 displays user interface 542 for a web browser application. As discussed above and further illustrated by FIGS. 5AE-5AF, in some embodiments, after changing the display of the user interface displayed outside of the session area, the first portion of session area 502-8 and the second portion of session area 502-9 continue to be displayed. For example, Figure 5AE illustrates a user input 580 such as a swipe up (or other home navigation input such as a button press) to return to view of home screen user interface 501-2 (e.g., from the scenario of Figure 5Y, where the user swipes left to switch from home screen user interface 501-1 to home screen user interface 501-2, as shown in Figure 5X, and then provides user input 564 on music application icon 422 to open music application user interface 566, as shown in Figure 5Z). In some embodiments, while displaying home screen user interface 501-2 in Figure 5AF, a first portion of session area 502-8 and a second portion of session area 502-9 continue to be displayed, animating each portion of the session area to periodically expand and contract.
[0198] FIG. 5AF illustrates a user input 582 selecting an application icon to open a voice memo application. In response to the user input 582, the device displays a user interface 584 (FIG. 5AG) for recording a voice memo in the voice memo application. In some embodiments, the user interface 584 includes a dark (e.g., black, dark gray, or dark blue) background with a lighter and / or brighter emissive color (relative to the background) of foreground elements such as text, buttons, and status indicators such as time and battery indicators. In some embodiments, in accordance with portions of the display surrounding portions of the session area including a first portion of session area 502-8 and a second portion of session area 502-9 having a color that meets a threshold amount of luminance (e.g., has less than a threshold amount of luminance), one or more visual characteristics of a respective border of each of the respective portions of the session area are changed (e.g., compared to when the portions of the display surrounding the session area portions have a luminance that exceeds a threshold amount), optionally without changing the content displayed within the session area (e.g., the inner portions of the session area are maintained in a color that matches the color of one or more sensors).
[0199] For example, when the user interface is displayed with dark user interface elements surrounding session regions 502-8 and 502-9 and / or when device 100 is in a dark mode, border 585c of session region 502-8 and / or border 585d of session region 502-9 are displayed with a more prominent visual effect than when the user interface is displayed with light user interface elements surrounding session region 502 (e.g., including portions 502-8 and 502-9) and / or when the device is in a light mode. For example, while the user interface is displayed with a light background, border 585a of session region 502-8 and border 585b of session region 502-9 are displayed with a less prominent visual effect than while the user interface is displayed with a dark background, the visual effect including a thinner, more transparent and / or dimmer border.
[0200] In some embodiments, the border of each of session regions 502-8 and 502-9 is displayed or updated according to the current size of the session region. For example, while the session region is minimized, the border is optionally displayed without a distinct color and / or contrast (e.g., instead, the border is displayed as a small number of pixels having a color that matches the color of the session region background to show sharp edges), and while the session region is expanded, the border is optionally displayed with a colored and / or more prominent key line.
[0201] In some embodiments, the boundaries of the individual session areas change over time, for example the boundaries of the session areas are animated to periodically increase and / or decrease in thickness, optionally according to the vibration speed of the session areas.
[0202] FIG. 5AG shows a user input 586 selecting a button to start recording a new voice memo. In response, as shown in FIG. 5AH, recording of a new voice memo is started with an updated voice memo user interface 584 and a privacy indicator 518 displayed in the corner of the display instead of the battery indicator 506. The privacy indicator 518 is displayed to indicate current or recent use of the camera, microphone, and / or position sensor to capture information about the device or the device's environment. In the example of FIG. 5AH, the privacy indicator 518 indicates at least the current use of the microphone to record a voice memo. In some embodiments, in a situation where the session area is empty while a voice recording is in progress, the privacy indicator 518 is displayed between the sensor areas in the session area. In some such situations, according to some embodiments, no border is displayed around the session area (e.g., because a session area with only the privacy indicator 518 displayed is not considered to be empty), as shown, for example, by the session area 5041 (FIG. 5BL). On the other hand, according to some embodiments, a border is displayed (e.g., because a session area with only the privacy indicator 518 displayed is not considered to be empty). In some embodiments, while the voice memo is being recorded, the user navigates to another user interface and requests to minimize and / or close the voice memo application user interface 584. For example, user input 588 in Figure 5AH corresponds to a swipe up gesture to return to the home screen user interface.
[0203] In some embodiments, in response to user input 588, device 100 displays home screen user interface 501. Note that while FIG. 5AI and other figures below show home screen user interface 501, in some embodiments device 100 instead returns to home screen user interface 501-2 based on home screen user interface 501-2 being the last home screen user interface displayed (e.g., just before the voice memo application is displayed, as in FIG. 5AF). In some embodiments, the session area displays an indication of up to two active sessions simultaneously. For example, after a user starts a voice recording, there are three active sessions in progress on device 100: a timer session, a media playback session, and a voice recording session. In some embodiments, the applications associated with the active sessions are ranked according to their relative priority. In some embodiments, the priority is determined automatically by device 100, e.g., a communication session and a map navigation session are prioritized higher than a timer session. In some embodiments, a user can change the priority by selecting which application sessions take higher priority over other application sessions (e.g., via a settings application, a settings file, assigning a high priority to an individual application, assigning an individual application to one of a number of predefined priority levels, or other methods of implementing user-defined priorities). In some embodiments, if the session area displays only a maximum of two active sessions at a time, the top two application sessions with the highest priority are displayed in the session area (e.g., as two separate parts of the session area). For example, in FIG. 5AI, voice recording session 502-10 has a higher priority than timer session 502-8 (e.g., media playback session 502-9 has a higher priority than timer session 502-8).), thus the portion of the session area that displayed timer session 502-8 is no longer displayed, and instead the session area displays information for audio recording session 502-10 (e.g., along with media playback session 502-9). Audio recording session area 502-10 also displays privacy indicator 518 (e.g., due to the microphone being used for voice notes being recorded).
[0204] In some embodiments where the assignment of active sessions to separate portions of the session area is for a maximum of two active sessions, a third newly displayed session is substituted for the lowest priority session based on priority (e.g., higher priority sessions are displayed in the left session area portion and lower priority sessions are displayed in the right session area portion), regardless of whether the newly displayed session is of higher or lower priority than the other displayed sessions. For example, if media playback session 502-9 is the lowest priority session and voice recording session 502-10 is the highest priority session, then voice recording session 502-10 is substituted for media playback session 502-9 in the right portion of the session area while timer session 502-8 continues to be displayed in the left portion of the session area (e.g., even though voice recording session 502-10 is of higher priority than timer session 502-8), to reduce the amount of shuffling in the session area.
[0205] In some embodiments, as shown in FIG. 5AJ, device 100 is capable of simultaneously displaying up to three active sessions in the session area. In some embodiments, the portions of the session area are smaller while the additional session portions are displayed (e.g., the portion when two sessions are displayed is smaller than when one session is displayed, and optionally, the portion when three sessions are displayed is smaller than when two sessions are displayed). For example, FIG. 5AJ shows three portions of the session area that display information for each of three active sessions: a timer session 502-13, a media playback session 502-12, and an audio recording session 502-11. In some embodiments, the arrangement of the three portions is based on session priority (e.g., the highest priority application session is on the left and the lowest priority application session is on the right). In some embodiments, the arrangement of the three portions is based on the order in which the sessions became active (e.g., the application with the longest ongoing session is displayed farthest to the right or left). In some embodiments, the placement of the three portions of the session area is determined to minimize displacement of application sessions that were already displayed before the third session became active (e.g., a media playback session is maintained at the right edge because the media playback session was previously displayed there, and / or a timer session is maintained in the left edge or center portion because the timer session was previously displayed there).
[0206] In some embodiments, one or more status indicators are optionally removed pursuant to a determination that the session region and / or portions of the session region take up more space (e.g., horizontally) as additional active sessions are added. For example, the session region may cease displaying a time and / or battery indicator because it requires additional space on the display, such that the session region may extend near the edge of the display (e.g., in portrait mode, it extends closer to the vertical edge while taking up more space along the horizontal edge). For example, as shown in FIG. 5S, the cellular network indicator 503 (FIG. 5R) is obscured (e.g., at least temporarily ceases display) due to the addition of a timer session that extends the session region from session region 502-1 to session region 502-7. In another example, as shown in FIG. 5AB, the Wi-Fi indicator 505 (FIG. 5AA) is obscured due to the further addition of a media playback session 502-9 to the session region. Additionally, whereas in Fig. 5AI the privacy indicator 518 is displayed in the voice memo session area 502-10, in Fig. 5AJ the privacy indicator 518 is displayed in a corner of the display (e.g., the upper right corner) outside the voice recording session area 502-11 (e.g., due to insufficient space within the voice recording session area 502-11 and / or because the privacy indicator 518 is not associated with the media playback session area 502-12 and therefore is not displayed between the sensor areas within the media playback session area 502-12) and replaces the battery indicator 506 (e.g., due to insufficient space at the top of the display outside the session area portions 502 and / or because, in some embodiments, the privacy indicator 518 is associated with two or more displayed session area portions). In some embodiments, adding a third active session (e.g., a voice recording session) to the status area causes one or more additional status indicators (e.g., the current time and / or the battery indicator) to be hidden, as shown in Fig. 5AJ.In some embodiments, as shown in FIG. 5AK, where the session area 502-14 of the current media playback session expands to occupy all or substantially all of the width of the display, when the session area is expanded further (e.g., to include one or more activatable control options), all status indicators, including the time, battery indicator, and privacy indicator 518, are no longer displayed.
[0207] In some embodiments, as shown in FIG. 5AK, the session area automatically extends (e.g., downward and / or outward, such as for a session area that is approximately centered at the top of the display) from the sensor cutout in accordance with an event that occurs in one of the active sessions, optionally without receiving one or more user inputs directed to the session area. For example, a media content item that was playing in a media playback session (indicated by the "A" cover art in session area 502-9 of FIG. 5AI and session area 502-12 of FIG. 5AJ) completes playback and a next media content item (e.g., "song B") begins playing. In some embodiments, an event that automatically initiates playback of a different media content item automatically extends the session area to display additional information about the session that corresponds to the event. For example, session area 502-14 includes additional information about the playback of "song B" and includes control options for controlling the session (e.g., skip back, pause, and skip forward). In some embodiments, the display of the extended session area 502-14 is animated as a transition from session area 502-8 extending outward.
[0208] In some embodiments, after a predetermined amount of time has elapsed or after detecting a user input outside of the session area 502-14, the expanded session area 502-14 automatically shrinks to the session area 502-9 shown in FIG. 5AL. In some embodiments, as the expanded session area shrinks to a smaller size, the content displayed in the expanded session (e.g., session information and / or controls) gradually fades, optionally while shrinking, at a rate selected such that the content disappears before the content is displayed overlapping the sensor. In some embodiments, the content displayed in the session area 502-14 is positioned within the session area 502-14 to avoid overlap with one or more sensor cutouts. More generally, for a session area that encompasses one or more sensor cutouts, active session content displayed in the session area is positioned within the session area to avoid overlap with one or more sensor cutouts.
[0209] As shown, session area 502-9 is updated to include current playback information regarding the media playback session (e.g., the "B" cover art associated with "Song B" is displayed in session area 502-9 in place of the "A" cover art associated with "Song A" displayed in session area 502-8 of FIG. 5AI and session area 502-9 of FIG. 5AJ). Session area 502-10 also continues to be updated with current information regarding the active session, e.g., session area 502-10 indicates that audio recording is in progress for 0:36 seconds in FIG. 5AL, increasing from 0:32 seconds in FIG. 5AI.
[0210] FIG. 5AL illustrates a user input 590 directed to the session area 502-9. In some embodiments, depending on the type of user input 590, the device 100 performs a different set of actions. For example, as shown in FIG. 5AM, in response to a first type of user input, such as a long press user input in which the user maintains contact with the display for a threshold time, the session area 502-9 expands to display additional information about the session in the session area 502-14 without opening an application associated with the session, or to display a user interface outside the session area of an application associated with the session, and the expanded session area 502-14 includes one or more selectable controls for performing respective actions associated with media playback in response to user input directed to the respective selectable controls (e.g., to play, pause, skip back, skip forward, and / or scrub the currently selected media). In some embodiments, the session area 502-9 expands without providing tactile and / or audio output. In some embodiments, as shown in FIG. 5AN, in response to a second type of input, such as a tap input, an application user interface 570 of an application associated with session area 502-9 is displayed without displaying optionally expanded session area 502-14 (e.g., without displaying the user interface shown in FIG. 5AM) (e.g., in response to second type user input 590 in FIG. 5AL, device 100 transitions directly from FIG. 5AL to FIG. 5AN).
[0211] In some embodiments, a second user input 591 (e.g., a tap input or other selection input) detected in the expanded version of the session area 502-14 of FIG. 5AM causes the device to open an application user interface 570 of a media player application (FIG. 5AN) from the expanded session area 502-14 (e.g., device 100 is configured to transition from intermediate FIG. 5AM to FIG. 5AN in response to user input 591 after a first type of user input 590 of FIG. 5AL causes device 100 to transition from FIG. 5AL to intermediate FIG. 5AM).
[0212] In some embodiments, the session area stops displaying information about the media player application in the session area, while the media player application user interface 570 is displayed outside the session area. For example, the session information does not need to be displayed in the session area either, since it is displayed in the application user interface. In some embodiments, while displaying the media player application user interface 570 of FIG. 5AN, the session area for the active timer session is redisplayed while continuing to display the session area 502-10 for the active voice recording. For example, because device 100 displays a maximum of two active sessions simultaneously, the device replaces the display of the media player session with a display of the timer session 502-15 in the session area.
[0213] FIG. 5AN further displays a user input 592 that selects the session area 502-10 of the active voice recording session. In some embodiments, pursuant to a determination that the user input is a second type of user input (e.g., a tap input or other selection input), in response to the user input 592, an application user interface 584 of the voice memory application is displayed, as shown in FIG. 5AO. In some embodiments, while the user interface 584 of the voice memory application is displayed, the session area stops displaying session information for the voice recording (e.g., in FIG. 5AO, the voice memo session area 502-10 is no longer displayed). In some embodiments, the device 100 optionally redisplays the media player session in the second portion of the session area 502-9 and moves the display of the active timer session area to the first portion of the session area 502-8 (e.g., pursuant to a determination that the active timer session was started before the media player session was started).
[0214] FIG 5AO illustrates a user input 594, such as a tap user input, selecting a "Stop" button in the voice memory application user interface 584. In some embodiments, following a user input requesting to stop the voice recording, as shown in FIG 5AP where the stopped recording is listed as "Recording 2", the voice recording session is no longer active, the "Stop" button is reverted to a "Record" button for starting a new recording, and the privacy indicator 518 is no longer displayed, instead redisplaying the battery indicator 506 (or another currently active, system-determined, or user-defined status indicator). Thus, if the user navigates away from the voice memory application user interface while no voice recording is in progress, there is no active session of the voice memory application and the session area does not display voice memory application information (e.g., as in FIG 5AQ and FIG 5AW) until, optionally, the next active session of the voice memory application is started (e.g., another voice recording is started).
[0215] 5AP displays user input 596 detected while displaying user interface 584 for an audio memory application. In some embodiments, user input 596 is a swipe gesture, a drag gesture, a press and hold gesture, or another gesture to navigate to a control user interface 598. In some embodiments, in response to user input 596, device 100 displays control user interface 598, as shown in FIG. 5AQ.
[0216] In some embodiments, the control user interface 598 allows a user to change one or more settings of the device 100 and / or quickly access one or more shortcuts for applications stored on the device 100. For example, the user can change volume and / or brightness settings (e.g., using one or more slider bars), turn on or off Bluetooth, Wi-Fi, network connectivity, flight mode, and select and / or deselect icons for turning on the flashlight, opening a camera application, opening a calculator application, and / or opening a timer application. In some embodiments, the control user interface 598 is customizable by the user such that the user can select which settings and / or shortcuts for applications to include in the control user interface 598.
[0217] In some embodiments, as shown in FIG. 5AQ, device 100 is configured to receive one or more user inputs, such as user input 5000 and / or user input 5002. In some embodiments, user input 5000 is a user input (e.g., a long press, tap, or other selection user input) on a physical or simulated button, optionally on the side of the display of device 100. In some embodiments, user input 5002 is a drag or swipe gesture initiated at the top edge of the display of the device. In some embodiments, each of user input 5000 and user input 5002 causes device 100 to display a wake screen user interface 5004 and / or a cover sheet user interface. For example, wake screen user interface 5004 is a user interface that is displayed after entering a low power state in which the display of device 100 is at least partially off, in response to an input prompting device 100 to exit the low power state. In some embodiments, in the low power state, the display optionally displays an "always on" indicator of the time and / or date. For example, in response to user input 5000 in FIG. 5AQ, device 100 enters a low power state (e.g., the display is turned off), and FIG. 5AR1 represents the user interface that is displayed when the device is prompted to exit the low power state.
[0218] In some embodiments, in response to user input 5000 of FIG. 5AQ, the device enters a locked state in which a password, passcode and / or biometric authentication is required to unlock the device, and the device has limited functionality in the locked state and must be unlocked before accessing the respective applications and / or data stored on the device 100. In some embodiments, the wake screen user interface is displayed regardless of whether the device is in a locked state or has already been unlocked (e.g., the wake screen user interface is displayed when the user boots the device before accessing the home screen user interface and / or other application user interfaces). In some embodiments, the cover sheet user interface includes the same characteristics as the wake screen user interface, but the user can optionally view the cover sheet user interface without entering the locked state and / or low power state of the device. In some embodiments, optionally in response to user input (e.g., a swipe gesture upwards in the center of the display such as user input 5010 of FIG. 5AQ, or another gesture), one or more alerts (e.g., system alerts and / or notifications) are displayed in the wake screen user interface and / or the cover sheet user interface.
[0219] FIG. 5AR1 illustrates that for one or more applications having an active session, in some embodiments, the active session is not displayed in the session area including the one or more sensor cutouts. For example, in FIG. 5AR1, the media playback session 502-9 is no longer displayed in the session area, but instead displayed as a separate user interface element 5006 on the wake screen user interface 5004. In some embodiments, the session is animated into and / or out of the user interface element 5006 from the session area. In some embodiments, the user interface element 5006 includes information about the currently playing media item (e.g., the title, artist, and / or cover art associated with the media item) and / or a number of controls (e.g., skip back, pause and / or play, skip forward, and optionally a volume control slider) for controlling the media playback session, which are actuable by user input to perform a respective action related to the media playback. In some embodiments, the user interface element 5006 is displayed in a predetermined portion of the display area (e.g., below the date, or at the bottom of the display area above the flashlight and / or camera icons). In some embodiments, the one or more applications continue to be represented in the session area (e.g., active timer session 502-16, optionally because wake screen user interface 5004 is not configured to display a separate user interface element for the timer application).
[0220] 5AR2-5AR3 show examples of moving two or more sessions from the session area to the cover sheet user interface 5004a. In some embodiments, in response to a user input 5002b requesting display of the cover sheet user interface 5004a while there are three active sessions, including a media playback session 502-9a, a timer session 502-8a, and a sports session 599, the device 100 displays the cover sheet user interface 5004a (FIG. 5AR3), and two or more of the active sessions are displayed outside the session area as user interface elements, including a user interface element 5006 (e.g., for the media playback session) and a user interface element 5007 (e.g., for the sports session). In some embodiments, one or more sessions optionally remain displayed in the session area (e.g., the timer session 502-16a).
[0221] In some embodiments, even when the device enters a low power state, the device maintains current information of active sessions. For example, upon exiting the low power state, displaying the wake screen user interface 5004 includes displaying the session area 502-16 for the active timer session (e.g., when the time remaining is 01:21). In some embodiments, a lock indicator 5008 is optionally displayed between the sensor cutouts of the session area to indicate that the device is in a locked state. In some embodiments, the lock indicator is displayed on the display outside of the session area (e.g., above the time "9:41" displayed on the wake screen user interface 5004).
[0222] FIG. 5AR1 illustrates a user input 5010 for requesting to view one or more alerts, which is detected in a wake screen user interface 5004. In some embodiments, the user input 5010 is a swipe gesture in a first direction (e.g., up, down, right, or left) within a predefined area of the wake screen user interface 5004 (e.g., away from an edge of the display). In some embodiments, in response to the user input 5010, the device 100 displays multiple alerts (e.g., system alerts and / or notifications associated with respective applications) received by or occurring on the device 100, as shown in FIG. 5AS. In some embodiments, while a notification view is displayed in the wake screen user interface, user interface elements displayed in the wake screen user interface 5004 appear to scroll up and down the display. For example, as shown in the wake screen user interface 5004-1 of FIG. 5AS, the user interface element 5006 has shifted upward in the user interface.
[0223] 5AS further illustrates that pursuant to a determination that device 100 has been unlocked (e.g., using a passcode, fingerprint recognition, facial recognition, iris recognition, or another biometric or non-biometric authentication), a lock indicator 5008 displayed between the sensor cutouts is updated to an unlocked appearance to indicate that device 100 is in an unlocked state. In some embodiments, the session area 502-16 continues to update as the timer session counts down the remaining time on the timer.
[0224] In some embodiments, a user can interact with multiple alerts, including opening applications associated with individual alerts and / or dismissing and / or silencing individual alerts. For example, FIG. 5AS shows a user input 5012, such as a right-to-left swipe gesture on a notification from a message application, and a subsequent user input 5014 selects an option to clear the notification, as shown in FIG. 5AT. Thus, in FIG. 5AU, the notification from the message application has been removed from the notification view, and the next notification in the list of alerts (e.g., for a news article) is displayed in place of the removed message application notification. In some embodiments, a user can scroll to view additional alerts. For example, user input 5016 in FIG. 5AU is an upward swipe gesture, and in response to user input 5016, additional notifications are displayed in user interface 5004-1, as shown in FIG. 5AV. According to some embodiments, session area 502-16 continues to update as the timer session counts down the remaining time on the timer through various interactions with the wake screen user interface described with respect to FIGS. 5AR1-5AU.
[0225] In some embodiments, as the user scrolls through the additional notifications, user interface element 5006 of the media playback session is scrolled out of view on the display, as in Figure 5AV. In some embodiments, pursuant to a determination that user interface element 5006 is no longer displayed in user interface 5004-1, the media playback session is displayed (e.g., re-displayed) in session area 502-9 (e.g., a second portion of the session area), optionally concurrently with another active session (e.g., active timer session 502-16b) in the first portion of the session area.
[0226] 5AV shows a user input 5018 requesting that the wake screen user interface 5004 be dismissed and the home screen user interface 501 be displayed. In some embodiments, the user input 5018 is a swipe up that is initiated in a predetermined area of the display near the bottom edge. In some embodiments, as the user input 5018 progresses, the wake screen user interface 5004 is displayed as shifting on the display in the same direction as the user input 5018 (e.g., the wake screen user interface 5004 is displayed as sliding up and off the display).
[0227] In some embodiments, after navigating away from the wake screen user interface 5004, the lock indicator 5008 is no longer displayed in the session area. In some embodiments, the lock indicator 5008 is displayed in an unlocked appearance (e.g., as an open lock) for a predetermined time (e.g., from the time the device 100 is unlocked) and then automatically stops displaying depending on the passage of the predetermined time. In some embodiments, the active session continues to be displayed in the session area (e.g., in the first and second portions of the session area) as the user navigates between the wake screen user interface 5004 and other system user interfaces (e.g., the home screen user interface 501) and / or application user interfaces for applications running on the device 100. In some embodiments, the user input 5020 is an input activating a physical or virtual button on the device side to put the device 100 into a low power state.
[0228] In some embodiments, device 100 is taken out of the low power state in response to a user input (e.g., a user touches the screen, a user picks up the device, and / or a user touches a button on the device). In some embodiments, device 100 automatically leaves the low power state in response to an event (e.g., in response to receiving an alert and / or in response to an event from an application). FIG. 5AX, similar to FIG. 5AR1, illustrates a wake screen user interface 5004 that is initially displayed in response to the device leaving the low power state (e.g., and transitioning to a wake state), with media playback session 502-9 transitioned from the session area to user interface element 5006 (e.g., a media playback widget for wake screen user interface 5004) and timer session 502-16 displayed in the session area. In some embodiments, the device automatically enters a locked mode while in the low power state.
[0229] In some embodiments, in response to a current session event, the session area automatically expands without detecting a user input on the session area. For example, in FIG. 5AY, a timer expires and session area 502-16 (FIG. 5AX) expands to session area 502-17 (FIG. 5AY). In some embodiments, device 100 optionally provides audio and / or haptic output in addition to expanding the session area in response to an ongoing session event. In some embodiments, device 100 provides audio and / or haptic output while expanding the session area in response to a determination that the session area has been expanded in response to a user input, but does not provide audio and / or haptic output (or optionally provides different audio and / or haptic output) in response to the session area expanding as a result of the session event. Examples of ongoing session events that optionally cause the device to expand the session area include when a new media item begins playing (FIG. AK), as shown in FIG. 5BG, when a timer expires (FIG. 5AY), when a new navigation command is in proximity, when a connection is made (e.g., to share a media item over a device-to-device network) or when a new media item is shared, and / or when the device is in proximity to a wireless terminal to access a payment method stored on device 100.
[0230] FIG. 5AZ shows that after the timer expires and the timer's session is no longer active, the session area is displayed as session area 502-1, optionally including displaying lock indicator 5008 as an open lock, for a predetermined period of time in accordance with having no active session and device 100 being unlocked.
[0231] 5BA1-5BA3 show examples of receiving an alert (e.g., a system alert or notification) while displaying the session areas in different states. For example, in FIG. 5BA1, an empty session area 502-18 does not contain an active session. FIG. 5BA2 shows an example of an incoming alert while there is one active session (e.g., a timer session) displayed in a condensed size in session area 502-19. FIG. 5BA3 shows an active session area 502-20 for an ongoing communication session including multiple controls for the session. In some embodiments, session areas 502-19 and / or 502-20 are outlined with a colored border, optionally with a color based on the application associated with the active session in the session area.
[0232] In some embodiments, the incoming alert (e.g., notification 5024-1, notification 5024-2, and / or notification 5024-3) is animated as it enters the display of device 100 from the top edge of the display. In some embodiments, the incoming alert is displayed sliding down from the top of the display to a lower position in the display. In some embodiments, the incoming alert is initially displayed with visual de-emphasis (e.g., blur effect, low opacity, and / or low level of brightness). In some embodiments, the incoming alert is displayed with visual characteristics that make it appear to slide behind the displayed session area. In some embodiments, the size of the alert changes as it slides onto the display (e.g., from smaller to larger) to make it appear as if the alert is approaching.
[0233] As shown in FIGS. 5BA1-5BA3, in some embodiments, the alert does not change its animation based on the size of the session region 502 (e.g., the same animation of the alert is displayed for session regions 502 of different sizes). In some embodiments, after the alert moves beyond the displayed session region such that it is no longer obscured by the session region, the alert is displayed with less visual de-emphasis (or, optionally, no visual de-emphasis) and / or the alert is increased in size (e.g., providing a visual effect simulating an alert moving towards the user after sliding behind the session region). In some embodiments, the alert is displayed below and / or near the session region and changes its final position based on the current size of the session region (e.g., alerts 5024-1 and 5024-2 are displayed at separate vertical positions due to the respective heights of session regions 502-18 and 502-19, while alert 5024-3 is displayed at a lower vertical position on the display due to the increased height of session region 5024-3).
[0234] Figure 5BB shows the final state alert 5024-1 of Figure 5BA1 displayed on the home user interface 501 and a user input 5026 (e.g., a tap input) requesting to open a map application associated with the icon 436 of the home user interface 501. In response to the user input 5026, a user interface 5028 of the map application is displayed, as shown in Figure 5BC. In some embodiments, the user interface 5028 of the map application surrounds the session area on all sides. Figure 5BC further shows a user input 5030 to initiate navigation to a destination using the map application.
[0235] 5BD shows a map application user interface 5028-1 including navigation instructions. In some embodiments, the session region 502-21 is optionally displayed with a border according to a determination that the user interface surrounding the session region meets a brightness threshold (e.g., if the content surrounding the session region is dark, a prominent border is displayed, and if the content surrounding the session region is light, a second border that is less prominent than the first border is displayed). In some embodiments, while the map application is providing navigation instructions, a user input 5032 corresponding to a request to leave the map application user interface 5028-1 is detected.
[0236] 5BE1-5BE2 illustrate an animated transition for minimizing the display of map application user interface 5028-1 and displaying home screen user interface 501 in response to user input 5032. In some embodiments, the animated transition includes shrinking map application user interface 5028-1, shrinking the content in user interface 5028-1 (optionally while maintaining the display of the content as it appears in full-sized user interface 5028-1), and moving the reduced version of user interface 5028-1 upward toward the session area. In some embodiments, user interface 5028-1 is reduced and moved upward on the display at a respective rate such that the width of reduced user interface 5028-1 matches the current width of session area 502-21 (e.g., when reduced user interface 5028-1 reaches a vertical position on the display that is at or near the vertical position of session area 502-21, as indicated by the dashed line in FIG. 5BE1). For example, the user interface 5028-1 appears to slide under and / or behind the session region 502-21 (e.g., as shown in FIG. 5BE2). In some embodiments, the animated transition includes an expansion of the session region 502-21 (e.g., the session region 502-21 is wider in FIG. 5BE2 than in FIG. 5BE1), optionally with a simultaneous contraction of the map application user interface 5028-1. In some embodiments, the home screen user interface 501 is not visually highlighted (e.g., appears blurred and / or less opaque and / or less bright) during the animated transition. In some embodiments, the user interface 5028-1 continues to move upward on the display until it contacts the top edge of the display of the device 100, which is optionally displayed above the session region 502-21.
[0237] In some embodiments, after the user interface 5028-1 is scaled down to a width that matches the width of the session area, as shown in FIG. 5BF, the device 100 stops displaying the scaled down version of the user interface 5028-1 and instead displays information about the navigation session in the session area 502-22. For example, instead of continuing to display a map with step-by-step instructions, the session area 502-22 includes an indication of the next step in the navigation (e.g., turn right at 2 miles). In some embodiments, the session area 502-22 remains for a threshold time (e.g., 0.1, 0.2, 0.5, 1, 2, 5, 15, 30, or 60 seconds) at the end of the animated transition before automatically minimizing to the session area 502-23 shown in FIG. 5BG. In some embodiments, the session area 502-23 is automatically removed from view without user input following a determination that the navigation instructions have ended (e.g., the user has reached the destination). In some embodiments, the session region 502-23 is no longer displayed pursuant to a determination that the user has requested termination of the navigation session.
[0238] FIG. 5BG illustrates a user input 5034 requesting to open a web browser application associated with icon 420 in home screen user interface 501. FIG. 5BH illustrates the user interface of the web browser application while device 100 is rotated from a portrait orientation (e.g., vertical orientation as in FIG. 5BG) to a landscape orientation (e.g., horizontal). In some embodiments, device 100 can be rotated such that a downward direction is defined relative to a user's view of the device. For example, when a user rotates the device from a vertical orientation to a horizontal orientation, in some embodiments, the display of device 100 is updated to change the orientation of text, images, and / or other content to match the device's current orientation, as described with reference to FIG. 5BK.
[0239] In some embodiments, while device 100 is in landscape orientation as in FIG. 5BH, sessions of a single application are displayed in separate portions of the session area (e.g., portions 502-24, 502-25, and 502-26). For example, in FIG. 5BH, a session for a wireless headphone connection is initiated. In some embodiments, the portions of the session area include separate user interface elements (e.g., air bubbles) that appear to animate from session area 502-24, which optionally includes one or more sensor cutouts, in a vertical direction (e.g., above and below the session). For example, as device 100 is oriented in landscape, the session area displays information about the session above and below session area 502-24 that encompasses the sensor cutouts to minimize intrusion of the session area blocking user interface elements in the user interface of a web browser application. In some embodiments, when there are two active sessions, portion 502-25 of the session area optionally displays information for a first of the two active sessions, and portion 502-26 of the session area optionally displays information for a second of the two active sessions.
[0240] FIG. 5BI illustrates that after the animation shown in FIG. 5BH, the user interface elements of the session area are maintained at a predetermined distance away from the session area, optionally as separate user interface elements. In some embodiments, at least one of the portions of the session area includes a sensor area(s) (e.g., at least one of the sensor area portions 502-25 or 502-26 extends and remains connected to the session area 502-24, rather than both being separate from the session area 502-24). In some embodiments, a user input 5036 (e.g., a tap input or other selection input) is detected at a portion of the session area (e.g., portion 502-25). In some embodiments, the portion 502-25 of the session area displays an indication of headphones currently connected and / or paired with the device 100. In some embodiments, the portion 502-26 includes an indication of the current battery level of the headphones (e.g., "82%"). In some embodiments, the session area portion 502-24 is maintained with a background color that matches the color of the sensor in the sensor cutout without displaying any information in the session area portion 502-24.
[0241] FIG 5BJ illustrates that in response to user input 5036, portion 502-26 of the session area expands to display additional information about headphones currently connected and / or paired with device 100. In some embodiments, portion 502-24 and / or portion 502-25 continue to be displayed without expanding or displaying the additional information. In some embodiments, portion 502-24 and / or portion 502-25 of the session area also expands and / or displays the additional information in response to user input 5036. FIG 5BJ further illustrates user input 5038 selecting an option to start playing a video in the web browser application.
[0242] FIG. 5BK illustrates multiple orientations of device 100 (e.g., device 100a, device 100b, device 100c, and device 100d) while displaying a playing video (e.g., movie A). In some embodiments, device 100a and device 100b are illustrated in landscape orientation. Device 100a illustrates a sensor cutout, shown as a dotted line, on the left side of device 100a. In some embodiments, while the user interface is dark, a sensor disposed in the sensor cutout is optionally not visible or is not otherwise displayed with any user interface elements to draw attention to the sensor cutout, such that the sensor blends with the user interface. In some embodiments, the user interface (e.g., or a portion of the user interface surrounding portions of session areas 502-25 and 502-26) is displayed with a dark background (e.g., having a threshold amount of luminance), while edge treatment is displayed as a border surrounding each of portions 502-25a and 502-26a of the session area. In some embodiments, portion 502-25a is displayed and portion 502-26a is not displayed. In some embodiments, portions of session regions 502-25a and 502-26a are optionally not displayed while content is being played in full screen mode. In some embodiments, pursuant to a determination that portions 502-25a and 502-26a display information about the same session (e.g., both portions display information about a headphone connected session), the color of the border of portion 502-25a is the same color as the color of the border of portion 502-26a. In some embodiments, when portions of a session region display information about different sessions, possibly associated with separate applications, the color of the border of the individual portions is selected based on the application associated with the individual session region (e.g., a timer session optionally displays an orange border, a communication session optionally displays a green border, and / or a map navigation session optionally displays a blue border).For example, in FIG. 5BN, the boundaries of the session area portion 5042-1 of the voice recording session are displayed in a first color, and the boundaries of the session area portion 5042-2 of the navigation session are displayed in a second color different from the first color.
[0243] In some embodiments, when a user rotates the device 180 degrees while device 100b is in landscape orientation such that the sensor cutout is on the right side of device 100b, portions of session area 502-25b and 502-26b continue to be displayed proximate (e.g., to the right) to the sensor cutout. In some embodiments, portions of the session area optionally change relative position. For example, portion 502-26b is displayed over portion 502-25b of device 100b, and portion 502-25a is displayed over portion 502-26a of device 100a. In some embodiments, portion 502-25b is displayed and portion 502-26b is not displayed. In some embodiments, while device 100 is in a landscape orientation (e.g., as in devices 100a and 100b of FIG. 5BK), the session area does not display information that is updated over time about an active session (e.g., a minimized session area may be displayed to minimize the appearance of one or more sensor areas, and the session area does not expand to continually display status information that is updated over time, such as timers, media playback, or other sessions described herein, although the session area may, in some embodiments, be temporarily expanded to display alerts and other session events, as shown in FIG. 5BK).
[0244] In some embodiments, following a determination that the devices (e.g., devices 100c and 100d) are in portrait orientation, the session regions are displayed as a single session region 502-24a (e.g., rather than as separate user interface elements proximate the sensor cutout). In some embodiments, the session region 502-24a optionally displays a battery icon and / or a percentage of battery life remaining to indicate the battery level of the connected headphones. In some embodiments, following a determination that the background is sufficiently dark (e.g., meets a threshold amount of luminance), a border is displayed around the session region 502-24a.
[0245] In some embodiments, when the device is rotated 180 degrees from the device 100c orientation to device 100d, session region 502-24b is displayed in a portion of the display that encompasses the sensor cutout and / or sensor of device 100. For example, session region 502-24b is displayed at the bottom of the display to encompass the sensor that was located at the bottom of the display while in the device 100d orientation.
[0246] In some embodiments, following a determination that the device is displaying content in full screen mode, for example in Figure BK, the video content is played to occupy the entire user interface area (not including the session area), and the respective borders of the respective portions of the session area are optionally not displayed, e.g., the session area is displayed inconspicuously to better blend with the content displayed in full screen mode.
[0247] FIG. 5BL illustrates an example of information displayed in session areas for multiple applications. It will be understood that the examples illustrated in FIG. 5BL represent a snapshot in time of the individual session areas, and as the state of the applications changes, state information is updated in the session areas. In some embodiments, the example session areas 5041 and 5040a-q are displayed in the session areas simultaneously with the individual user interfaces displayed outside the session areas on device 100. For example, in FIG. 5E-5F, session area 502-4 is optionally displayed with session areas 5041 or one of 5040a-q depending on the type of session, if any, that is active at the individual time. For example, FIG. 5E-5F illustrates a communication session, but any of the example session areas 5041 and 5040a-q can be displayed in place of session area 502-4 (FIG. 5E). In some embodiments, session area 5041 includes a privacy indicator (e.g., privacy indicator 518) without displaying state information about the active sessions (e.g., if there are no active sessions). In some embodiments, the session area 5040a corresponds to a communication session (e.g., a telephone call and / or video conference) including an icon of an application associated with the session (e.g., a phone icon), the length of time that the communication session has been in progress (e.g., “0:08”), optionally a privacy indicator displayed between the sensor notches to indicate that the microphone of the device 100 is currently being accessed by an application (e.g., a dot between the sensor notches in the sensor area 5040a), and an audio waveform indicating incoming and / or outgoing audio data during the communication session.
[0248] In some embodiments, session area 5040b corresponds to a timer session and includes an icon of a timer application and the time remaining on the timer. In some embodiments, session area 5040c corresponds to a media playback session and includes a representation of a currently playing media content item (e.g., cover art for the media content item) and a playback icon indicating that playback is in progress. In some embodiments, session area 5040c-2 corresponds to a media playback session (e.g., for a music application or other media playback application) that includes a representation of a currently playing media content item (e.g., cover art for the media content item) and an audio waveform indicator. In some embodiments, the audio waveform indicator is generated based on a current audio output of the currently playing media content item such that the audio waveform is updated as the audio output changes over time. In some embodiments, the audio waveform indicator is displayed in one or more colors selected based on a representation of the currently playing media content item. For example, one or more prominent colors are sampled or otherwise identified from the cover art of the media content item and the waveform is displayed in one or more prominent colors. In some embodiments, the sonic waveform indicator is superimposed on at least a portion of the covering technique, optionally on at least a portion of a blurred or visually modified version of the covering technique.
[0249] In some embodiments, the session area 5040d corresponds to an accessibility session and includes an icon for the accessibility application and an indication of a connected device (eg, a connected hearing aid) for the accessibility session.
[0250] In some embodiments, session area 5040e corresponds to a navigation session for a map application and includes an indication of the next navigation step (e.g., turn right) and an indication of when the next step will occur (e.g., in 2 miles).
[0251] In some embodiments, session area 5040f indicates a health and / or wellness session and includes a training application icon and an indication of the current workout (e.g., distance covered, amount of time of the ongoing workout, and / or calories burned in the workout).
[0252] In some embodiments, the session area 5040g indicates a session for sharing content (e.g., over device-to-device Wi-Fi), which includes a sharing icon and a progress indicator that updates as the sharing process progresses until it is complete.
[0253] In some embodiments, session area 5040h indicates a session area for an audio recording session, which includes an audio waveform indicating the audio data being recorded in the audio recording, a recording indicator, and the amount of time the audio recording is in progress. Session area 5040h optionally includes a privacy indicator indicating that the microphone of device 100 is being accessed by an application.
[0254] In some embodiments, session area 5040i indicates a session area for remotely viewing and / or sharing content from device 100 on a separate display device, including an indication that device 100 is sharing its display and, optionally, an indication of an application being displayed on the separate display device (e.g., an application for streaming video content, where the video content is being played on the separate display device).
[0255] In some embodiments, session area 5040j represents a session area for a sports application that includes an indication of the current score (eg, during an individual game) of each team participating in an active sports session.
[0256] In some embodiments, the session area 5040k shows a session area for a carpool session, including an icon representing a carpool application and the time until the carpool arrives and / or drops off the user.
[0257] In some embodiments, session area 5040l shows a session area that is displayed while the device is in locked mode and the device is authenticating a user to unlock the device, hi some embodiments, session area 5040l includes a lock icon and a progress animation that indicates that the device is in the process of authenticating a user.
[0258] In some embodiments, session area 5040m indicates a session area for biometric authentication that includes an icon representing an authentication prompt to prompt the user to perform biometric authentication (e.g., by looking at the camera and / or providing a fingerprint).
[0259] In some embodiments, session field 5040n indicates a low battery session provided by system software of device 100 pursuant to a determination that the battery of device 100 has less than a threshold amount of power remaining.
[0260] In some embodiments, session area 5040o shows a session area for connected devices (e.g., connected headphones and / or speaker(s)), including an indication of the connected device and, optionally, an indication of the battery level of the connected device.
[0261] In some embodiments, session area 5040p shows a session area for silent mode, including an indication that silent mode has been started or enabled (e.g., ringer mode has been disabled) (e.g., a bell icon with a line through it) and, optionally, a description of "silent mode." In some embodiments, session area 5040p-2 shows a session area for ringer mode, including an indication that ringer has been started or enabled (e.g., silent mode has been disabled) (e.g., a bell), optionally including animating the bell indication as a shake, and, optionally, a description of "ringer." In some embodiments, a user can toggle device 100 between silent mode and ringer mode. In some embodiments, in response to detecting that the mode has been switched between silent mode and ringer mode, session area 5040p and / or 5040p-2 shakes (e.g., repeatedly from right to left as described in the shake animations of Figures 6H-6J), optionally including shaking a bell indicator within the session area while the session area is shaking.
[0262] In some embodiments, session area 5040q shows a session area for an active hotspot session, including an icon representing at least one device currently linked via a hotspot connection, and optionally an indication of the signal strength of the hotspot connection.
[0263] In some embodiments, one or more of session areas 5040a-5040q are displayed while the corresponding session is ongoing. In some embodiments, session areas associated with individual applications and / or software are displayed for a predetermined time before automatically ceasing display of the session area. For example, silent mode session area 5040p is optionally displayed for 2 seconds or 10 seconds after device 100 enters silent mode, but is not maintained on the display while device 100 remains in silent mode.
[0264] 5BM1-5BM2 illustrate updating a session area of the same application to display two or more sets of information for the application. For example, during an active session for a sports application, two or more games active at the same time have different sets of information for the individual games (e.g., different scores and / or different teams). In some embodiments, while two or more sets of information are available, the session area rotates between displaying the two or more sets of information periodically. For example, in FIG. 5BM1, session area 5040j-1 shows the current score of GS vs. CHI, and in FIG. 5BM2, session area 5040j-2 shows the current score of LA vs. NY. In some embodiments, the games need not be of the same type (e.g., basketball, baseball, football, and / or hockey). Instead, pursuant to a determination that an individual application associated with a session area includes two or more sets of information, each set of information is displayed for a predetermined amount of time (e.g., 1, 2, 5, 10, 15, 30, 60, 90, or 120 seconds) before replacing the display with another set of information for the predetermined amount of time. In some embodiments, the set of information automatically changes from a first set of information to a second set of information according to an event occurring for the respective set. For example, while displaying the first set of information, a team scores in a game related to the second set of information, and the device 100 automatically replaces the display of the first set of information with the second set of information to indicate the event occurrence (optionally without regard to a predetermined amount of time). In some embodiments, in response to a user input (e.g., of a certain type, such as a long press) directed at the session area 5040j-1 in its condensed form as shown in FIG. 5BM1, the device 100 expands the session area to session area 5040-j3 to display additional information, such as the current play duration (e.g., a quarter or half) and / or the time remaining in the current play duration, as shown in FIG. 5BM3. A similar user input directed at the session area 5040j-2 of FIG. 5BM2 similarly results in the session area being expanded to the session area 5040-j4 of FIG. 5BM4.
[0265] 5BM3-5BM4 illustrate displaying two sets of information in the expanded session area, optionally in response to a user input 5999b. For example, a user input 5999b is detected while an expanded session area 5040j-3 including a display of a first set of information regarding a first event (e.g., a first game for a sports application) is displayed. In some embodiments, the user input 5999b is, for example, a left-to-right swipe input, and in response to the user input 5999b, the device 100 updates the display in the expanded session area to display a second set of information regarding a second event (e.g., a second game for a sports application).
[0266] FIG. 5BN illustrates an example of multiple active sessions displayed simultaneously in a session area. The session area illustrated in FIG. 5BN is an example of concurrent sessions, however, it will be understood that alternative and / or additional arrangements of active sessions may also be used depending on which application is currently the active session and / or the relative priority of the active sessions. To that end, the session area includes a portion 5042-1 for a current audio recording that encompasses a sensor cutout (e.g., a sensor area), optionally including a privacy indicator displayed between the sensor cutouts, and simultaneously displays a portion 5042-2 for a navigation session. In some embodiments, the concurrent sessions are displayed in different sizes, and the information displayed in each portion of the session area is based on the size assigned to the individual applications within the portion of the session area.
[0267] In some embodiments, device 100 displays animations while transitioning the session region from a first state to a second state, including expanding and / or minimizing the session region and / or updating the session region with information from a first session of a first type to a second session of a second type (e.g., as described with reference to FIGS. 5BT-5BU). In some embodiments, the type of animation displayed during the transition between session states is based at least in part on the current size of the session region. For example, device 100 displays a first animated transition from an expanded timer session to another session state (e.g., a condensed timer session or another session such as a music session) including animating numbers to optionally shift numbers up or down while decreasing the text size of the numbers (e.g., to represent the time remaining on the timer). In some embodiments, if the timer session is a condensed timer session (e.g., smaller than an expanded timer session), device 100 displays a second animated transition from the condensed timer session to another session state that is different from the first animated transition, including cross-fading the numbers of the condensed timer session with text and / or icons displayed in the other sessions, without shifting the numbers up or down as they change. Thus, the animated transition between sessions displayed in a session area depends on the current size (e.g., condensed or expanded) of the session area when the transition is initiated.
[0268] In some embodiments, different types of animations are used during different display states of the session area. For example, animating the numbers of a timer to shift the numbers up or down while changing the displayed value (e.g., mimicking the mechanical rotation of a clock or flipping of the numbers) is optionally performed while the session area is in an expanded state (e.g., has more space for animation), while cross-fading the numbers without shifting the numbers up or down while changing the displayed value is optionally performed while the session area is in a condensed timer session (e.g., has less space for animation).
[0269] As another example, portion 5043-1 of the session area includes a timer session and optionally includes a sensor cutout, and portion 5043-2 includes a session for biometric authentication. In some embodiments, portion 5044-1 of the session area includes an indication of a currently connected set of headphones, and portion 5044-2 includes session information for a workout session and optionally includes a sensor cutout. In some embodiments, portion 5045-1 of the session area indicates a session for silent mode and optionally includes a sensor cutout, and portion 5045-2 includes an indication of a media streaming application session, simultaneously with portion 5045-3 including an indication of a currently connected set of headphones. Thus, in some embodiments, three or more sessions are displayed simultaneously in portions of the session area, optionally with each session displayed in a separate user interface element as a separate portion of the session area.
[0270] In some embodiments, portion 5046-1 of the session area indicates an active hotspot connection session, and portion 5046-2 of the session area indicates a media playback session. In some embodiments, portion 5047-1 displays a sharing session for sharing media content with other devices (e.g., via device-to-device Wi-Fi, NFC, and / or Bluetooth), and portion 5047-2 displays a session for an audio recording. In some embodiments, a separate user interface element is displayed containing the sensor cutout, and the portion of the session area displaying information about the active session is displayed next to (e.g., to the right and / or left of) the user interface element containing the sensor cutout.
[0271] In some embodiments, portion 5048-1 of the session area displays an indication of a connected set of headphones, portion 5048-2 of the session area displays an indication of silent mode and optionally includes one or more sensor cutouts, and portion 5048-3 displays a screen recording session.
[0272] FIG. 5BO illustrates multiple examples of expanded session areas. As discussed above, in some embodiments, the session area expands in response to user input (e.g., tap input and / or long press input) directed at the session area (e.g., while minimized, e.g., as session areas 5040a-5040q). In some embodiments, the session area expands in response to events and / or updates occurring in an application associated with the session without detecting a user input (e.g., turning off a timer, changing a song, or next navigation instruction). In some embodiments, each expanded session area 5050 includes one or more controls for interacting with an application associated with the session. For example, expanded session area 5050a illustrates an expanded timer session, optionally accompanied by a control 5050a-1 for aborting (e.g., pausing) and / or a control 5050a-2 for canceling the timer. In some embodiments, session area 5050b illustrates an extended communication session, optionally with controls including a control 5050b-1 for muting, a control 5050b-2 for accessing a keypad, a control 5050b-3 for initiating speakerphone use, a control 5050b-4 for initiating a video call, and / or a control 5050b-5 for terminating the communication session. In some embodiments, session area 5050c illustrates an extended screen recording session, optionally with a control 5050c-1 for ceasing screen recording. In some embodiments, session area 5050d illustrates an extended carpool session, optionally including a control 5050d-1 for contacting a carpool driver. In some embodiments, session area 5050e illustrates an extended media playback session, optionally including controls including a control 5050e-1 for skipping to the previous content item, a control 5050e-2 for pausing the content item, and / or a control 5050e-3 for skipping to the next content item.In some embodiments, session area 5050f shows an extended navigation session, optionally including controls including control 5050f-1 for pausing and / or control 5050f-2 for terminating navigation instructions. In some embodiments, a user can select any of the controls described above to cause device 100 to perform one or more actions associated with the respective control (e.g., in response to user input selecting control 5050c-1, device 100 stops screen recording).
[0273] 5BP-5CY illustrate transitions between application user interfaces. In some embodiments, an active media playback session is displayed in session region 502-30 while displaying user interface 5060 for a first application, such as a map application. In some embodiments, session region 502-30 includes a border (e.g., similar to border 552 (FIG. 5W)) displayed around an edge of session region 502-30. In some embodiments, session region 502-30 is displayed with a background of a first color, optionally selected to match the color of one or more sensors disposed within the region, as described above with reference to FIG. 5A. In some embodiments, device 100 detects user input 5062 corresponding to a request to switch applications, such as an edge swipe gesture or other application switching user input to switch the display of user interface 5060 to another application user interface. For example, in response to user input 5062, device 100 optionally displays an animated transition that includes sliding user interface 5060 in a first direction (optionally corresponding to the direction of user input 5062) and simultaneously sliding a music application user interface 5066 over the user interface.
[0274] 5BQ shows an example of an animated transition displayed on a user interface 5064, including a portion of a user interface 5060 sliding out of a display area and a portion of a user interface 5066 sliding onto the display area. In some embodiments, while displaying the animated transition for switching between application user interfaces, the session region 502-30 is optionally minimized as a session region 502-32, which does not display state information about active sessions even if one or more sessions remain active (e.g., a media playback session and / or a navigation session are active but not displayed in the session region during the animated transition). In some embodiments, the border of the session region 502-30 is displayed gradually shrinking or closing to indicate the edge of the session region 502-30 surrounding the smaller display area as the session region 502-30 decreases in size to the minimized session region 502-32. For example, as the size of the session region 502-30 decreases, the border of the session region 502-30 is maintained along the edges of the decreasing size of the session region 502-30. In some embodiments, the background color of the session region 502-30 is maintained as the size of the session region changes, including displaying the same background color of the minimized session region 502-32.
[0275] In some embodiments, as displayed in FIG. 5BR, while user interface 5066 of a music application corresponding to a media playback session is displayed in session area 502-30 (FIG. 5BP), device 100 does not display an indication of the media playback session in the session area, but instead displays the active navigation session in session area 502-34 associated with continued operation of a maps application whose user interface is no longer displayed by device 100. For example, because user interface 5066 is for an application associated with the media playback session, the media playback session is not displayed in the session area.
[0276] In some embodiments, device 100 detects another user input 5068, such as an edge swipe gesture or other user input, to switch the display of user interface 5066 to another application user interface. In some embodiments, as shown in FIG. 5BS1, in response to user input 5068, device 100 optionally displays an animated transition (e.g., the animated transition described with reference to FIG. 5BQ) and displays user interface 5070 for a messaging application (or another application different from the application corresponding to user interfaces 5060 and 5062). In some embodiments, while displaying user interface 5070 for the messaging application, device 100 displays indications for an active media playback session and an active navigation session in the session area. For example, a media playback session is displayed in portion 502-36 of the session area, optionally including one or more sensor notches, simultaneously with a navigation session displayed in portion 502-38 of the session area.
[0277] In some embodiments, following a determination that user input 5068 meets the quick switching criteria and / or following a determination that user interface 5066 is displayed for less than a threshold time before detecting user input 5068, device 100 optionally ceases extending session region 502-32 into session region 502-34 (or ceases to extend entirely). For example, if a user navigates between displays of user interface 5060, user interface 5066, and user interface 5070 in less than the threshold time, device 100 ceases extending session region into session region 502-34 and maintains minimized session (or nearly minimized) region 502-32 without displaying state information for the active session.
[0278] FIG. 5BS1 further illustrates detection of a user input 5102, such as an edge swipe gesture (optionally corresponding to an edge of the display area, such as a bottom edge) or other user input, to switch display of user interface 5070 to another application user interface. In some embodiments, in response to user input 5102, device 100 optionally displays an animated transition of a user interface in the display area outside the session area (e.g., the animated transition described with reference to FIG. 5BQ), as shown in user interface 5104 of FIG. 5BS2. In some embodiments, the animated transition of the display area outside the session area includes displaying the previously displayed user interface 5070 sliding off the display (e.g., gradually sliding off the display with a progression based on the progress of a user input, such as a swipe gesture, over time and / or a rate based on the rate of a user input, such as a swipe gesture), optionally in a direction corresponding to the direction of user input 5102, and a user interface for another application sliding across the display (e.g., user interface 5106 for a web browsing application in FIG. 5BS2). In some embodiments, device 100 further displays an animated transition that includes optionally minimizing the session area, including optionally merging and / or reducing the size of portions 502-36 and 502-38 of the session area to a minimum size shown as minimized session area 502-40. In some embodiments, as described with reference to FIG. 5BY, the animated transition includes gradually minimizing and / or expanding the session area as content displayed in the session area is optionally updated (e.g., from displaying session information for one or more first applications to displaying session information for one or more second applications).More generally, it should be noted that one or more aspects of the animated transition of the display area outside the session area (e.g., including sliding a previously displayed user interface off the display and sliding a user interface for another application onto the display) optionally apply similarly to any of the switching application user interface examples described herein, such as with respect to the examples of Figures 5BP-5BS7, 5BV-5BW, and / or 5CE-5CG, and optionally Figures 5R-5T.
[0279] In some embodiments, after displaying the animated transition(s) (e.g., for the display area outside the session area and / or for the session area), the device 100 displays a user interface 5106 for a web browsing application (FIG. 5BS3), or a user interface for another application, in the display area outside the session area. In some embodiments, as shown in FIG. 5BS3, the session area is updated from a minimized session area 502-40 to re-display the active session that was displayed before detecting the user input 5102. For example, in FIG. 5BS3, the portion 502-36 corresponding to the music session and the portion 502-38 corresponding to the navigation session are re-displayed simultaneously with the display of the user interface 5106 for the web browsing application. Thus, the device 100 maintains and / or re-displays the active session while the user interface displayed outside the session area is updated to a different application user interface.
[0280] FIG. 5BS3 further illustrates detection of a user input 5108 directed at the portion 502-36 corresponding to the music session. In some embodiments, the user input 5108 is a tap input or other user input corresponding to a request to open an application user interface for an application associated with the music session displayed in the portion 502-36. In response to the user input 5108, the device 100 displays a user interface 5066 for the music application, as shown in FIG. 5BS4. In some embodiments, in response to the user input 5108, the device 100 displays an animated transition of the session regions. For example, as shown in FIGS. 5BS3-5BS5, during the animated transition, the first and second portions 502-36 and 502-38 of the session regions (FIG. 5BS3) appear to decrease in size and merge into the minimized session region 502-40 (FIG. 5BS4), which then expands to display the session region 502-42 (FIG. 5BS5).
[0281] In some embodiments, as shown in Figure 5BS5, while user interface 5066 for a music application is displayed, device 100 displays session area 502-42 corresponding to a navigation session, optionally without displaying an active session corresponding to a music session in the session area. For example, as described above with reference to Figure 5AN, while a user interface for an application corresponding to an individual session is currently displayed in a display area outside the session area, individual session information is not displayed.
[0282] FIG. 5BS5 illustrates detecting user input 5110, optionally a swipe user input in a distinct direction, such as from right to left, and optionally a swipe user input corresponding to an edge of the display area, such as a bottom edge, corresponding to a request to display a user interface of another application. In response to detecting the user input 5110, the device 100 displays an animated transition of the display area out of the session area, as illustrated by the user interface 5116 of FIG. 5BS6. For example, the user interface of the music application is displayed sliding out of the display, optionally in the direction of the user input, such as from right to left, and the user interface of the map application is displayed sliding onto the display, optionally from an opposite edge of the display, in the direction of the user input. In some embodiments, the animated transition is similar to the animation (e.g., a reversal thereof) described with reference to FIG. 5BQ. As discussed above, in some embodiments, the device 100 displays an animated transition of the session area, including displaying a minimized session area 502-44 during the animated transition.
[0283] In response to detecting user input 5110, a user interface 5060 for a map application is displayed, as shown in FIG. 5BS7. In some embodiments, the session area gradually increases in size from the minimized session area 502-44 to display one or more active sessions, including replacing the display of navigation session 502-42 with a display of music session 502-46. For example, device 100 displays session area 502-46 corresponding to a music session that was not previously displayed while a user interface for the music application was displayed in a display area outside the session area. Similarly, in some embodiments, user interface 5060 for the map application is displayed in a display area outside the session area (FIG. 5BS7), but the session area does not display a session corresponding to a navigation session associated with the map application.
[0284] 5BT and 5BU show examples of session regions that optionally expand while transitioning between user interfaces for the respective applications. FIG. 5BT shows a session region (e.g., in user interfaces 5080a-5080e) due to a determination that a user input (e.g., user input 5068) does not meet the quick-switching criteria (e.g., a user input slower than the user input detected in FIG. 5BU). For example, if a user navigates between a user interface for a map application 5080a and a user interface for a music application 5080c and views the user interface for music application 5080c for at least a threshold time, the user input does not meet the quick-switching criteria. In some embodiments, while the user is viewing the user interface for music application 5080c, the session region expands (e.g., to session region 502-34) to display state information of the navigation session. In some embodiments, in response to detecting a user input to navigate to a messaging application, the session area is minimized (e.g., in user interface 5082d) during the transition before displaying state information for the active session in the session area (e.g., in portions 502-36 and 502-38 of the session area). In some embodiments, the session area is minimized during an animated transition between user interfaces (e.g., the animated transition in user interfaces 5080b and 5080d).
[0285] FIG. 5BU illustrates a session region (e.g., in user interfaces 5082a-5082e) pursuant to a determination that a user input (e.g., user input 5068) meets a quick-switching criterion (e.g., a user input faster than the user input detected in FIG. 5BT). For example, pursuant to a determination that a user navigates from a user interface for a map application 5082a to a user interface for a music application 5082c to a user interface for a messaging application 5082e without pausing for a threshold time while the user interface for the music application 5082c is displayed, the session region continues to be displayed in its minimized state without expanding the session region to display state information of the navigation session. In some embodiments, the session region is minimized during animated transitions between user interfaces (e.g., animated transitions in user interfaces 5082b and 5082d).
[0286] 5BV shows a display of a user interface 5060 for a map application and a session region 502-48 for a timer session. In response to detecting a user input to switch to another application user interface (e.g., user input 5118-1), device 100 displays an animated transition between the application user interfaces, optionally without displaying an animated transition of the session region 502-48. For example, device 100 displays an animated transition of a display area outside the session region, optionally without displaying an animated transition of the session region.
[0287] In some embodiments, the device 100 displays an animated transition of the session region 502-48, including minimizing the session region after detecting that the user input has stopped (e.g., detecting lift-off of the user input 5118-1). For example, FIG. 5BW shows the end of the user input 5118-1 as lift-off 5118-2. FIG. 5BW further shows that the session region 502-48 is maintained without animating the session region prior to detecting lift-off 5118-2. In some embodiments, the animated transition of the display area outside the session region shown in the user interface 5120 of FIG. 5BW begins in response to detecting the user input 5118-1 and is displayed while the user input 5118-1 is ongoing (e.g., optionally without requiring lift-off 5118-2 to occur, as opposed to what is required to display the animated transition of the session region 502-48).
[0288] FIG. 5BX shows that pursuant to a determination that user input has stopped (e.g., by detecting lift-off 5118-2), an animated transition of the session area is initiated, including gradually reducing the size of the session area 502-48 to a minimized session area 502-50, and optionally continuing the animated transition of the display area outside the session area, while simultaneously sliding a user interface for the maps application off the display and sliding a user interface for the music application onto the display.
[0289] In some embodiments, the animated transition of the session region further includes displaying a minimized session region 502-50 as it increases in size to display the first portion 502-52 and the second portion 502-54 of the session region, as shown in FIG. 5BY. For example, the animated transition of the session region, optionally initiated upon detecting lift-off 5118-2, includes shrinking the session region 502-48 by gradually decreasing the size of the text and / or icons displayed in the session region and / or removing the text and / or icons, and shrinking the area of the session region by gradually displaying a border of the session region that moves inward until the session region is displayed as a minimized session region 502-50 (FIG. 5BX). In some embodiments, the animated transition of the session region 502-48 is initiated in response to initially detecting a user input 5118-1 (e.g., without requiring lift-off 5118-2 to occur and without requiring the user input 5118-1 to meet additional criteria). In some embodiments, the animated transition of the session region 502-48 is optionally initiated in response to satisfaction of additional criteria, such as detection of user input 5118-1 and detection of movement of user input 5118-1, according to a determination that the user input 5118-1 has moved a threshold amount, is moving at a threshold speed, and / or is moving in a particular range of directions (e.g., within a threshold angular distance of a reference direction). In some embodiments, the minimized session region 502-50 continues to be displayed until the animated transition between application user interfaces in the display area outside the session region is completed. For example, in FIG. 5BY, the user interface for the music application 5124 is displayed at the end of the animated transition shown in the user interface 5122 in the display area outside the session region.
[0290] In some embodiments, the animated transition of the session area continues by gradually increasing the size of the minimized session area 502-50 to display the first portion 502-52 and the second portion 502-54 of the session area, including gradually displaying text and / or icons in the session area (e.g., by fading in, increasing in size, focusing or sharpening, and / or otherwise animating). In some embodiments, the portions are displayed extending outward from the minimized session area. For example, in FIG. 5BY, as shown in the first portion 502-56 of FIG. 5BZ, the first portion 502-52 displays an icon for a timer application, and as the first portion 502-52 expands, the first portion 502-52 displays the icon as a larger icon and / or displays additional text (e.g., the time remaining on the timer, 14:43). In some embodiments, one or more portions of the session area optionally separate to the right and / or left of the area corresponding to the minimized session area. For example, as shown in second portion 502-58 of Figure 5BZ, second portion 502-54 is animated to gradually separate from the minimized session area (e.g., as a separate user interface element resembling an air bubble), and the navigation icon gradually increases in size. More generally, it is noted that one or more aspects of the session area animated transitions described herein optionally apply similarly to any of the example session area transitions described herein, such as with respect to Figures 5BP-5BZ and / or 5CE-5CH.
[0291] 5BZ illustrates that while displaying a user interface for a music application 5124, device 100 detects user input 5126 directed at a first portion 502-56 of a session area corresponding to a timer session. In some embodiments, user input 5126 includes a first type of user input, such as a long press input, also referred to herein as a long press input. In some embodiments, user input 5126 includes a second type of user input, such as a tap input. In some embodiments, device 100 updates the display based on the type of input 5126 detected in response to user input 5126.
[0292] For example, pursuant to a determination that user input 5126 is a first type of user input, such as a long press input, device 100 expands (e.g., via a transition to FIG. 5CC) a session region corresponding to the session displayed in first portion 502-56. In some embodiments, pursuant to a determination that user input 5126 is a second type of user input, such as a tap input, device 100 displays (e.g., via a transition to FIG. 5CE) a user interface for an application (e.g., a clock application) associated with the session displayed in first portion 502-56 corresponding to a timer session.
[0293] In some embodiments, in response to detecting a user input 5126 (e.g., a first type input or a second type input), the device 100 animates a session area transition that includes shrinking, merging, and / or otherwise reducing the size of the first portion 502-56 and / or the second portion 502-58 of the session area, including shrinking of text and / or icons displayed within the individual portions, as shown in FIG. 5CA. In some embodiments, the session area transition further includes moving the first portion 502-56 and / or the second portion 502-58 closer to a midpoint (e.g., optionally predefined as a point between two or more sensors (e.g., speaker 111 and light sensor 164) of the session area, or another point substantially in the center of the display width). For example, as shown in FIG. 5CA, the distance between the separate user interface elements of the first portion 502-56 and the second portion 502-58 gradually decreases.
[0294] In some embodiments, the animated transition of a session region gradually reduces the size of a portion of the session region until the session region is minimized as session region 502-60 (FIG. 5CB) or session region 502-64 (FIG. 5CD). For example, the text and / or icons decrease in size, optionally approaching a halfway point, before the device stops displaying the text and / or icons.
[0295] In some embodiments, the device 100 displays the transition between displaying text and / or icons in the session regions by cross-fading or otherwise partially overlapping the display of the text and / or icons in the session regions displayed before and after the animated transition. For example, the device 100 begins displaying the text and / or icons in the session region 502-62, including fading the text and / or icons onto the display and / or gradually increasing the size of the text and / or icons while decreasing the size of the text and / or icons displayed in the first portion 502-56 and / or the second portion 502-58 of the session region. It will be appreciated that in some embodiments, the device refrains from minimizing the session region into the fully minimized session region 502-60 in order to display the text and / or icons in the session region with cross-fading. For example, the animated transition includes shrinking the text and / or icons in the first portion 502-56 and the second portion 502-58 without displaying the minimized session region 502-60 without including the text and / or icons. In other words, during the animated transition, the text and / or icons displayed in the session area 502-62 at least partially overlap with the display of the text and / or icons in the first and second portions 502-56 and 502-58 of the session area. In some embodiments, the display of the minimized session area 502-60 occurs during a cross-fade of the session areas displayed before and after the animated transition. For example, some of the overlap between the text and / or icons in the session area displayed before the animated transition and the text and / or icons in the session area displayed after the animated transition is displayed before the minimized session area 502-60 is displayed (e.g., as the size of the session area decreases), and some (e.g., the remaining) of the overlap is displayed after the minimized session area 502-60 is displayed (e.g., as the size of the session area increases).
[0296] FIG. 5CC illustrates that in response to detecting a first type of user input as user input 5126, the device continues to display a user interface for the music application 5124 while displaying an expanded session area 502-62 that includes one or more additional controls for a timer session. In some embodiments, the expanded session area 502-62 is automatically displayed for a threshold time without additional user input by the device 100, and redisplays the session area that was displayed before detecting the user input 5126, including displaying the first portion 502-56 and the second portion 502-58. In some embodiments, ceasing to display the expanded session area includes performing an animated transition in reverse. For example, the expanded session area is animated to decrease in size to the minimized session area 502-60, and the first portion 502-56 and the second portion 502-58 are animated as they increase in size and move outward from the minimized session area 502-60.
[0297] FIG. 5CD illustrates that in response to detecting a second type of user input, such as a tap input as user input 5126, device 100 displays a user interface for clock application 5128 associated with the session of first portion 502-56 in a display area outside the session area. In some embodiments, while replacing the display of the user interface for music application 5124 with the display of the user interface for clock application 5128, device 100 displays the animated transition of the session area described with reference to FIG. 5CA (e.g., by gradually decreasing the size of the session area until it is displayed as a minimized session area, and then gradually increasing the size of the session area to display the active session). In some embodiments, device 100 animates minimized session area 502-64 to gradually increase in size and display text and / or icons until it displays session area 502-66 corresponding to the navigation session, as shown in FIG. 5CE. As described with reference to FIG. 5AN, in some embodiments, the user interface for clock application 5128 is displayed in a display area outside the session area, but the portion of the session area corresponding to the timer session associated with the clock application is not displayed. For example, in FIG. 5CE, only the navigation session is displayed in session area 502-66.
[0298] Figure 5CE illustrates detecting a user input 5130 corresponding to a request to switch the display of an application user interface from the clock application user interface 5128 to another application user interface (e.g., a user interface for a messaging application 5134; Figure 5CG). In some embodiments, in response to detecting the user input 5130, device 100 displays an animated transition of the session area (e.g., as described with reference to Figure 5BQ) including minimizing the session area as minimized session area 502-68, and / or displays an animated transition of the display area outside the session area including animating a sliding transition in user interface 5132 between the user interface for the clock application 5128 and the user interface for the messaging application 5134.
[0299] FIG. 5CG illustrates device 100 displaying a user interface for messaging application 5134. In some embodiments, the session area is updated by optionally displaying an animated transition that displays portions of the session area that extend outside (e.g., to the right and / or left) of the minimized session area 502-68. For example, FIG. 5CG illustrates that a first portion 502-70 of the session area corresponding to a timer session includes a timer icon and a second portion 502-72 of the session area corresponding to a navigation session includes a right turn icon. In some embodiments, the animation of the session area includes displaying the first portion 502-70 to continue to increase in size, as illustrated by the first portion 502-74 of FIG. 5CH, and displaying additional information, such as text indicating the amount remaining on the timer. In some embodiments, during the animated transition, the second portion 502-72 continues to increase in size, including increasing the size of the second portion and / or increasing the size of the right turn icon, shifting away from the display area in which the minimized session area was displayed. For example, in FIG. 5CH, second portion 502-76 is displayed as a separate user interface element with a non-...
Claims
1. 1. A method comprising:
1. A computer system in communication with a display generation component having a display area including one or more sensors, a first display area, and a second display area different from the first display area, the one or more sensors being disposed within one or more sensor areas bounded by the display area, the display generation component being incapable of displaying content within the one or more sensor areas, displaying a user interface of a first application in the second display area, the second display area being adjacent to the first display area, the first display area encompassing the one or more sensor areas; detecting a user input corresponding to a request to display a user interface of a second application, different from the first application, in the second display area while displaying the user interface of the first application in the second display area; simultaneously, in response to detecting the user input corresponding to a request to display a user interface of the second application; a first user interface object in the first display region of the display area that encompasses the one or more sensor regions, the first user interface object including status information provided by the first application regarding a status of the first application; displaying a user interface of the second application in a second display area of the display area, the second display area being distinct from and adjacent to the first display area; continuing to display the first user interface objects in the first display area when the state of the first application changes, including displaying updated state information provided by the first application regarding the changed state of the first application in the first user interface objects, the updated state information being represented by visual elements placed within the first user interface objects to avoid overlapping positions of the one or more sensors within the first display area, while continuing to display the first user interface objects. detecting a user input corresponding to a request to display a user interface for a separate piece of software distinct from the first application and distinct from the second application; and in response to detecting the user input corresponding to the request to display the user interface for the individual software, displaying the user interface for the individual software in the second display area simultaneously with the first user interface object in the first display area.
2. The method of claim 1 , wherein the first user interface object containing state information surrounds the one or more sensor regions.
3. The status information is state information about the communication session; Information associated with the accessory connection; Information associated with an ongoing timer; information associated with media playback; information associated with sharing media content over wireless communication channels; Information associated with your virtual assistant; Information associated with biometrics; Information associated with your training activities; Information associated with ongoing events with regular updates; Navigation instructions, an indication of the battery level of the computer system; and / or 2. The method of claim 1, further comprising: an indication of the computer system participating in one or more hotspot connections.
4. The method of claim 3 , wherein the state information about the communication session includes audio waveforms, the audio waveform including a first audio waveform representing incoming audio and a second audio waveform representing outgoing audio.
5. 2. The method of claim 1 , further comprising displaying a second user interface object including second state information in a third display region of the display area, the third display region being between a first sensor region of the one or more sensor regions and a second sensor region of the one or more sensor regions.
6. 6. The method of claim 5, wherein the second status information includes a privacy indicator that is displayed pursuant to a determination that a microphone or camera of the computer system is currently collecting data, and / or the second status information includes a lock icon representing a locked state of the computer system.
7. the second state information is displayed on the second user interface object in the third display area while the first user interface object is displayed in a first state, the method further comprising:
6. The method of claim 5, further comprising: moving the second user interface object including the second state information to a fourth display area outside the first display area while the first user interface object is displayed in a second state.
8. updating a display of the first user interface object to a display of a third user interface object; the third user interface object includes additional state information provided by the first application beyond the state information included in the first user interface object; The first user interface object comprises: Detecting an event associated with the first application; Detection of said computer system located within a predetermined vicinity of a wireless terminal; and / or The method of claim 1 , further comprising updating the third user interface object in response to detecting a first type of user input.
9. providing non-visual feedback to a user in response to detecting the first type of user input and in accordance with a determination that the first user interface object is updated to the third user interface object; and ceasing to provide non-visual feedback to the user in accordance with a determination that the first user interface object is updated to the third user interface object in response to detecting an event associated with the first application.
10. The method of claim 8 , wherein displaying the third user interface object comprises displaying one or more user-selectable controls for controlling the first application.
11. The method of claim 8 , wherein the third user interface object is displayed in a portion of the user interface proximate to a portion of the user interface in which the first user interface object is displayed.
12. performing a first action in response to detecting a user input corresponding to any portion of the first user interface object while displaying the first user interface object; While displaying the third user interface object, performing a second action in response to detecting a user input on a first portion of the third user interface object; 10. The method of claim 8, further comprising: in response to detecting a user input on a second portion of the third user interface object, performing a third action different from the second action.
13. displaying the third user interface object in response to detecting the first type of user input without displaying a user interface for the first application; and displaying a user interface for the first application in response to detecting a second type of user input on a portion of the third user interface object.
14. in response to detecting a user input overlapping at least a portion of the first display area; updating a display of the first user interface object to display the third user interface object in accordance with a determination that an active session is displayed in the first display area and in accordance with a determination that the user input is a third type of user input; displaying a user interface associated with an application for the active session in accordance with a determination that an active session is displayed in the first display area and in accordance with a determination that the user input is a fourth type of user input; and displaying a fourth user interface object in the first display area in accordance with a determination that there is no active session displayed in the first display area.
15. detecting a user input within the first display area corresponding to the first user interface object that includes state information provided by the first application regarding the state of the first application; in response to detecting the user input within the first display area corresponding to the first user interface object that includes state information provided by the first application regarding the state of the first application; ceasing to display, in the first display area, the first user interface object that includes state information provided by the first application; 2. The method of claim 1, comprising: displaying a user interface of the first application in the second display area, the user interface of the first application being different from the user interface of the second application and different from the user interface for the individual software.
16. displaying the first user interface object in the first display area encompassing the one or more sensor areas in accordance with determining that the computer system is in a first orientation; 2. The method of claim 1, further comprising: displaying the first user interface object in a portion of the first display area that is different from a portion of the first display area that includes the one or more sensor areas in accordance with a determination that the computer system is in a second orientation.
17. automatically shrinking the third user interface object and displaying the first user interface object pursuant to a determination that a time threshold has been met while displaying the third user interface object; or 9. The method of claim 8, further comprising: in response to detecting user input outside the first display area, automatically shrinking the third user interface object to display the first user interface object.
18. 10. The method of claim 8, further comprising displaying a shadow proximate to the third user interface object while displaying the third user interface object, the shadow overlapping content displayed in the second display area of the display generation component.
19. displaying the first user interface object having a background color that matches a color of the one or more sensors disposed within the one or more sensor regions; and increasing a size of the first user interface object including extending the background color outward from the one or more sensor areas.
20. 9. The method of claim 8, further comprising displaying status information in the third user interface object, the status information in the third user interface object comprising visual elements selected to avoid overlapping positions of the one or more sensors in the first display area.
21. displaying an animation for the individual application, the displaying of the animation comprising: initially displaying a fifth user interface object in a display area between the one or more sensor regions; increasing a size of the fifth user interface object to a position overlapping the one or more sensor areas; and displaying the fifth user interface object in the second display area while decreasing a size of the first display area that encompasses the one or more sensor areas.
22. The method of claim 1 , comprising detecting one or more user inputs provided on the one or more sensor areas.
23. displaying one or more status indicators in the second display area while the first display area does not contain any user interface objects; ceasing to display at least one of the one or more status indicators in accordance with determining that the first user interface object is displayed in the first display area; and and ceasing to display one or more additional status indicators of the one or more status indicators in accordance with a determination that the first user interface object in the first display area increases in size.
24. determining whether the first status indicator or the second status indicator corresponds to an active connection; in response to a determination that the first status indicator corresponds to the active connection and the second status indicator does not correspond to the active connection, continuing to display the first status indicator; and ceasing display of the second status indicator.
25. while continuing to display the first user interface object in the first display area; 2. The method of claim 1, wherein in response to detecting the user input corresponding to a request to display the user interface for the particular software, the user interface for the particular software is displayed in the second display area without displaying an indication of the particular software in the first display area.
26. 2. The method of claim 1, further comprising: switching a display of the state information for the first event to a display of the state information for the second event within the first user interface object displayed in the first display area in accordance with a determination that the first application provides state information for a first event and a second event.
27. displaying the user interface of the first application in the second display area without displaying state information in the first display area; 2. The method of claim 1, further comprising: in response to detecting the user input corresponding to the request to display the user interface of the second application, displaying in the first display area the first user interface object including the state information provided by the first application regarding a state of the first application.
28. displaying in the second display area, simultaneously with the user interface of the first application, a user interface object in the first display area including state information provided by a third application relating to a state of the third application, the third application being different from the first application; in response to detecting the user input corresponding to a request to display a user interface of the second application; 2. The method of claim 1, comprising: displaying, in a first portion of the first display area, the first user interface object including the state information provided by the first application regarding a state of the first application; and simultaneously displaying, in a second portion of the first display area different from the first portion of the first display area, the user interface object including state information provided by the third application regarding a state of the third application.
29. 1. A computer system comprising: a display generation component having a display area including a first display area and a second display area different from the first display area; one or more sensors disposed within one or more sensor regions surrounded by the display area, wherein the display generation component is not capable of displaying content within the one or more sensor regions; one or more processors; 29. A computer system comprising: a memory having stored thereon one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing a method according to any one of claims 1 to 28.
30. 29. A computer program comprising instructions, when executed by a computer system having a display generation component having a display area including a first display area and a second display area different from the first display area, and one or more sensors disposed in the one or more sensor areas surrounded by the display area, wherein the display generation component is unable to display content in the one or more sensor areas, the computer program comprising instructions that, when executed by a computer system, cause the computer system to perform the method of claims 1 to 28.
Citation Information
Patent Citations
Steering gear for agricultural four-wheel-drive vehicle
JP1984032563A
X-ray mask
JP1988048820A
Recorder, recording method, sound waveform display device and sound waveform display method
JP2008263460A
Mobile device event notifications
JP2011516936A
Creative camera
JP2019197519A
