Device, method, and graphical user interface for updating session areas

Abstract

To provide an interface for accessing controls for controlling applications.SOLUTION: A method includes: displaying, in a first display region, a first user interface object that includes status information provided by respective software about a state of a computer system, the first display region encompassing one or more sensor regions; displaying, in a second display region that is different from the first display region, a user interface of an application that is different from the respective software, the second display region at least partially surrounding the first display region; as the state of the computer system changes, displaying, in the first user interface object, updated status information provided by the respective software about the changed state of the computer system, the updated status information including visual elements that are selected so as to avoid overlapping locations of the one or more sensors in the first display region.SELECTED DRAWING: None

Description

Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 403,681, filed Sep. 2, 2022, U.S. Provisional Patent Application No. 63 / 348,437, filed Jun. 2, 2022, and U.S. Provisional Patent Application No. 63 / 339,406, filed May 6, 2022, which are hereby incorporated by reference in their entirety.

Technical Field

[0002] This relates generally, but not limited to, electronic devices having a touch-sensing surface that includes a display area having one or more sensors and a session area, and more particularly to electronic devices having a touch-sensing surface.

Background Art

[0003] The use of touch-sensing surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch-sensing surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interfaces and objects therein. Exemplary user interface objects include digital images, videos, text, icons, and control elements such as buttons and other graphics.

[0004] Exemplary operations include adjusting the position and / or size of one or more user interface objects, activating buttons, or opening files / applications represented by user interface objects, as well as associating metadata with one or more user interface objects or otherwise manipulating the user interface. Exemplary user interface objects include digital images, videos, text, icons, and control elements such as buttons and other graphics.

[0005] However, the methods for performing these operations are cumbersome and inefficient. For example, using a series of mouse-based inputs to select one or more user interface objects and then performing one or more actions on the selected user interface objects is tedious and places a significant cognitive burden on the user. In addition, these methods take more time than necessary, thereby wasting energy. This latter consideration is particularly important in battery-powered devices. [Overview of the Initiative]

[0006] Therefore, there is a need for electronic devices with faster and more efficient methods and interfaces for viewing state information and accessing controls for controlling applications. Such methods and interfaces optionally complement or replace conventional methods for viewing state information and accessing controls for controlling applications. Such methods and interfaces reduce the number, range, and / or types of user input, resulting in a more efficient human-machine interface. With respect to battery-operated devices, such methods and interfaces conserve power and extend the time between battery charges.

[0007] The above-mentioned defects and other problems associated with user interfaces for electronic devices having touch-sensitive surfaces are reduced or eliminated by the devices of this disclosure. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device such as a watch). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a “touchscreen” or “touchscreen display”). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger touch and gestures on the touch-sensitive surface. In some embodiments, the functions are optional and include image editing, drawing, presentation, word processing, spreadsheet creation, gameplay, making phone calls, video conferencing, sending emails, instant messaging, training support, digital photography, digital videography, web browsing, digital music playback, note-taking, and / or digital video playback. The executable instructions for performing those functions are optional and may be contained in a non-temporary computer-readable storage medium or in other computer program products configured to be executed by one or more processors.

[0008] According to some embodiments, the method includes a computer system communicating with one or more sensors and a display generating component having a display area, wherein the method includes displaying a first user interface object in a first display area of ​​the display area, which includes state information provided by separate software relating to the state of the computer system, wherein the one or more sensors are located within one or more sensor areas enclosed by the display area, the display generating component cannot display content within the one or more sensor areas, and the first display area encompasses the one or more sensor areas. The method also includes displaying a user interface for an application different from the separate software in a second display area of ​​the display generating component, which is different from the first display area, wherein the second display area at least partially surrounds the first display area. When the state of the computer system changes, the method further includes displaying updated state information provided by separate software relating to the changed state of the computer system in the first user interface object, wherein the updated state information includes visual elements selected to avoid overlapping locations of one or more sensors within the first display area.

[0009] According to some embodiments, a method in a computer system that communicates with a display generating component having a display area including a state area includes displaying a first user interface in a display area outside the state area, which includes the user interface of a particular application running on the computer system, and updating the user interface of the particular application in accordance with one or more changes in the state of the particular application. The method includes detecting user input corresponding to a request to terminate the first user interface, and stopping the display of the first user interface, which includes the user interface of the particular application, in the display area outside the state area in response to detecting user input corresponding to a request to terminate the first user interface. The method also includes displaying an indication of the current state of a particular application in the state area while the state of the particular application is continuously changing, and displaying a second user interface associated with a different piece of software than the particular application in the display area outside the state area. The method further includes detecting user input for navigating from a second user interface associated with a separate software distinct from the individual application to a third user interface, and, in response to detecting user input for navigating from the second user interface, displaying the third user interface in a display area outside the state area while continuing to display an indication of the current state of the individual application in the state area.

[0010] According to some embodiments, the method includes, in a computer system communicating with a display generating component having a display area, displaying a first user interface of a first application in an application user interface area, wherein the display area includes a state area and an application user interface area separate from the state area, and the application user interface area at least partially surrounds the state area. The method includes detecting user input corresponding to a request to display a second user interface in the application user interface area, and, in response to detecting user input corresponding to a request to display a second user interface in the application user interface area, stopping the display of the first user interface of the first application in the application user interface area and displaying the second user interface in the application user interface area, according to a determination that the second user interface is associated with separate software different from the first application. The method further includes, in accordance with the determination that the state area is associated with an active session of a second application different from the first application, displaying a first indication of the current state of the first application in a first portion of the state area, and updating the first indication as the state of the first application changes; and displaying a second indication of the current state of the second application in a second portion of the state area different from the first portion, and updating the second indication as the state of the second application changes, wherein the first and second portions of the state area are displayed simultaneously with the second user interface.This method includes, in accordance with the determination that the state area is not associated with an active session of a second application different from the first application, displaying a first indication of the current state of the first application in the state area without displaying an indication of the current state of the second application in the state area, in a state area that is displayed simultaneously with the second user interface.

[0011] According to some embodiments, the method includes, in a computer system that communicates with a display generating component having a display area that at least partially encompasses a state area, detecting user input corresponding to a state area, and, in response to the detection of user input corresponding to a state area, performing an action associated with a particular application according to a determination that the state area is associated with an active session of a particular application. The method also includes, according to a determination that the state area is not associated with an active session of a particular application, providing feedback associated with user input without displaying information about a particular application in the state area and without performing an action associated with a particular application.

[0012] According to some embodiments, the method includes, in a computer system communicating with a display generating component having a display area including a state area, displaying a first user interface in a display area outside the state area, which includes a separate user interface element for initiating the registration of a user's biometric features. The method includes detecting user input directed to the separate user interface element to initiate the registration of the user's biometric features. The method further includes, in response to the detection of user input, displaying a biometric registration user interface in the state area while maintaining the display of the first user interface in the display area outside the state area, and updating the biometric registration user interface in the state area during the biometric registration process, while capturing biometric information relating to the user's biometric features to indicate the current state of the biometric registration process.

[0013] According to some embodiments, the method includes detecting user input to invoke a virtual assistant in a computer system that communicates with one or more sensors and a display generating component having a display area, wherein the one or more sensors are located within one or more sensor regions enclosed by a display area, the display generating component cannot display content within the one or more sensor regions, and a first display region of the display area encompasses one or more sensor regions. The method includes displaying a visual indication in the first display region that the virtual assistant is active in response to detecting user input to invoke the virtual assistant. The method further includes receiving a voice command directed to the virtual assistant and updating the first display region in response to receiving the voice command directed to the virtual assistant, which includes the computer system performing an action in response to the voice command and displaying the result of the action performed in response to the voice command, wherein the updating includes displaying visual elements selected to avoid overlapping positions of one or more sensors within the first display region.

[0014] According to some embodiments, the method includes displaying a first user interface in a display area outside the system user interface area in a computer system that communicates with a display generating component having a display area including a system user interface area. The method includes displaying the system user interface area by a first visual edge processing according to a determination that the system user interface area is displayed in a first display state, and displaying the background inside the system user interface area by a first appearance. The method further includes displaying the system user interface area by a second visual edge processing different from the first visual edge processing according to a determination that the system user interface area is displayed in a second display state different from the first display state, and displaying the background inside the system user interface area by a first appearance.

[0015] According to some embodiments, a method in a computer system that communicates with a display generating component having a display area including a state region includes displaying first information in the state region, which includes state information relating to first software. The state region is associated with an active session of the first software. The method includes detecting the occurrence of a first event corresponding to a transition to displaying second information different from the first information while displaying the first information in the state region. In response to detecting the occurrence of the first event, the method includes displaying a transition that includes stopping the display of the first information in the state region; displaying second information in the state region, wherein the second information includes state information relating to second software different from the first software, and the state region is associated with an active session of the second software; and reducing the size of the state region between the transition from displaying the first information in the state region to displaying the second information in the state region, and then expanding the size of the state region after reducing its size.

[0016] According to some embodiments, the electronic device includes a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, one or more processors, and a memory storing one or more programs, the one or more programs being configured to be executed by the one or more processors, and the one or more programs including instructions to perform or cause to perform any of the operations described herein. According to some embodiments, a computer-readable storage medium, when executed by an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators, internally stores instructions to cause the device to perform or cause to perform any of the operations described herein. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, memory, and one or more processors for executing one or more programs stored in memory includes one or more elements displayed in any of the methods described herein, the elements being updated in response to input as described in any of the methods described herein. According to some embodiments, the electronic device includes a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, and means for performing or causing to perform any of the operations described herein. According to some embodiments, an information processing device for use in an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators includes means for performing or causing to perform any of the operations described herein.

[0017] Thus, an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, optionally one or more device orientation sensors, and optionally a voice system provides an improved method and interface for providing state updates, thereby enhancing the effectiveness, efficiency, and user satisfaction of such a device. Such a method and interface can complement or replace conventional methods for providing state updates. [Brief explanation of the drawing]

[0018] To better understand the various embodiments described, the following “Modes for Carrying Out the Invention” should be referenced in conjunction with the following drawings, and similar reference numbers throughout the following drawings refer to the corresponding parts.

[0019] [Figure 1A] This is a block diagram showing a portable multifunctional device having a touch-sensitive display according to several embodiments.

[0020] [Figure 1B] This is a block diagram showing exemplary components for event handling according to several embodiments.

[0021] [Figure 2] Several embodiments of a portable multifunctional device having a touchscreen are shown.

[0022] [Figure 3] This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface, according to several embodiments.

[0023] [Figure 4A] The following are exemplary user interfaces for application menus on a portable multifunction device, according to several embodiments.

[0024] [Figure 4B] An exemplary user interface for a multi-functional device having a touch sensing surface separated from a display, according to some embodiments, is shown.

[0025] [Figure 5A] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5B] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5C] [[ID=1%]]An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5D] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5E] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5F] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5G] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5H] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5I] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5J] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5K] An exemplary user interface for updating state information within a session area, according to some embodiments, is shown. [Figure 5L] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5M] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5N] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5O] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5P] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5Q] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5R] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5S] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5T] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5U] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5V] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5W] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5X]The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5Y] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5Z] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AA] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AB] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AC] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AD] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AE] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AF] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AG] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AH] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AI] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AJ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AK] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AL] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AM] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AN] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AO] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AP] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AQ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AR1] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AR2] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AR3] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AS] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AT] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AU] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AV] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AW] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AX] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AY] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5AZ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BA1] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BA2] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BA3] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BB] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BC] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BD] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BE1] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BE2]The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BF] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BG] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BH] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BI] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BJ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BK] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BL] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BM1] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BM2] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BM3] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BM4] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BN] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BO] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BP] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BQ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BR] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS1] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS2] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS3] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS4] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS5] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS6] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BS7] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BT] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BU] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BV] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BW] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BX] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BY] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5BZ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CA] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CB] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CC] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CD] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CE] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CF] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CG] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CH]The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CI] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CJ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CK] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CL] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CM] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CN] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CO] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CP] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CQ] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CR] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CS] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CT] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CU] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CV] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CW] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CX] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments. [Figure 5CY] The following are exemplary user interfaces for updating state information within a session area, according to several embodiments.

[0026] [Figure 6A] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6B] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6C] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6D] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6E1] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6E2] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6F] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6G] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6G1]Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6G2] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6H] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6I] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6J] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6K] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6L] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication. [Figure 6M] Several embodiments illustrate exemplary user interfaces for registering for biometric authentication.

[0027] [Figure 7A] The following are exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments. [Figure 7B1] The following are exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments. [Figure 7B2] The following are exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments. [Figure 7C] The following are exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments. [Figure 7D]The following are exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments. [Figure 7E] The following are exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments.

[0028] [Figure 8A] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. [Figure 8B] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. [Figure 8C] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. [Figure 8D] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. [Figure 8E] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. [Figure 8F] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. [Figure 8G] The following are exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments.

[0029] [Figure 9A] This is a flowchart of a process for updating state information within a display area, according to several embodiments. [Figure 9B] This is a flowchart of a process for updating state information within a display area, according to several embodiments. [Figure 9C]This is a flowchart of a process for updating state information within a display area, according to several embodiments. [Figure 9D] This is a flowchart of a process for updating state information within a display area, according to several embodiments. [Figure 9E] This is a flowchart of a process for updating state information within a display area, according to several embodiments. [Figure 9F] This is a flowchart of a process for updating state information within a display area, according to several embodiments. [Figure 9G] This is a flowchart of a process for updating state information within a display area, according to several embodiments.

[0030] [Figure 10A] This is a flowchart of a process for continuously displaying updated state information in the state area while switching between different user interfaces, according to several embodiments. [Figure 10B] This is a flowchart of a process for continuously displaying updated state information in the state area while switching between different user interfaces, according to several embodiments. [Figure 10C] This is a flowchart of a process for continuously displaying updated state information in the state area while switching between different user interfaces, according to several embodiments. [Figure 10D] This is a flowchart of a process for continuously displaying updated state information in the state area while switching between different user interfaces, according to several embodiments.

[0031] [Figure 11A] This is a flowchart of a process for displaying updated state information for at least one application in the state area of ​​a display, according to several embodiments. [Figure 11B]This is a flowchart of a process for displaying updated state information for at least one application in the state area of ​​a display, according to several embodiments. [Figure 11C] This is a flowchart of a process for displaying updated state information for at least one application in the state area of ​​a display, according to several embodiments. [Figure 11D] This is a flowchart of a process for displaying updated state information for at least one application in the state area of ​​a display, according to several embodiments.

[0032] [Figure 12A] This is a flowchart of a process for interacting with a state domain to perform an action, according to several embodiments. [Figure 12B] This is a flowchart of a process for interacting with a state domain to perform an action, according to several embodiments. [Figure 12C] This is a flowchart of a process for interacting with a state domain to perform an action, according to several embodiments. [Figure 12D] This is a flowchart of a process for interacting with a state domain to perform an action, according to several embodiments. [Figure 12E] This is a flowchart of a process for interacting with a state domain to perform an action, according to several embodiments.

[0033] [Figure 13A] This is a flowchart of a process for displaying biometric authentication registration information in a state area, according to several embodiments. [Figure 13B] This is a flowchart of a process for displaying biometric authentication registration information in a state area, according to several embodiments. [Figure 13C] This is a flowchart of a process for displaying biometric authentication registration information in a state area, according to several embodiments. [Figure 13D]This is a flowchart of a process for displaying biometric authentication registration information in a state area, according to several embodiments. [Figure 13E] This is a flowchart of a process for displaying biometric authentication registration information in a state area, according to several embodiments.

[0034] [Figure 14A] This is a flowchart of the process for displaying updates for a virtual assistant in a sensor area, according to several embodiments. [Figure 14B] This is a flowchart of the process for displaying updates for a virtual assistant in a sensor area, according to several embodiments. [Figure 14C] This is a flowchart of the process for displaying updates for a virtual assistant in a sensor area, according to several embodiments.

[0035] [Figure 15A] This is a flowchart of a process for changing the appearance of a session area according to the current state of a device, according to several embodiments. [Figure 15B] This is a flowchart of a process for changing the appearance of a session area according to the current state of a device, according to several embodiments. [Figure 15C] This is a flowchart of a process for changing the appearance of a session area according to the current state of a device, according to several embodiments. [Figure 15D] This is a flowchart of a process for changing the appearance of a session area according to the current state of a device, according to several embodiments. [Figure 15E] This is a flowchart of a process for changing the appearance of a session area according to the current state of a device, according to several embodiments.

[0036] [Figure 16A] This is a flowchart of a process for switching the display of different types of information in a state domain, according to several embodiments. [Figure 16B] This is a flowchart of a process for switching the display of different types of information in a state domain, according to several embodiments. [Figure 16C] This is a flowchart of a process for switching the display of different types of information in a state domain, according to several embodiments. [Figure 16D] This is a flowchart of a process for switching the display of different types of information in a state domain, according to several embodiments. [Figure 16E] This is a flowchart of a process for switching the display of different types of information in a state domain, according to several embodiments. [Figure 16F] This is a flowchart of a process for switching the display of different types of information in a state domain, according to several embodiments. [Modes for carrying out the invention]

[0037] Many electronic devices have a graphical user interface that allows users to navigate between application user interfaces and / or system user interfaces. Several methods for navigating between user interfaces enable multitasking so that individual applications continue to update in the background even after the user has navigated away from their individual application user interface. For example, these methods may require the user to return to the individual application user interface to view the updates. The embodiments described below provide an improved method for providing state updates for multiple applications within a persistent session area. This method streamlines the user's ability to view real-time state information for an active session, thereby eliminating the need for an extra, separate step of returning to the individual user interface of an individual application to view the state updates.

[0038] The methods, devices, and GUIs described herein use tactile feedback to improve user interface interaction in several ways. For example, they make it easier to indicate hidden thresholds and user interface components that represent selectable options.

[0039] The processes described below improve the usability of the device and make the user device interface more efficient (for example, by helping the user provide appropriate input and reducing user errors when operating / interacting with the device) through various technologies, including providing the user with improved visual, auditory, and / or haptic feedback; reducing the number of inputs required to perform an operation; providing additional control options without cluttering the user interface with additional displayed controls; performing an operation when a set of conditions is met without requiring further user input; improving privacy and / or security; reducing the amount of display area required to display notifications and / or status information, and thus increasing the amount of display area available for other applications to display information; and / or additional technologies. These technologies also reduce power consumption and improve the battery life of the device by enabling the user to use the device more quickly and efficiently. Saving battery power and therefore weight improves the ergonomics of the device.

[0040] In the following, Figures 1A, 1B, 2, and 3 provide a description of exemplary devices. Figures 4A-4B and 5A-5CY show exemplary user interfaces for updating state information within the session area. Figures 6A-6M show exemplary user interfaces for registering for biometric authentication. Figures 7A-7E show exemplary user interfaces for updating the session area to use a stored payment method, according to several embodiments. Figures 8A-8G show exemplary user interfaces for updating the session area for a virtual assistant, according to several embodiments. Figures 9A-9G are flowcharts of a process for updating state information within the display area, according to several embodiments. Figures 10A-10D are flowcharts of a process for continuously displaying updated state information in the state area while switching displays between different user interfaces, according to several embodiments. Figures 11A-11D are flowcharts of a process for displaying updated state information for at least one application in the state area of ​​a display, according to several embodiments. Figures 12A-12E are flowcharts of a process for interacting with the state area to perform an action, according to several embodiments. Figures 13A to 13E are flowcharts of a process for displaying biometric registration information in the state area according to several embodiments. Figures 14A to 14C are flowcharts of a process for displaying updates for a virtual assistant in the sensor area according to several embodiments. Figures 15A to 15E are flowcharts of a process for changing the appearance of the session area according to the current state of the device according to several embodiments. Figures 16A to 16F are flowcharts of a process for switching the display of different types of information in the state area according to several embodiments. The user interfaces in Figures 5A to 5CY, 6A to 6M, 7A to 7E, and 8A to 8G are used to illustrate the processes in Figures 9A to 9G, 10A to 10D, 11A to 11D, 12A to 12E, 13A to 13E, 14A to 14C, 15A to 15E, and 16A to 16F.

[0041] Example device Herein, a detailed reference is made to embodiments shown in the accompanying drawings. The following detailed description includes numerous specific details to provide a complete understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other examples, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the aspects of the embodiments.

[0042] In this specification, terms such as "first," "second," etc., are used to describe various elements in several embodiments, but it will be understood that these elements should not be limited by those terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first and second contacts are contacts, but they are not the same contact unless the context explicitly indicates otherwise.

[0043] The terminology used in the descriptions of the various embodiments described herein is intended solely to describe specific embodiments and is not intended to be limiting. In the descriptions of the various embodiments and the accompanying claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless otherwise explicitly stated in the context. Furthermore, it should be understood that, as used herein, the term “and / or” refers to and includes any and all possible combinations of one or more of the enumerated items relating to the invention. It will be further understood that, as used herein, the terms “includes,” “comprises,” and / or “comprising,” specify the presence of the described features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0044] Where used herein, the term "if" is interpreted, at its discretion and in context, to mean "when," "upon," "in response to determining," or "in response to detecting." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" are interpreted, at its discretion and in context, to mean "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]."

[0045] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, which also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptop or tablet computers having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad), are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device but a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad).

[0046] The following discussion describes electronic devices including displays and touch-sensitive surfaces. However, please understand that the definition of an electronic device is optional and may include one or more other physical user interface devices such as a physical keyboard, mouse, and / or joystick.

[0047] The device generally supports a variety of applications, including note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0048] Various applications running on this device optionally utilize at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, are optionally adjusted and / or modified on an application-by-application basis and / or within individual applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally supports a variety of applications with intuitive and transparent user interfaces for the user.

[0049] Here, we turn our attention to embodiments of portable devices having a touch-sensitive display. Figure 1A is a block diagram of a portable multifunction device 100 having a touch-sensitive display system 112 according to several embodiments. The touch-sensitive display system 112 may be referred to as a “touchscreen” for convenience, or simply as a touch-sensitive display. The device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (for example, a touch-sensitive surface such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 that generate tactile outputs on Device 100 (for example, on touch-sensitive surfaces such as the touch-sensitive display system 112 of Device 100 or the touchpad 355 of Device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0050] As used herein and in the claims, the term “tactile output” means a physical displacement of the device relative to its previous position, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by the user through the user’s sense of touch. For example, in a situation where the device or a component of the device is in contact with the touch-sensitive surface of the user (e.g., the user’s fingers, palm, or other part of their hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) may be optionally interpreted by the user as a “down-click” or “up-click” of a physical actuator button. In some cases, the user may feel a tactile sensation such as a “down-click” or “up-click” even when there is no movement of a physical actuator button associated with a touch-sensitive surface that is physically pressed (e.g., displaced) by the user’s action. As another example, movement of a touch-sensitive surface may be interpreted or perceived by the user as "roughness" of that surface, even if there is no change in the smoothness of the touch-sensitive surface. Such user interpretations of touch depend on the user's personal sensory perception, but there are many touch sensory perceptions common to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., "up-click," "down-click," "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the device or its components that produce the described sensory perception of a typical (or average) user.Providing haptic feedback to the user using tactile output improves device usability and makes the user device interface more efficient (for example, by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, reduces power consumption and improves the device's battery life by enabling the user to use the device more quickly and efficiently.

[0051] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the motion waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.

[0052] When a device generates tactile outputs with various different tactile output patterns (for example, via one or more tactile output generators that move a movable mass to generate a tactile output), the tactile output can evoke various different tactile sensations when the user holds or touches the device. While the user's sensation is based on their perception of the tactile output, most users are able to identify changes in the waveform, frequency, and amplitude of the tactile output generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different actions are being performed. Therefore, in some situations, tactile outputs having tactile output patterns designed, selected, and / or developed to simulate the properties (e.g., size, material, weight, stiffness, smoothness, etc.), behavior (e.g., vibration, displacement, acceleration, rotation, expansion, etc.), and / or interactions (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or any combination of the above), provide the user with useful feedback that reduces input errors and increases the efficiency of the user's operation of the device. In addition, tactile outputs are optionally generated to correspond to feedback unrelated to simulated physical properties, such as input thresholds or object selection. In some situations, such tactile outputs provide the user with useful feedback that reduces input errors and increases the efficiency of the user's operation of the device.

[0053] In some embodiments, tactile output having a suitable tactile output pattern acts as a signal for the occurrence of a target event behind a scene in the user interface or device. Examples of target events include the activation of affordances provided on the device or within the user interface (e.g., physical buttons, virtual buttons, or toggle switches), the success or failure of a requested action, reaching or crossing a boundary within the user interface, entering a new state, switching the focus of input between objects, activating a new mode, reaching or exceeding an input threshold, and detecting or recognizing the type of input or gesture. In some embodiments, tactile output is provided to serve as a warning or alert for an upcoming event or result that will occur unless a redirection or interruption input is detected in a timely manner. Tactile output is also used in other contexts to improve the user experience, improve the accessibility of the device for users with visual or motor impairments or other accessibility needs, and / or improve the efficiency and functionality of the user interface and / or device. Tactile output, optionally accompanied by audio output and / or visible user interface changes, further enhances the user experience when the user interacts with the user interface and / or device, facilitates better communication of information regarding the state of the user interface and / or device, reduces input errors, and increases the efficiency of user operation of the device.

[0054] It should be understood that device 100 is merely an example of a portable multifunction device, and that device 100 may optionally have more or fewer components than those shown, may optionally be a combination of two or more components, or may optionally have different configurations or arrangements of those components. The various components shown in Figure 1A are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.

[0055] Memory 102 optionally includes high-speed random-access memory and optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by CPU(s) 120 and other components of device 100 such as peripheral interface 118 is optionally controlled by memory controller 122.

[0056] A peripheral device interface 118 is used to connect the device's input and output peripherals to the CPU(s) 120 and memory 102. One or more processors 120 operate or execute various software programs and / or instruction sets stored in memory 102 to perform various functions for device 100 and process data.

[0057] In some embodiments, the peripheral interface 118, the CPU(s) 120, and the memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0058] The RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, which include, but are not limited to, antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC chipsets, subscriber identity module (SIM) cards, and memory. The RF circuit 108 optionally communicates wirelessly with networks such as the Internet, also known as the World Wide Web (WWW), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), as well as with other devices. Wireless communication is optional and includes Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), and Code Division Multiple Access (W-CDMA).Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), Instant Messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE)), Instant Messaging and Presence Services Using any of several communication standards, communication protocols, and communication technologies, including, but not limited to, a Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including a communication protocol not yet developed as of the filing date of this document.

[0059] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts this audio data into an electrical signal, and transmits this electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits this audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or RF circuit 108 by the peripheral interface 118. In some embodiments, the audio circuit 110 further includes a headset jack (e.g., 212, Figure 2). The headset jack provides an interface between the audio circuit 110 and a detachable audio input / output peripheral device, such as an output-only headphone or a headset that has both an output (e.g., headphones for one or both ears) and an input (e.g., a microphone).

[0060] The I / O subsystem 106 connects input / output peripherals on device 100, such as the touch-sensitive display system 112 and other input or control devices 116, to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from and transmit electrical signals to the other input or control devices 116. The other input or control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) 160 are optionally connected to (or not connected to) any of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., 208 in Figure 2) optionally include up / down buttons (e.g., a single button that swings in opposite directions, or separate up and down buttons) for controlling the volume of speaker 111 and / or microphone 113. One or more buttons optionally include push buttons (e.g., 206, Figure 2).

[0061] The touch-sensitive display system 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays a visual output to the user. This visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively, “graphics”). In some embodiments, some or all of the visual output corresponds to user interface objects. As used herein, the term “affordance” refers to a user-interactive graphical user interface object (for example, a graphical user interface object configured to respond to input directed toward the graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0062] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or set of sensors that accept user input based on tactile and / or haptic contact. The touch-sensitive display system 112 and the display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of contact) on the touch-sensitive display system 112 and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0063] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light-emitting polymer display) technology, or LED (light-emitting diode) technology, but other display technologies are used in other embodiments. The touch-sensitive display system 112 and the display controller 156 optionally use any of several currently known or future-developed touch-sensing technologies, including but not limited to capacitive technology, resistive technology, infrared technology, and surface acoustic wave technology, as well as other proximity sensor arrays or other elements for determining one or more contact points with the touch-sensitive display system 112, to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as that found in iPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, California.

[0064] The touch-sensitive display system 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the video resolution of the touchscreen is greater than 400 dpi (e.g., 500 dpi, 800 dpi, or higher). The user optionally touches the touch-sensitive display system 112 using any suitable object or attachment such as a stylus or finger. In some embodiments, the user interface is designed to function with finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen than that of a stylus. In some embodiments, the device translates coarse finger input into a precise pointer / cursor position or command to perform an action desired by the user.

[0065] In some embodiments, in addition to the touchscreen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0066] Device 100 also includes a power system 162 that supplies power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with generating, managing, and distributing power within the portable device.

[0067] Device 100 also optionally includes one or more optical sensors 164 (for example, as part of one or more cameras). Figure 1A shows an optical sensor coupled with an optical sensor controller 158 in the I / O subsystem 106. The optical sensor(s) 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor(s) 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with the imaging module 143 (also called the camera module), the optical sensor(s) 164 optionally captures still images and / or video. In some embodiments, the optical sensors are located on the back of device 100, opposite the touch-sensitive display system 112 on the front of the device, so that a touchscreen can be used as a viewfinder for acquiring still images and / or video images. In some embodiments, a separate light sensor is positioned on the front of the device so that an image of the user can be captured (for example, for selfies, or for video conferencing while the user is viewing other video conference participants on the touchscreen).

[0068] Device 100 also optionally includes one or more contact strength sensors 165. Figure 1A shows a contact strength sensor coupled with a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor(s) 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, pressure-power sensors, optical force sensors, capacitive touch-sensing surfaces, or other strength sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensing surface). The contact strength sensor(s) 165 receives contact strength information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact strength sensor is positioned juxtaposed with or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112). In some embodiments, at least one contact strength sensor is located on the back of Device 100, opposite the touchscreen display system 112 which is located on the front of Device 100.

[0069] The device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows a proximity sensor 166 coupled to a peripheral interface 118. Alternatively, the proximity sensor 166 is coupled to an input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is positioned near the user's ear (for example, when the user is making a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0070] Device 100 also optionally includes one or more tactile output generators 167. Figure 1A shows a tactile output generator coupled with a tactile feedback controller 161 in the I / O subsystem 106. In some embodiments, the tactile output generator(s) 167 includes one or more electroacoustic devices such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). The tactile output generator(s) 167 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output on the device 100 that can be sensed by the user of the device 100. In some embodiments, at least one tactile output generator is positioned alongside or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112) and optionally generates a tactile output by moving the touch-sensing surface vertically (e.g., inward / outward from the surface of device 100) or horizontally (e.g., forward / backward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is positioned on the back of device 100, opposite the touch-sensing display system 112 which is located on the front of device 100.

[0071] The device 100 also optionally includes one or more accelerometers 168. Figure 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in a portrait or landscape view based on an analysis of data received from one or more accelerometers. In addition to the accelerometers 168, the device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0072] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, as shown in Figures 1A and 3, memory 102 stores device / global internal state 157. The device / global internal state 157 includes one or more of the following: an active application state indicating which application is active, if there is an application currently active; a display state indicating which application, view, or other information occupies various areas of the touch-sensitive display system 112; a sensor state including information obtained from various sensors and other input or control devices 116 of the device; and position and / or orientation information relating to the position and / or orientation of the device.

[0073] An operating system (for example, an embedded operating system such as iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks) includes various software components and / or drivers for controlling and managing overall system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0074] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to connect to other devices directly or indirectly via a network (e.g., the Internet, Wi-Fi, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors identical or similar to and / or compatible with the 30-pin connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external ports are Lightning connectors identical or similar to and / or compatible with the Lightning connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external port is a USB Type-C connector identical, similar to, and / or compatible with the USB Type-C connector used in some electronic devices from Apple Inc. in Cupertino, California.

[0075] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in cooperation with the display controller 156) and contact with other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger down event), determining the intensity of contact (e.g., the force or pressure of contact, or an alternative to the force or pressure of contact), determining whether there is movement of contact and tracking movement across the touch-sensitive surface (e.g., detecting a drag event of one or more fingers), and determining whether contact has stopped (e.g., detecting a finger up event or interruption of contact). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point, represented by a series of contact data, optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions can optionally be applied to a single contact (e.g., a single finger or stylus contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0076] The contact / motion module 130 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motion, timing, and / or intensity of detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture involves detecting a finger down event, followed by a finger up (lift-off) event at the same location (or substantially the same location) as the finger down event (e.g., at the icon location). In another embodiment, detecting a finger swipe gesture on the touch-sensitive surface involves detecting a finger down event, followed by a drag event of one or more fingers, and then a finger up (lift-off) event. Similarly, taps, swipes, drags, and other gestures are optionally detected with respect to the stylus by detecting specific contact patterns with respect to the stylus.

[0077] In some embodiments, detecting a finger tap gesture depends on the length of time between detecting a finger down event and detecting a finger up event, but not on the intensity of finger contact between the two events. In some embodiments, a tap gesture is detected according to the determination that the length of time between the finger down event and the finger up event is shorter than a predetermined value (e.g., shorter than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of finger contact between taps meets a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light or deep pressure intensity threshold. Thus, a finger tap gesture can satisfy certain input criteria that do not require the characteristic intensity of contact to meet a given intensity threshold in order for certain input criteria to be met. To clarify, finger contact in a tap gesture generally needs to meet a nominal contact detection intensity threshold below which contact is not detected in order to detect a finger down event. A similar analysis applies to detecting tap gestures or other contacts by a stylus. In cases where the device can detect contact from a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold is optional and does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0078] In a similar manner, the same concept applies to other types of gestures. For example, swipe gestures, pinch gestures, de-pinch gestures, and / or long press gestures are optional and are detected based on whether they meet criteria that are either unrelated to the intensity of the contacts involved in the gesture, or do not require the contact(s) performing the gesture to reach an intensity threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts. A pinch gesture is detected based on the movement of two or more contacts toward each other. A de-pinch gesture is detected based on the movement of two or more contacts toward each other. A long press gesture is detected based on the duration of contact with a touch-sensitive surface involving movement less than a threshold amount. Therefore, the statement that a particular gesture recognition criterion does not require the intensity of one or more contacts to meet a corresponding intensity threshold for that criterion to be satisfied means that a particular gesture recognition criterion can be satisfied when one or more contacts in a gesture do not reach a corresponding intensity threshold, and can also be satisfied when one or more of the contacts in a gesture reach or exceed a corresponding intensity threshold. In some embodiments, a tap gesture is detected based on the determination that finger-down and finger-up events were detected within a predetermined time period, regardless of whether the contacts are above or below their respective intensity thresholds during a predetermined time period, and a swipe gesture is detected based on the determination that the movement of the contacts is greater than a predetermined magnitude, even if the contacts exceed their respective intensity thresholds at the end of the movement of the contacts. Even in embodiments where gesture detection is influenced by the intensity of the contact performing the gesture (for example, the device detects longer presses more quickly when the contact intensity exceeds an intensity threshold, or the device is slower to detect tap inputs when the contact intensity is higher), the detection of those gestures does not require the contact to reach a specific intensity threshold, as long as the criteria for recognizing the gesture can be met in situations where the contact does not reach a specific intensity threshold (for example, even if the amount of time required to recognize the gesture changes).

[0079] Contact intensity thresholds, duration thresholds, and movement thresholds are combined in various different combinations to create heuristics for distinguishing two or more different gestures directed towards the same input element or region in certain situations, thereby enabling multiple different interactions with the same input element to provide a richer set of user interactions and responses. The statement that a particular set of gesture recognition criteria does not require the intensity of a contact(s) to meet its respective intensity threshold for that particular gesture recognition criterion to be satisfied does not preclude simultaneously evaluating other intensity-dependent gesture recognition criteria to identify other gestures that have criteria that are satisfied when the gesture includes contact with an intensity exceeding its respective intensity threshold. For example, in some situations, a first gesture recognition criterion for a first gesture that does not require the intensity of a contact(s) to meet a corresponding intensity threshold for the first gesture recognition criterion to be satisfied is in competition with a second gesture recognition criterion for a second gesture that depends on the contact(s) reaching a corresponding intensity threshold. In such competition, a gesture is optional and will not be recognized as satisfying the first gesture recognition criterion for the first gesture if the second gesture recognition criterion for the second gesture is first satisfied. For example, if a contact reaches the corresponding intensity threshold before it moves a predetermined amount, a deep press gesture will be detected instead of a swipe gesture. Conversely, if a contact moves a predetermined amount before it reaches the corresponding intensity threshold, a swipe gesture will be detected instead of a deep press gesture. Even in such situations, the first gesture recognition criterion for the first gesture still does not require the intensity of the contact(s) to meet the corresponding intensity threshold for the first gesture recognition criterion to be satisfied, because if the contact remains below the corresponding intensity threshold until the end of the gesture (e.g., a swipe gesture with a contact that does not increase to an intensity above the corresponding intensity threshold), the gesture is recognized as a swipe gesture by the first gesture recognition criterion.In this way, a particular gesture recognition criterion that does not require the intensity of a contact(s) to meet a corresponding intensity threshold for that particular gesture recognition criterion to be satisfied still depends on the intensity of the contact(s) with respect to the intensity threshold in some situations, and / or (B) in some situations, the particular gesture recognition criterion (e.g., for a long press gesture) does not function if a competing set of intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-dependent gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input (e.g., for a long press gesture competing with a deep press gesture for recognition).

[0080] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other display, including components for modifying the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term “graphics” includes, but is not limited to, any object that can be displayed to a user, including, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, and animations.

[0081] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like, as needed, along with coordinate data and other graphic characteristic data, specifying the graphics to be displayed, and then generates screen image data to output to the display controller 156.

[0082] The haptic feedback module 133 includes various software components that generate commands (for example, commands used by the haptic feedback controller 161) that create haptic outputs at one or more locations on the device 100 using a haptic output generator(s) 167, in response to user interaction with the device 100.

[0083] The text input module 134 is optionally a component of the graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other applications that require text input).

[0084] The GPS module 135 determines the device's location and provides this information for use in various applications (for example, to the phone 138 for location-based phone calls, to the camera 143 as metadata for photos / videos, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0085] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: ● Contact module 137 (sometimes called the address book or contact list), ●Telephone module 138, ●Video conferencing module 139, ● Email client module 140, ● Instant messaging (IM) module 141, ●Training support module 142, ● Camera module 143 for still images and / or video images, ●Image management module 144, ● Browser module 147, ● Calendar module 148, ● A widget module 149 that optionally includes one or more of the following: weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, other widgets obtained by the user, and user-created widgets 149-6. ●Widget creation module 150 for creating user-created widget 149-6, ●Search module 151, ● A video and music player module 152, which optionally consists of a video player module and a music player module. ●Memo Module 153, ●Map module 154 and / or ● Online video module 155.

[0086] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0087] Together with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the contact module 137 includes executable instructions for managing an address book or contact list (for example, stored in the application internal state 192 of the contact module 137 in memory 102 or memory 370), which include adding names to the address book, removing names from the address book, associating telephone numbers, email addresses, physical addresses, or other information with names, associating images with names, categorizing and sorting names, providing telephone numbers and / or email addresses to initiate and / or facilitate communication by telephone 138, video conferencing 139, email 140, or IM 141, and so on.

[0088] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the telephone module 138 includes executable commands for inputting a series of characters corresponding to a telephone number, accessing one or more telephone numbers in the address book 137, changing the entered telephone number, dialing each telephone number, conducting a conversation, and disconnecting or ending the call when the conversation is complete. As described above, the wireless communication may optionally use any of several communication standards, protocols, and technologies.

[0089] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, the video conferencing module 139 includes executable commands to start, conduct, and end a video conference between the user and one or more other participants, in accordance with the user's commands.

[0090] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the email client module 140 includes executable commands for creating, sending, receiving, and managing emails in response to user commands. In conjunction with the image management module 144, the email client module 140 makes it extremely easy to create and send emails containing still or video images captured by the camera module 143.

[0091] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the instant messaging module 141 includes executable instructions for inputting a series of characters corresponding to an instant message, modifying previously entered characters, sending each instant message (for example, using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant messaging, or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0092] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, text input module 134, GPS module 135, map module 154, and video and music player module 152, the training support module 142 includes executable commands for creating training (e.g., having time, distance, and / or calorie consumption targets), communicating with training sensors (in sports devices and smartwatches), receiving training sensor data, calibrating sensors used to monitor training, selecting and playing music for training, and displaying, storing, and transmitting training data.

[0093] Together with the touch-sensitive display system 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0094] Together with the touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, text input module 134, and camera module 143, the image management module 144 includes executable commands for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still and / or video images.

[0095] Together with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the browser module 147 includes executable commands for browsing the internet in accordance with user commands, including searching, linking, receiving, and displaying web pages or parts thereof, as well as attachments and other files linked to web pages.

[0096] Together with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, the calendar module 148 includes executable instructions that, in accordance with user commands, create, display, modify, and store a calendar and data associated with the calendar (e.g., calendar items, to-do lists, etc.).

[0097] Together with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget module 149 is optionally a mini-application downloaded and used by the user (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a mini-application created by the user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0098] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget creation module 150 includes executable commands for creating widgets (for example, converting user-specified portions of a web page into widgets).

[0099] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions to search for text, music, sound, images, videos, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) according to user commands.

[0100] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music player module 152 includes executable commands that enable the user to download and play recorded music or other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable commands to display, present, or otherwise play video (for example, on the touch-sensitive display system 112, or on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0101] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the memo module 153 includes executable commands that create and manage notes, to-do lists, and the like, according to user commands.

[0102] In relation to the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 includes executable commands that, in accordance with user commands, receive, display, modify, and store maps and map-related data (e.g., driving directions, data about stores and other points of interest in a particular location or nearby, and other location-based data).

[0103] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes executable instructions that enable a user to access, view, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen 112, or on an external display connected wirelessly or via external port 124), send emails containing links to specific online videos, and otherwise manage them. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos.

[0104] Each of the modules and applications identified above corresponds to one or more of the functions described above, as well as an executable instruction set for performing the methods described in this application (e.g., methods performed by a computer and other information processing methods described herein). These modules (e.g., instruction sets) do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of these modules may, at their discretion, be combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 may, at its discretion, store a subset of the modules and data structures identified above. Furthermore, memory 102 may also, at its discretion, store additional modules and data structures not described above.

[0105] In some embodiments, device 100 is a device in which the operation of a predetermined set of functions in the device is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for device 100 to operate, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.

[0106] A predetermined set of functions performed only through a touchscreen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, home menu, or root menu. In such embodiments, a “menu button” is implemented using a touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0107] Figure 1B is a block diagram showing exemplary components for event processing according to several embodiments. In some embodiments, memory 102 (in Figure 1A) or 370 (in Figure 3) includes an event sorting unit 170 (e.g., within the operating system 126) and each application 136-1 (e.g., any of the applications 136, 137-155, 380-390 described above).

[0108] The event sorting unit 170 receives event information and determines the application 136-1 that distributes the event information, and the application view 191 of application 136-1. The event sorting unit 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view(s) displayed on the touch-sensitive display system 112 when the application is active or running. In some embodiments, a device / global internal state 157 is used by the event sorting unit 170 to determine which application(s) are currently active, and the application internal state 192 is used by the event sorting unit 170 to determine the application view(s) to which the event information is distributed.

[0109] In some embodiments, the application internal state 192 includes additional information such as resume information to be used when the application 136-1 resumes execution, user interface state information that indicates or is ready to display information displayed by the application 136-1, a state queue that allows the user to return to a previous state or view of the application 136-1, and one or more redo / undo queues of previous actions performed by the user.

[0110] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (for example, a user touch on the touch-sensitive display system 112 as part of a multi-touch gesture). The peripheral interface 118 transmits information received from the I / O subsystem 106, or from sensors such as the proximity sensor 166, one or more accelerometers 168, and / or the microphone 113 (via the audio circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display system 112 or the touch-sensitive surface.

[0111] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when a significant event occurs (e.g., receiving input that exceeds a predetermined noise threshold and / or for a longer period than predetermined).

[0112] In some embodiments, the event sorting unit 170 also includes a hit view determination module 172 and / or an active event recognition determination module 173.

[0113] The hit view determination module 172 provides software procedures for determining where in one or more views a sub-event occurred when the touch-sensitive display system 112 displays two or more views. A view consists of control devices and other elements that the user can see on the display.

[0114] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of an individual application) in which a touch is detected optionally corresponds to a program level within the application's program hierarchy or view hierarchy. For example, the lowest-level view in which a touch is detected optionally refers to a hit view, and the set of events recognized as appropriate input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch gesture.

[0115] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views arranged in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchy from which the sub-events should be processed. In most situations, the hit view is the lowest-level view from which the initiating sub-event (e.g., the first sub-event in a sub-event sequence that forms an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source from which it was identified as a hit view.

[0116] The active event recognition determination module 173 determines which view(s) in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognition determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognition determination module 173 determines that all views, including the physical location of the sub-event, are actively involved views, and therefore all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if the touch sub-event is entirely confined to an area associated with one particular view, higher-level views in the hierarchy still remain actively involved views.

[0117] The event dispatcher module 174 dispatches event information to an event recognition unit (e.g., an event recognition unit 180). In embodiments including an active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores the event information acquired by each event receiving unit module 182 in an event queue.

[0118] In some embodiments, the operating system 126 includes an event sorting unit 170. Alternatively, application 136-1 includes an event sorting unit 170. In yet another embodiment, the event sorting unit 170 is a standalone module or part of another module stored in memory 102, such as a contact / motion module 130.

[0119] In some embodiments, application 136-1 includes a plurality of event processing units 190 and one or more application views 191, each containing instructions for handling touch events occurring within a separate view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognition units 180. Typically, a separate application view 191 includes a plurality of event recognition units 180. In other embodiments, one or more of the event recognition units 180 are part of a separate module, such as a user interface kit or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a separate event processing unit 190 includes one or more event data 179 received from a data update unit 176, an object update unit 177, a GUI update unit 178, and / or an event sorting unit 170. The event processing unit 190 optionally utilizes or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 to update the application's internal state 192. Alternatively, one or more of the application views 191 include one or more respective event processing units 190. In some embodiments, one or more of the data update unit 176, object update unit 177, and GUI update unit 178 are included in individual application views 191.

[0120] Each individual event recognition unit 180 receives event information (e.g., event data 179) from the event sorting unit 170 and identifies an event from the event information. The event recognition unit 180 includes an event receiving unit 182 and an event comparison unit 184. In some embodiments, the event recognition unit 180 also includes at least a subset of metadata 183 and event distribution commands 188 (optionally including sub-event distribution commands).

[0121] The event receiving unit 182 receives event information from the event sorting unit 170. The event information includes information about sub-events, such as touches or the movement of touches. Depending on the sub-event, the event information also includes additional information such as the position of the sub-event. When the sub-event involves the movement of a touch, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, an event includes the rotation of the device from one orientation to another (e.g., from portrait to landscape, or vice versa), and the event information includes corresponding information about the current orientation of the device (also called the device's orientation).

[0122] The event comparison unit 184 compares event information with a predetermined event or sub-event definition and, based on the comparison, determines the event or sub-event, or determines or updates the state of the event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. The event definition 186 includes definitions of events (e.g., a predetermined sequence of sub-events), such as event 1 (187-1) and event 2 (187-2). In some embodiments, sub-events in event 187 include, for example, touch start, touch cancellation, touch movement, touch abortion, and multiple touches. In one embodiment, the definition for event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch (touch start) for a predetermined stage on the displayed object, a first lift-off (touch end) for a predetermined stage, a second touch (touch start) for a predetermined stage on the displayed object, and a second lift-off (touch end) for a predetermined stage. In another embodiment, event 2(187-2) is defined as a drag on a displayed object. The drag includes, for example, touching (or contacting) a predetermined number of stages on the displayed object, moving the touch across the touch-sensitive display system 112, and lifting off the touch (end of the touch). In some embodiments, the event also includes information about one or more associated event processing units 190.

[0123] In some embodiments, the event definition 187 includes event definitions for individual user interface objects. In some embodiments, the event comparison unit 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, the event comparison unit 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with an individual event processing unit 190, the event comparison unit uses the results of the hit test to determine which event processing unit 190 should be activated. For example, the event comparison unit 184 selects the sub-event and the event processing unit associated with the object that triggers the hit test.

[0124] In some embodiments, the definition of each event 187 also includes a delay action that delays the delivery of event information until it is determined whether a series of sub-events correspond to the event type of the event recognition unit.

[0125] If an individual event recognition unit 180 determines that a series of sub-events does not match any of the events in the event definition 186, the individual event recognition unit 180 enters an event impossible, event failed, or event terminated state, and thereafter ignores subsequent sub-events of the touch-based gesture. In this situation, if there are other event recognition units that remain active for the hit view, those event recognition units continue to track and process the sub-events of the ongoing touch-based gesture.

[0126] In some embodiments, an individual event recognition unit 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate to the actively involved event recognition unit how the event distribution system should perform sub-event distribution. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognition units interact with each other, or how they can interact with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are distributed to various levels in the view hierarchy or program hierarchy.

[0127] In some embodiments, an individual event recognition unit 180 activates an event processing unit 190 associated with an event when one or more specific sub-events of an event are recognized. In some embodiments, the individual event recognition unit 180 delivers event information associated with the event to the event processing unit 190. Activating the event processing unit 190 is separate from sending (and delaying the sending of) sub-events to individual hit views. In some embodiments, the event recognition unit 180 sets a flag associated with the recognized event, and the event processing unit 190 associated with that flag captures the flag and executes a default process.

[0128] In some embodiments, the event distribution command 188 includes a sub-event distribution command that distributes event information about a sub-event without activating an event processing unit. Instead, the sub-event distribution command distributes event information to an event processing unit associated with a set of sub-events, or to an actively involved view. The event processing unit associated with the set of sub-events or the actively involved view receives the event information and executes a predetermined process.

[0129] In some embodiments, the data update unit 176 creates and updates data used in application 136-1. For example, the data update unit 176 updates telephone numbers used in contact module 137 or stores video files used in video and music player module 152. In some embodiments, the object update unit 177 creates and updates objects used in application 136-1. For example, the object update unit 177 creates new user interface objects or updates the position of user interface objects. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares display information and sends it to graphics module 132 for display on touch-sensitive display.

[0130] In some embodiments, the event processing unit(s) 190 includes or has access to a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are contained in a single module of an individual application 136-1 or application view 191. In other embodiments, they are contained in two or more software modules.

[0131] The foregoing description regarding the handling of user touch events on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 using input devices, but it should be understood that not all of this begins on the touchscreen. For example, mouse movement and mouse button presses, touch movements such as taps, drags, and scrolls on a touchpad, pen stylus input, device movement, verbal commands, detected eye movements, biometric input, and / or any combination thereof may be optionally used as inputs corresponding to sub-events that define the events to be recognized.

[0132] Figure 2 shows a portable multifunctional device 100 having a touchscreen (e.g., a touch-sensitive display system 112, Figure 1A) according to several embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In these embodiments, and in embodiments described later, the user can select one or more graphics by making gestures on the graphics using, for example, one or more fingers 202 (not shown in the figure to an exact scale) or one or more styluses 203 (not shown in the figure to an exact scale). In some embodiments, the selection of one or more graphics is performed when the user interrupts contact with that one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, right to left, upward and / or downward), and / or rolling of a finger in contact with the device 100 (from right to left, left to right, upward and / or downward). In some implementations or situations, accidental contact with a graphic does not constitute a selection of that graphic. For example, if the gesture corresponding to selection is a tap, a swipe gesture sweeping over an application icon does not arbitrarily select the corresponding application.

[0133] Device 100 optionally also includes one or more physical buttons, such as a "Home" or menu button 204. As previously mentioned, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally run on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display, or as a system gesture such as an upward edge swipe.

[0134] In some embodiments, device 100 includes a touchscreen display, a menu button 204 (sometimes referred to as a home button 204), a push button 206 for turning the device on / off and locking the device, one or more volume buttons 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and / or an external port 124 for docking / charging. The push button 206 is optionally used to turn the device on / off by pressing down the button and holding it down for a predetermined period of time, to lock the device by pressing down the button and releasing it before a predetermined time has elapsed, and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts verbal input through a microphone 113 to activate or deactivate certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output to the user of device 100.

[0135] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to several embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), game system, or control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication buses 320 optionally include circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 typically includes an input / output (I / O) interface 330, which includes a display 340, which is a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350, as well as a touchpad 355, a tactile output generator 357 for generating tactile output on a device 300 (similar to the tactile output generator(s) 167 described above with reference to Figure 1A), and a sensor 359 (e.g., an optical sensor, an accelerometer, a proximity sensor, a touch sensor, and / or a contact intensity sensor (similar to the contact intensity sensor(s) 165 described above with reference to Figure 1A). The memory 370 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or subsets thereof, that are stored in memory 102 of the portable multifunction device 100 (Figure 1A). Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of the portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk authoring module 388, and / or a spreadsheet module 390, while memory 102 of the portable multifunction device 100 (Figure 1A) optionally does not store those modules.

[0136] Each of the elements identified in Figure 3 above is optionally stored in one or more of the previously mentioned memory devices. Each of the modules identified above corresponds to an instruction set that performs the function described above. The modules or programs (i.e., instruction sets) identified above do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of those modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0137] Here, we focus on embodiments of a user interface ("UI") that are optionally implemented on the portable multi-functional device 100.

[0138] Figure 4A shows an exemplary user interface for an application menu on a portable multifunction device 100 according to several embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or subsets or supersets thereof. ● Signal strength indicators (single or multiple) for wireless communication (single or multiple) such as cellular signals and Wi-Fi signals. ●Time, ●Bluetooth (registered trademark) indicator, ●Battery status indicator, ●Tray 408 containing icons for frequently used applications, as shown below. ○Optionally including an indicator 414 for the number of missed calls or voicemail messages, an icon 416 of the telephone module 138 labeled "Telephone", ○Optionally including an indicator 410 for the number of unread emails, an icon 418 of the email client module 140 labeled "Mail", ○ Icon 420 of browser module 147, labeled "Browser", ○ Icon 422 of the video and music player module 152 labeled "Music", ● Icons of other applications, such as the following: ○ Icon 424 of IM module 141, labeled "Message", ○ Icon 426 of calendar module 148, labeled "Calendar", ○ Icon 428 of image management module 144, labeled "Photo" ○ Icon 430 of camera module 143, labeled "Camera" ○ Icon 432 of online video module 155, labeled "online video" ○ Icon 434 of stock price widget 149-2, labeled "Stock Price" ○ Icon 436 of map module 154, labeled "Map" ○ Icon 438 of weather widget 149-1, labeled "Weather" ○ Icon 440 of the alarm clock widget 149-4, labeled "Clock" ○ Icon 442 of training support module 142, labeled "Training Support" ○ Icon 444 of memo module 153, labeled as "Memo" ○ An icon 446 for a configuration application or module that provides access to settings related to device 100 and its various applications 136.

[0139] Please note that the icon labels shown in Figure 4A are merely examples. For example, other labels may be used selectively for various application icons. In some embodiments, the labels for individual application icons include the name of the application to which the individual application icon corresponds. In some embodiments, the label for a particular application icon is different from the name of the application to which that particular application icon corresponds.

[0140] Figure 4B shows an exemplary user interface on a device (e.g., device 300, Figure 3) having a touch-sensitive surface 451 (e.g., tablet or touchpad 355, Figure 3) separate from the display 450. Many of the following embodiments are given by reference to input on the touchscreen display 112 (when the touch-sensitive surface and the display are combined), but in some embodiments, the device detects input on the touch-sensitive surface separate from the display, as shown in Figure 4B. In some embodiments, the touch-sensitive surface (e.g., 451 in Figure 4B) has a principal axis (e.g., 452 in Figure 4B) corresponding to a principal axis (e.g., 453 in Figure 4B) on the display (e.g., 450). According to those embodiments, the device detects contact (e.g., 460 and 462 in Figure 4B) with the touch-sensitive surface 451 at positions corresponding to each of the positions on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in Figure 4B). Thus, when the touch-sensitive surface is separated from the display, user input detected by the device on the touch-sensitive surface (e.g., 451 in Figure 4B) (e.g., touches 460 and 462, and their movement) is used by the device to operate the user interface on the display of the multifunction device (e.g., 450 in Figure 4B). It should be understood that a similar method may be optionally used for other user interfaces described herein.

[0141] In addition, while the following embodiments are given primarily with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture, etc.), it should be understood that in some embodiments, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of touch), followed by a mouse click with cursor movement along the swipe path (e.g., instead of touch movement). As another example, a tap gesture may optionally be replaced by a mouse click (e.g., instead of touch detection and subsequent cessation of touch detection) while the cursor is located over the tap gesture position. Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice may optionally be used simultaneously, or mice and finger touch may optionally be used simultaneously.

[0142] As used herein, the term “focus selector” refers to an input element that indicates the current portion of the user interface with which the user is interacting. In some implementations, including a cursor or other location marker, the cursor functions as a “focus selector” such that when input (e.g., press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in Figure 3, or touch-sensitive surface 451 in Figure 4B) while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element is adjusted according to the detected input. In some implementations, including a touchscreen display (e.g., touch-sensitive display system 112 in Figure 1A, or touchscreen in Figure 4A) that enables direct interaction with user interface elements on the touchscreen display, contact detected on the touchscreen functions as a “focus selector” such that when input (e.g., press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of ​​the user interface to another without the movement of a corresponding cursor or touch on the touchscreen display (for example, by moving focus from one button to another using the tab key or arrow keys). In these implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally a user-controlled user interface element (or touch on the touchscreen display) to communicate the user's intended interaction with the user interface (for example, by indicating to the device the user interface element that the user intends to interact with).For example, the position of a focus selector (e.g., cursor, touch, or selection box) over an individual button while pressure input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that individual button (rather than other user interface elements displayed on the device's display).

[0143] User interface and related processes Here, we draw our attention to embodiments of user interfaces ("UI") and related processes that may be implemented on an electronic device such as a portable multifunction device 100 or device 300, comprising a display, a touch-sensitive surface, one or more tactile output generators for (optionally) generating tactile output, and one or more sensors for (optionally) detecting the intensity of contact with the touch-sensitive surface.

[0144] Figures 5A to 5CY show exemplary user interfaces for updating state information within a session area according to several embodiments. Figures 6A to 6M show exemplary user interfaces for registering for biometric authentication according to several embodiments. Figures 7A to 7E show exemplary user interfaces for updating a session area to use a stored payment method according to several embodiments. Figures 8A to 8G show exemplary user interfaces for updating a session area for a virtual assistant according to several embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 9A to 9G, 10A to 10D, 11A to 11D, 12A to 12E, 13A to 13E, 14A to 14C, 15A to 15E, and 16A to 16F. For convenience of explanation, some embodiments are discussed with reference to operations performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally the contact of each finger or stylus, a representative point corresponding to the finger or stylus contact (e.g., the center of each contact or a point associated with each contact), or the center of two or more contacts detected on the touch-sensitive display system 112. However, similar operations are optionally performed on the display 450 and the device having separate touch-sensitive surfaces 451 in response to the detection of contact on the touch-sensitive surface 451 while displaying the user interface shown in the figure on the display 450.

[0145] Figures 5A to 5CY show exemplary user interfaces for updating state information within a session area according to several embodiments.

[0146] Figure 5A shows an exemplary user interface of the home screen (also called the home user interface) 501 of a portable multifunction device 100. In some embodiments, the portable multifunction device 100 is a computer system, a handheld mobile device, a tablet, or other client device. In some embodiments, the home screen user interface includes icons for navigating to multiple applications that are run or can be run by the device 100. In some embodiments, the user can interact with the device 100 using one or more gestures, including touch input. For example, a tap on an individual application icon launches the individual application on the display area of ​​the device 100 or otherwise opens the user interface of the individual application. In some embodiments, multiple views (also called pages) of the home screen user interface are available. For example, the user can swipe or otherwise navigate between multiple views, and multiple views of the home screen user interface, and in some embodiments, each view, include different application icons for different applications. In some embodiments, application icons are of different sizes, such as application widgets that display information about individual applications, and application widgets are larger than application icons.

[0147] In some embodiments, device 100 includes a session area 502-1 containing one or more sensors (e.g., a speaker 111 and / or one or more optical sensors 164). In some embodiments, one or more sensors are located within one or more notches (also called sensor areas) in the display of device 100. In some embodiments, session area 502-1 encompasses one or more sensor notches for one or more sensors. In some embodiments, additional sensors are located within session area 502-1, and the notches shown in Figure 5A contain one or more sensors within the notches (e.g., one or more additional sensors are located in the same notch as the speaker 111, and / or one or more additional sensors are located in the same notch as the optical sensors (one or more) 164, such as a structured optical emitter or projector). In some embodiments, it will be understood that alternative shapes and / or numbers of notches (e.g., three or more or less than two), as well as the number of sensors in individual notches, are implemented. In some embodiments, the notches are not visible from the surface of device 100. In some embodiments, the device displays an outline of session region 502-1. For example, the device displays a black session region 502-1 encompassing notches for speaker 111 and an optional sensor 164. In some embodiments, the device displays session region 502-1 by a color that matches or otherwise merges with the color of the sensor located within the notch.

[0148] In some embodiments, the area between two sensor notches is maintained in the same color as the sensor. For example, the area between two of the sensor notches includes a display that shows a color selected to match the hardware color of the sensor. In some embodiments, at least one of the sensor notches includes a camera as the sensor within the sensor notch. In some embodiments, the area between two of the sensor notches displays content (e.g., a privacy indicator and / or a lock indicator). Specific examples of the areas between sensor notches are provided in the drawings and accompanying description herein.

[0149] In some embodiments, the privacy indicator 518 is optionally displayed in the area between two of the sensor notches. In the example in Figure 5A, the privacy indicator 518 is displayed in the session area 502-1 in the area between the speaker 111 notch and the sensor(s) 164 notches. As will be described in more detail herein, the privacy indicator 518 typically indicates the current or recent use of the camera, microphone, and / or location sensor to capture information about other user-specific and / or user-identifying information by the device, the device environment, or applications running on the device 100. In some embodiments, displaying the privacy indicator to indicate the use of the camera, microphone, and / or location sensor is optional in applications where the session is not displayed in the session area. In some embodiments, the privacy indicator is displayed with increased brightness in certain situations, such as when the device 100 is in a bright environment (e.g., sunlight), to make the privacy indicator more visible to the user. In some embodiments, the privacy indicator has different colors for different types of data being captured (for example, if location data is being collected, the privacy indicator is a first color such as blue or purple; when visual data is being collected, the privacy indicator is a second color such as green, white, or yellow; and / or when audio data is being collected, the privacy indicator is a third color such as orange, red, or brown). In some embodiments, if multiple types of data are being collected simultaneously, multiple privacy indicators are displayed simultaneously. In some embodiments, if two or more types of data are being collected, a single privacy indicator is displayed (for example, if both visual and audio data are being collected, a privacy indicator indicating that visual data is being collected is displayed; or if both audio and location data are being collected, a privacy indicator indicating that audio data is being collected is displayed).

[0150] In some embodiments, the session area 502-1 is displayed with a shape that includes curvature matching the curvature of the device 100's display (e.g., the corner of the display), such as the shape of the corner of the touchscreen 112 in Figure 5A. In some embodiments, the session area maintains curvature matching the curvature of the display even when the session area is extended to show additional information and / or control of the session, as will be described later.

[0151] In some embodiments, session areas 502-1 displayed without an active session (e.g., without state information) and / or session areas described herein displayed with at least one active session (e.g., with state information) are displayed in predetermined locations on the display as the user navigates between different user interfaces. For example, the session areas are displayed within the same area of ​​the display, and device 100 displays an application user interface, a home screen user interface, and optionally a wake screen user interface (e.g., at the top of the touchscreen 112 as shown throughout the figure).

[0152] Figure 5B shows the home screen user interface described with reference to Figure 5A. Figure 5B further shows a session area 502-2 that does not contain one or more sensors. For example, in some embodiments, one or more sensors are not located within the notch of the display of device 100. In some embodiments, a session area 502-2 that does not contain one or more sensors is made capable of performing all of the functions described herein (e.g., any of the same functions described herein with respect to session area 502-1). Most examples described herein show one or more sensors in the session area, but in some embodiments, the session area is displayed whether or not one or more sensors are included in the session area.

[0153] Figure 5A shows user input 504-1, such as a tap input to select an icon 424 for a messaging application. User input 504-2 in Figure 5B is similar to user input 504-1 in Figure 5A. Depending on user input 504-1 (or user input 504-2), device 100 displays the user interface 500 for the messaging application, as shown in Figure 5C. As shown in Figure 5C, device 100 maintains the display of session area 502-1 (or similarly, session area 502-2) while the user navigates between user interfaces on device 100 (e.g., from the home screen user interface to the user interface for the messaging application). In some embodiments, while device 100 is displaying the user interface 500, device 100 receives a request to initiate a communication session. For example, device 100 receives an incoming call (e.g., from "Max"). In some embodiments, upon receiving an incoming telephone call, device 100 updates session area 502-1 to an extended session area 502-3, as shown in Figure 5D. In some embodiments, the extended session area 502-3 includes information about the incoming request and / or one or more controls for interacting with the incoming request. For example, session area 502-3 includes contact information associated with the incoming call (e.g., information indicating that the call is from "Max") and / or a stored contact photo or icon associated with the contact. In some embodiments, session area 502-3 further includes several control options, including an option 510 to accept the call and an option 508 to reject the call, as shown in the example in Figure 5D.

[0154] In some embodiments, the session area 502-3 is displayed with a visual effect on at least one side of the session area 502-3. For example, a shadow 514-1 is displayed on the session area 502-3. In some embodiments, the size and / or darkness of the shadow is selected according to the current size of the session area. While the session area is displayed at its minimum size (e.g., its minimum size) (e.g., session area 502-1), for example, when there is no active session associated with the session area, the session area is displayed without a shadow or other visual effect. In some embodiments, while the session area is minimized, the session area is considered an empty session area (e.g., even if some state information is displayed in a state area, if the displayed state area fits within a minimized state area, such as between sensor areas such as privacy indicator information or device lock / unlock state information). In some embodiments, the session area is displayed with a first size larger than its minimum size (e.g., a condensed size such as the size of session area 502-3, where a first set of information and / or interaction options of the active session are displayed), but the session area is displayed by a shadow 514-1, which has a first size and a first darkness level (or collectively, a first intensity). In some embodiments, the session area is displayed with a second size larger than the first size (e.g., an expanded size such as the size of session area 502-5 in Figure 5I, where a larger second set of information and / or interaction options of the active session are displayed), but the session area is displayed by a shadow 514-2, which has a second size and a second darkness level (or collectively, a second intensity), where the second size and / or second darkness level is greater than the first size and / or first darkness level (e.g., the second intensity is greater than the first intensity).

[0155] FIG. 5D further shows that while the session area 502-3 is displayed in an extended size, the extended size includes one or more controls and / or additional information of the current session (e.g., in the case of a phone call) associated with the session area, while a visual effect is applied to a portion of the display area outside the session area. For example, a blur visual effect 512 is applied to a portion of the user interface 500 to emphasize the session area 502-3 in that the session area 502-3 appears to be displayed over a visually unemphasized portion of the user interface 500. In some embodiments, the blur visual effect 512 is applied to the upper third or upper half of the user interface 500, or other ratio or percentage. In some embodiments, the blur visual effect 512 has a predetermined size (e.g., the size is maintained) regardless of the current size of the session area, including whether the session area is expanded or folded (or minimized). In some embodiments, the blur visual effect 512 varies in intensity based on the area of the user interface 500 to which it is applied. For example, in some embodiments, the blur visual effect 512 is a gradient effect that generates more blur in the user interface closer to the session area and less blur in the user interface farther from the session area. For example, the blur visual effect 512 gradually decreases the amount of blur in the user interface 500 as the distance from the session area increases. In some embodiments, the gradient of the blur visual effect 512 is applied (e.g., progresses) in one direction (e.g., downward) along the user interface.

[0156] In some embodiments, as the size of the session area increases, the blur visual effect 512 is displayed gradually (for example, as it moves outward and / or downward from the session area, and / or as its intensity increases over time, by progressing through a number of intermediate states over time). For example, while the session area is collapsed or minimized, the blur visual effect 512 is optionally not applied to the user interface surrounding the session area. In some embodiments, as the session area gradually expands, the blur visual effect 512 is optionally shifted on the display gradually (optionally at the same or proportional rate as the expansion of the session area), while animating outward from the collapsed session area to the expanded session area. In some embodiments, the blur visual effect 512 is optionally shifted downward from the top edge of the display. In some embodiments, as the session area collapses from its expanded size to its condensed size or its minimum size, the blur visual effect 512 gradually shifts away from the display by optionally sliding up and down the upper edge of the display and / or by decreasing its intensity to fade out.

[0157] In some embodiments, device 100 detects user input 516, such as a tap input or other gesture, to select option 510 for answering a call. In response to user input 516, the device establishes a communication session (e.g., between the calling user "Max" and the user of device 100), and automatically shrinks session area 502-3 to a condensed active session size, as shown as session area 502-4 in FIG. 5E, where the condensed active session size is a non-expanded view of the active session. However, the non-expanded view of the active session is expanded relative to a minimized (e.g., minimum) session area size (e.g., the size of session area 502-1). For example, in FIG. 5C, the session is not active, and thus, session area 502-1 is displayed at a minimized size that does not include information about the active session, while in FIG. 5E, there is an active session of the communication session (e.g., a call from "Max"), and the session area is expanded (e.g., not more expanded than in FIGS. 5D and 5I) relative to its minimized size to include information about the ongoing session (e.g., to the left and / or right, and / or above, and / or below).

[0158] In some embodiments, as shown in Figure 5C, one or more status indicators (e.g., current device time labeled “Time”, battery indicator 506, cellular network indicator 503, and / or Wi-Fi indicator 505) are displayed while the session area is minimized. In some embodiments, as the size of the status area increases from a minimum size (Figure 5C) to a condensed or expanded size (Figure 5D), one or more status indicators are rearranged on the display (e.g., to accommodate the increasing size of the session area). For example, when session area 502-1 (Figure 5C) is updated to session area 502-3 (Figure 5D), the “Time” indicator is optionally moved to the left, and the battery indicator 506, cellular network indicator 503, and / or Wi-Fi indicator 505 are optionally moved to the right. In some embodiments, the battery indicator 506, the cellular network indicator 503, and / or the Wi-Fi indicator 505 are compressed and displayed closer to each other when one or more of the battery indicator 506, the cellular network indicator 503, and / or the Wi-Fi indicator 505 are optionally stopped displaying.

[0159] In some embodiments, the display of one or more status indicators (e.g., a battery indicator 506, a cellular network indicator 503, and / or a Wi-Fi indicator 505) is stopped while the session area is being expanded. For example, one or more status indicators are removed from the display based on the current size of the session area. In some embodiments, for example, while the session area is expanded to cover the entire portion of the display where one or more status indicators are displayed, all of the status indicators are no longer displayed (e.g., as shown in Figure 5D). In some embodiments, the one or more status indicators to be removed are selected based on the current state of device 100, particularly in situations where fewer than one or more status indicators are removed. For example, the device can switch between a mode using cellular data and a mode using Wi-Fi. In some embodiments, indicators that match the current mode of device 100 continue to be displayed, and indicators for modes of device 100 that are not currently active are not displayed (e.g., removed). For example, in Figure 5E, the cellular network indicator 503 is displayed while the Wi-Fi indicator 505 is hidden, depending on the optional determination that device 100 is using a cellular network and is not connected to or actively using Wi-Fi (e.g., in the case of an ongoing phone call). In Figure 5F, the cellular network indicator 503 is hidden while the Wi-Fi indicator 505 is displayed, depending on the determination that device 100 is connected to a cellular network and / or using Wi-Fi and is not actively using the cellular network (e.g., in the case of an ongoing phone call). In some embodiments, when the device changes from being connected to Wi-Fi and / or actively using Wi-Fi to being connected to a cellular network and / or actively using the cellular network (or vice versa), the currently displayed status indicator is dynamically updated to reflect the current connection type.

[0160] In some embodiments, the number and spacing of status indicators change dynamically as the session area resizes (e.g., in Figure 5C, time is shown on the left side of the session area, and the cellular signal indicator, Wi-Fi indicator, and battery indicator are shown on the right side of the session area), and the number of status indicators changes as the session area expands (or additional session areas are added) (e.g., the cellular signal indicator is removed as shown in Figures 6E and 6F). In some embodiments, as the session area size changes, one or more status indicators gradually move to create more space (e.g., if the session area size increases) or to fill empty space (e.g., if the session area size decreases). In some embodiments, the movement of status indicators changes the spacing between status indicators (e.g., increasing or decreasing the space between adjacent status indicators). In some embodiments, if only a single status indicator is present, the status indicator moves to the left or right as the session area grows larger or smaller to maintain the status indicator approximately in the center of the area between the edge of the display and the edge of the session area closest to the status indicator. In some embodiments, the size of the session area changes gradually, and the corresponding changes in the number and / or intervals of the status bar indicators are also gradual. While specific examples of status indicators have been described above, it should be understood that a device can display a variety of status indicators in the status area, including indicators such as notifications, network activity, wireless connection, Bluetooth connection, headphone connection, location usage, and virtual private network connection status.

[0161] In some embodiments, in the condensed active session size shown in Figure 5E, session areas 502-4 contain information about the current session. For example, session areas 502-4 display information about an ongoing communication session. In some embodiments, the information includes indications of the application or software currently used in the active session. For example, a phone icon is displayed, and the duration of the ongoing call is shown. In some embodiments, an audio waveform is displayed to indicate incoming and / or outgoing audio information (e.g., it is part of a call). In some embodiments, different parts of the waveform (e.g., along one axis such as horizontal) represent different audio frequencies, and in some such embodiments, the height of a distinct part of the waveform (e.g., a distinct part) represents the amplitude of the audio signal at the frequency or frequency band corresponding to the distinct part of the waveform.

[0162] In some embodiments, the audio waveform includes at least two parts, including a first part (e.g., the leftmost or left-aligned part) and a second part (e.g., the rightmost or right-aligned part). In some embodiments, the first part displays incoming audio (e.g., audio received during a call), and the second part displays outgoing audio (e.g., audio detected by one or more microphones of device 100, transmitted to one or more other call participants during a call), or vice versa. In some embodiments, the first part of the audio waveform is displayed in a first color (or a first color range, such as the blue-green spectrum of colors), such as blue or green, and the second part of the audio waveform is displayed in a second color (or a second color range, such as the red-orange spectrum of colors), such as red or orange, which is different from the first color. Thus, the audio waveform visually distinguishes different audio sources (e.g., incoming audio data and outgoing audio data). In some embodiments, the audio waveforms overlap at least partially. In some embodiments, each audio waveform (for example, for incoming and outgoing audio) is assigned a different range of the waveform such that, when displayed simultaneously, the two audio waveforms overlap at least partially and not overlap at all. For example, incoming audio is displayed only or primarily in the left-aligned portion of the audio waveform, and outgoing audio is displayed only or primarily in the right-aligned portion of the audio waveform.

[0163] In some embodiments, the session area 502-4 optionally includes a privacy indicator 518 that is displayed in a display area between sensor notches (e.g., for speaker 111 and / or sensor 164) and indicates the current or recent use of a camera, microphone, and / or position sensor for capturing information about the device or the device's environment (e.g., as described herein with reference to Figure 5A). For example, the session area 502-4 may enclose each of the sensor notches and display one or more icons or other visual elements between the sensor notches (e.g., the area between sensors is part of the display of device 100). In some embodiments, while the session area displays an active session, the privacy indicator 518 is displayed outside the session area, for example, in the corner of the display (e.g., next to or above the battery indicator 506). In some embodiments, for example, in response to device 100 initiating the use of one or more sensors that trigger the display of the privacy indicator 518, the initial display of the privacy indicator 518 includes gradually changing (e.g., reducing) the degree of blurring of the privacy indicator 518 (e.g., progressing through a plurality of intermediate display states). In some embodiments, for example, in response to device 100 ceasing the use of one or more sensors that triggered the display of the privacy indicator 518, the deactivation of the display of the privacy indicator 518 is performed without gradually changing the degree of blurring of the privacy indicator 518 (e.g., the degree of blurring of the privacy indicator 518 is maintained regardless of whether other visual properties such as the opacity and / or brightness of the privacy indicator 518 gradually change as the display of the privacy indicator 518 is deactivated).

[0164] In some embodiments, the session area 502-4 updates the information displayed while the session remains active. For example, while the communication session is ongoing, the length of time the call has been ongoing is updated (e.g., from 0:08 in Figure 5E to 0:09 in Figure 5F), and the audio waveform is updated according to the current audio data. For example, if "Max" is providing audio data, the portion of the audio waveform for incoming audio is animated according to the audio data, but if the user of device 100 is not providing audio data (e.g., the user of device 100 is not speaking), the portion of the audio waveform for outgoing audio is displayed as flat (e.g., no waveform or a waveform with zero volume).

[0165] In some embodiments, in response to user input 520 (e.g., swipe input, drag gesture, button press, or other user input), device 100 stops displaying user interface 500 and displays home user interface 501. In some embodiments, the display of session area 502-4 is maintained when the user navigates between application user interfaces such as messaging application user interface 500 and other application user interfaces and / or other system user interfaces such as home screen user interface 501. Thus, the user can view updates about the ongoing session when the user interacts with other applications on device 100.

[0166] Figures 5G to 5H show the session area 502-5 displayed on device 100 in an embodiment in which device 100 is configured without one or more sensors within the session area 502-5. For example, the features and functions described above with reference to Figures 5E to 5F are also features and functions enabled for devices such as those shown in Figures 5G to 5H. Thus, the session area 502-5 displays the same information about the active session, whether or not there is one or more sensor notches and / or one or more sensors located within the sensor area. For example, the session area 502-5 in Figure 5G has the same appearance as the session area 502-4 in Figure 5E, and the session area 502-5 in Figure 5H has the same appearance as the session area 502-4 in Figure 5F.

[0167] In some embodiments, upon detection of user input 522 (or a similar user input 522-2, as shown in Figure 5H) such as a tap input, the session area 502-4 (or session area 502-5) extends to display additional information about the active session in session area 502-6, as shown in Figure 5I. In some embodiments, in addition to extending the display of the session area to session area 502-6, the device 100 provides non-visual outputs such as audio and / or haptic outputs. As described above, in some embodiments, the size and / or darkness of the shadow 514-2 is based on the size of session area 502-6. For example, a larger session area corresponds to a larger and / or darker shadow (as shown for session area 502-6) than the shadow of a smaller area (as shown for session area 502-3). In some embodiments, one or more session regions are displayed without shadows, for example, in a minimized session region and / or condensed session region associated with a particular application (e.g., session regions 502-1 and 502-4).

[0168] In some embodiments, visual effects such as a blurring visual effect 512 are displayed on the same portion of the currently displayed user interface 500 in Figure 5I as described with reference to Figure 5D.

[0169] Figure 5I shows an extended session area 502-6 that includes multiple control options for controlling the operation of software and / or applications associated with the current session. For example, during an ongoing communication session, depending on whether the user provides a first type of input (e.g., tap input 522) on the session area 502-4, multiple control options are displayed, including a control 530-1 for ending the communication session, a control 530-2 for starting a video communication session, and a control 530-3 for muting / unmuting the microphone of device 100. In some embodiments, additional and / or alternative control options are also displayed, such as a keypad control for entering numbers and / or a speaker control for switching the speakerphone on or off for the communication session. It will be understood that different applications and / or software associated with individual sessions will provide different sets of controls for individual sessions. For example, controls for controlling a communication session are provided in Figure 5I. In another example, controls for stopping and / or pausing a timer are provided during a timer session. In yet another example, controls for media playback are provided for a media player session. In some embodiments, in response to a user interacting with individual controls within the session area 502-6, device 100 performs actions corresponding to the controls. For example, in response to a user-selected control 530-3, device 100 may mute (if currently muted) or unmute (if currently unmuted) the microphone, as described with reference to Figures 5K-5M, and stop providing outgoing voice data to the communication session with "Max".

[0170] In some embodiments, the privacy indicator 518 is optionally displayed in a different location while the session area 502-6 is expanded compared to when the session area is condensed or minimized. For example, the privacy indicator 518 is optionally displayed at the edge of the expanded session area 502-6 or outside the session area 502-6 (e.g., in the background next to the battery status indicator, or in the upper corner of the display), and not between sensor notches. For example, Figure 5I shows a privacy indicator 518-a optionally and alternatively displayed outside the expanded session area 502-6 in the corner of the display area, as opposed to a privacy indicator 518 displayed at the edge of the expanded session area 502-6. In some embodiments, the session area is extended to a size that leaves a small area near the edge of the display unoccupied (for example, as shown in Figures 5D, 5AK, 5AM, and 5AY, in which case the privacy indicator is displayed on the top, left, or right side in Figure 5D or 5AY, or between the edge of the session area including the top edge and the edge of the display close to the session area in Figure 5AK or 5AM). In some embodiments, the privacy indicator 518 is optionally displayed in a different location while the session area is condensed (and yet another different location while the session area is extended) than while the session area is minimized. For example, in Figure 5J, the privacy indicator 518-b is optionally displayed outside the condensed session area 502-5, rather than inside the session area 502-5 (e.g., between the sensor areas), and optionally next to the session area 502-5 (e.g., to its right or left).

[0171] In some embodiments, while the session area 502-6 is expanded to display multiple control options, the user cannot interact with user interface elements displayed in the user interface 501 outside the session area 502-6. For example, user input 526 on an application icon 438 may optionally not launch the application associated with the icon 438. Instead, in some embodiments, user input 526 detected outside the session area 502-6 simply shrinks the session area 502-6, or otherwise reduces its size back to its condensed size as the session area 502-5, and optionally stops displaying any corresponding visual effects such as shadows and / or blurring effects. In some embodiments, the session area continues to update information related to the ongoing session, such as updating the length of time the communication session has been running, even as the size of the session area changes. In some embodiments, while the session area 502-6 is expanded, at least a portion of the content displayed in the user interface outside the session area 502-6 is at least partially obscured by a visual effect (e.g., a shadow and / or blur effect around the session area 502-6) and / or completely obscured by the session area 502-6. Thus, Figure 5J shows the session area 502-6 reduced to its condensed size as the session area 502-5 in response to user input 526, with the shadow 514-2 and blur effect 512 whose display has been stopped.

[0172] Figure 5J illustrates the detection of user input 528 that at least partially overlaps with session area 502-5. In some embodiments where one or more sensors are located within the session area, device 100 is capable of detecting user input occurring across one or more sensors and treating such user input as input directed to the session area. For example, if a user taps a portion of device 100 corresponding to at least one sensor within the session area, device 100 interprets the user input on the sensor as input directed to the session area. In some embodiments, session area 502-5 is extended to session area 502-6 (e.g., in Figure 5K) depending on the user input 528 directed at at least partially to session area 502-5, particularly to the sensor area within session area 502-5. In some embodiments, different actions are performed based on the type of user input 528 directed to session area 502-5. For example, in response to detection of user input directed to session area 502-5, if the user input is a tap user input, device 100 displays an application user interface corresponding to the active session displayed in session area 502-5 (for example, a call user interface displayed outside the session area instead of the home user interface 501), and if the user input is a long-press user input, device 100 displays an extended session area 502-6 (for example, without displaying an application user interface, while maintaining the display of the home user interface 501).

[0173] Figures 5K to 5M show examples of a user selecting one of the control options displayed in the expanded session area 502-6. In some embodiments, the user maintains the user input 528 shown in Figure 5J by continuing to touch the display of the device 100. For example, the user can start touching the user input 528 (Figure 5J) and, while maintaining contact with the display, move the contact to perform a drag or swipe gesture. In some embodiments, the session area 502-5 is expanded into the expanded session area 502-6 in response to a first portion of the user input 528 (e.g., the long-press portion), and the movement of the contact (e.g., the drag portion) occurs between a second consecutive portion of the user input 528 that immediately follows the first portion. In some embodiments, a position on the display where the user pays attention to the contact (also referred to herein as the lift-off of the contact) is selected. For example, in Figure 5K, the user drags the user input 528 to the position on the display corresponding to control 530-3 before the user lifts the contact 528-1 off control 530-3. In response to the user lifting contact 528-1 off control 530-3, device 100 mutes the microphone, stops providing audio output to the communication session, and therefore performs the actions associated with control 530-3, and optionally provides audio and / or haptic feedback in response to the lift-off of contact 528-1.

[0174] In some embodiments, as the user moves a contact around a display area, the device 100 optionally provides audio and / or haptic output to indicate that the user input is enabled to select a control option, depending on whether the contact partially overlaps with a control option, and to indicate that the control option is now selected for activation when the user input 528 is lifted off its current position. For example, in the process of moving to control 530-3 in Figure 5K, the contact 528-1 moves over the keypad control 528-4, and in response, the device 100 generates audio and / or haptic feedback indicating that the contact 528-1 is on an activatable control option. However, in scenarios where the contact 528-1 is not released from the keypad control 528-4, the keypad is not displayed in response to the user input 528. For example, when a contact from user input 528 moves through the session area 502-6, device 100 provides non-visual outputs (e.g., audio and / or haptic outputs) to alert the user when the contact moves over any of the control options (e.g., each time the contact moves over a particular control option). For example, a first non-visual output is generated in response to user input 528 moving over a first control option, followed by a second non-visual output generated in response to user input 528 moving away from the first control option and over a second control option (or, in some embodiments, moving away from the same first control option and then back).

[0175] Figure 5L shows an example where a user lifts off contact 528-2, which was initiated as user input 528 to control option 530-2 for starting a video communication session. In response to the user lifting off contact 528-2 over control option 530-2, device 100 starts a video call. Similarly, in Figure 5M, in response to the user releasing contact 528-3 while it is in a position overlapping with control option 530-1 for ending a communication session, device 100 ends the ongoing communication session (e.g., "Max" and hangs up).

[0176] In some embodiments, the session area 502-1 is redisplayed as shown in Figure 5N, in accordance with the determination that the ongoing session has ended, for example, by ending a call as shown in Figure 5M. In some embodiments, the session area 502-1 is maintained on the display while the device 100 is on and active (e.g., not in sleep or low power state). Thus, even after the user has ended the communication session that was displayed in the session area 502-6, the device 100 continues to display the session area 502-1 by reducing it to a minimized state where there are no active sessions.

[0177] Figure 5N shows the detection of user input 532 on the session area 502-1. In some embodiments, in response to user input 532, the session area 502-1 is expanded according to the determination that an active session is not currently associated with the session area 502-1, and the expanded session area 502-1b shown in Figure 5O optionally expands the background color of the session area (e.g., black) in all directions. In some embodiments, the session area 502-1 expands in one direction more than the other, or in one direction but not the other. For example, the session area expands horizontally but not vertically (e.g., while device 100 is in portrait mode as shown in Figure 5O), or more generally, it becomes longer (e.g., by a longer dimension) with little (or no) change in width (e.g., a shorter dimension). In some embodiments, the session area expands to session area 502-1b over a predetermined period (e.g., 0.2 seconds, 0.5 seconds, 1 second, 5 seconds, 10 seconds, 30 seconds, or 60 seconds). In some embodiments, after expanding to session area 502-1b, the session area automatically shrinks back to its minimized size as session area 502-1 in Figure 5P (e.g., upon expiration of the predetermined period).

[0178] In some embodiments, the session area 502-1 expands to the session area 502-1b in direct response to the initial detection of the user input 532. In some embodiments, the session area 502-1 expands to the session area 502-1b according to a determination that the user input 532 is in progress (e.g., being maintained) over a threshold time (e.g., the user input 532 is an input such as a press-and-hold input maintained for at least 1 second, 2 seconds, 3 seconds, or 5 seconds). In some embodiments, the session area 502-1b optionally shrinks or reduces in size according to the end of the user input 532 (e.g., in response to the lift-off of the user input 532). In some embodiments, the session area 502-1b optionally shrinks regardless of the state of the user input 532. For example, even if the user input 532 is still in progress, the session area 502-1b shrinks after a predetermined period has elapsed (e.g., after the initial detection of the user input 532, or after the session area 502-1 has expanded to the session area 502-1b). In some embodiments, in response to the user input 532 and optionally according to a determination that the user input 532 continues for a threshold time, the device 100 optionally outputs non-visual feedback such as haptic output and / or audio output (e.g., as shown by the output 533 in FIG. 5O). In some embodiments, if there is an active session associated with the session area and a user input is detected in the session area, the device 100 optionally provides different non-visual feedback (e.g., different haptic outputs and / or different audio outputs), or provides haptic and / or audio outputs while providing additional information about the active session as described with reference to FIGS. 5AL-5AM without providing non-visual feedback.

[0179] Figure 5P shows user input 534, such as a tap input or press-and-hold gesture, detected on the icon 440 for the clock application. In some embodiments, in response to the detection of user input 534, the device 100 displays a user interface 536-1 for the clock application, as shown in Figure 5Q. In some embodiments, the user interface 536-1 provides the user with the option to set a timer on the device 100 and select a voice or other notification to alert the user when the timer is complete. Figure 5Q shows the user starting a 15-minute timer by selecting the start button on the user interface 536-1 via user input 538. In some embodiments, the session area 502-1 remains displayed while the user interacts with the user interface for the clock application, as shown in Figure 5Q. In Figures 5Q to 5R, the session area 502-1 is not associated with a currently active session and therefore appears as a solid background (e.g., black or another solid color).

[0180] Figure 5R shows a second user interface 536-2 for a clock application, which includes a timer countdown display. In some embodiments, device 100 detects user input to switch from the display of the second user interface 536-2 for the clock application to another user interface. For example, the user can navigate from an application user interface for a first application to an application user interface for a second application and / or a system user interface (e.g., the home screen and / or wake screen). In some embodiments, a swipe gesture such as input 538 in a specific part of the display (e.g., the lower edge in the horizontal direction (e.g., left or right)) allows the user to navigate between application user interfaces of applications currently open or running in other ways on device 100. For example, as shown in Figures 5R to 5S, the user switches from the second user interface 536-2 for the clock application to a user interface 500 for a messaging application. In some embodiments, the application user interface displayed in response to the application switching input 538 is the user interface of the most recent previously opened application, or the user interface of the next (or previous) application in a series of open or recently opened applications after the clock application (e.g., a messaging application or another recently opened application).

[0181] Figure 5S further illustrates that, in response to a user input 538 that causes device 100 to stop displaying the second user interface 536-2, information about an ongoing timer is displayed in the session area 502-7 while the user interface 500 is displayed on the rest of the display (for example, because the information is no longer visible in the main application area). In some embodiments, the timer is one of several event types associated with an application displayed in the session area 502-7. For example, as will be described in more detail below with respect to Figure 5BL, the session area displays continuous updates of several session types while the session is active and / or ongoing, including an active timer session. In some embodiments, the session area 502-7 displays an application icon (e.g., a timer icon to the left of the sensor) that identifies the application associated with the active session, and displays a first set of information about the active session (e.g., to the right of the sensor), such as a countdown of the remaining time on the timer (e.g., "14:49" in Figure 5S). In some embodiments, as described above, the session area is large enough to display information about the currently active session, but if the session area is extended to occupy at least a portion of the display area where the status indicators are displayed (e.g., the upper right area of ​​the display in Figure 5S), one or more status indicators are removed (e.g., the cellular connection status indicator is not displayed in Figure 5S). In some embodiments, all status indicators are optionally removed (e.g., according to a determination that the session area is extended horizontally to overlap with all status indicators, such as by overlapping at least a portion of all status indicators).

[0182] In some embodiments, the session area 502-7 remains visible as the user navigates between other application user interfaces. For example, in Figure 5S, user input 540 (e.g., a left-to-right edge swipe gesture) is detected, and in response to user input 540, device 100 displays the user interface 542 of another application (e.g., a web browser), as shown in Figure 5T, and optionally, the application is the most recently opened on device 100 or the next (or previous) application in a series of open applications after the messaging application. As shown in Figure 5T, the session area 502-7 continues to update by changing the remaining time on the timer (e.g., "14:48" in Figure 5T).

[0183] In some embodiments, as shown in Figure 5T, in response to a user input 544 (e.g., a swipe-up gesture initiated at the bottom or bottom edge of the display or other system navigation input or home navigation input), the device displays the home screen user interface 501 to navigate to a system user interface (e.g., the home screen user interface) and continues to update the active timer session in session area 502-7, as shown in Figure 5U, which indicates that the remaining time of the timer has changed further (e.g., to "14:47"). In some embodiments, the user can navigate to other system user interfaces, such as another view of the home screen user interface 501-2, via a left swipe input 546, as shown in Figure 5V. As the user navigates between system user interface views, session area 502-7 is maintained and updated with the time remaining in the active timer session (e.g., "14:46" in Figure 5V). As shown throughout the figures, the remaining time in the active timer session continues to count down until the timer expires (e.g., as shown in Figure 5AY) or is aborted.

[0184] In some embodiments, device 100 detects user input 548 to enter an application switching view that displays multiple representations of applications corresponding to, for example, currently running in the background and / or recently opened applications on device 100. In some embodiments, user input 548 is a swipe gesture in a first direction (e.g., up, down, left to right, or right to left) which is optionally initiated at the edge of the display. In some embodiments, in response to user input 548, an application switching user interface 550 is displayed, as shown in Figure 5W. In some embodiments, in the application switching user interface 550, the background of the application switching user interface 550 is hidden (e.g., blurred, darkened, or otherwise not visually highlighted).

[0185] As shown in Figure 5W, in some embodiments, the session area 502-7 is optionally maintained while the application switching user interface 550 is displayed, and continues to be updated, as indicated by the amount of time it remains dropped (e.g., to "14:38" in Figure 5W). In some embodiments, the session area 502-7 is not displayed while the application switching user interface is displayed. In some embodiments, the application switching user interface allows the user to select which applications to display (e.g., by tapping on a representation of an individual application) and / or navigate between other open applications (e.g., by swiping across an open representation of an application to view additional representations of an application). In some embodiments, a border 552 (sometimes called a keyline) is displayed around the edges of the session area 502-7 while the background of the area behind the session area 502-7 is visually de-emphasized. In some embodiments, the border 552 is displayed in a color selected according to the application associated with the current session. For example, in a timer session, boundary 552 is orange (for example, in a session of another application such as a call, the boundary is green, and / or in a map navigation session, boundary 552 is blue). In some embodiments, the color of boundary 552 is selected according to the content being updated in the session area. For example, in a media playback session, the color of boundary 552 is selected based on the color associated with the media item currently playing (or currently selected; if playback is paused or has not yet started). In some embodiments, the color associated with the media item currently playing includes a color sampled from the media item's cover art (for example, from the album cover of the current song, or from the book cover of the current audiobook).Therefore, for the same application having an active session represented in the session area, the session area boundary 552 optionally changes according to the content currently presented or updated by the application within the session area.

[0186] In some embodiments, the boundary 552 is present in all views of the session area while the session area is associated with an active session. For example, in some situations, the boundary 552 is present but displayed in a color similar to the background color of the user interface surrounding the session area. Thus, in some user interfaces, the boundary 552 is not conspicuously displayed to the user (e.g., the boundary blends into the background).

[0187] In some embodiments, the border 552 highlights the display of the session area 502-7 so that it is depicted differently from the background (for example, even if the fill color of the session area is the same as the fill color of the surrounding user interface just outside the session area). In some embodiments, the border 552 is displayed to distinguish the session area while it is expanded (e.g., to include state information for one or more applications) and is not displayed while it is minimized (e.g., so that the session area optionally merges into the background). For example, as will be described in more detail below, in dark mode and / or while displaying a user interface with a dark background having a similar color to the session area, the session area is displayed with a more prominent keyline.

[0188] In some embodiments, device 100 detects user input 554 coming out of the application switching user interface 550, such as a tap input (or directed to other selection inputs) on the background portion of the application switching user interface 550 that is not associated with an application representation. In response to user input 554, device 100 updates the display to return to a system user interface (e.g., the home screen user interface immediately before navigating to the application switching user interface 550, or optionally, the second page 501-2 of the same home screen as in Figure 5V), as shown in Figure 5X. In some embodiments, the session area 502-7 remains displayed along with the updated remaining time for the active timer session (e.g., further reduced to "14:34" in Figure 5X). In some embodiments, the user interacts with the home screen user interface 501-2 via user input 556 (e.g., swipe or other navigation input) in Figure 5X, for example, to navigate to a first view of the home screen user interface 501 (e.g., from left to right), as shown in Figure 5Y, and the session area 502-7 continues to update (e.g., to "14:28"), as indicated by the further reduced remaining time in Figure 5Y.

[0189] Figure 5Y shows a number of distinct inputs that can be directed to different user interface objects displayed on the home screen user interface 501. In some embodiments, each input corresponds to a request to open an application associated with a specific application icon selected by the input. For example, user input 558 on icon 424 for a messaging application opens a user interface for a messaging application (e.g., user interface 500), user input 560 on icon 426 for a calendar application opens a user interface for a calendar application, user input 562 on icon 446 for a settings application opens a user interface having a menu for changing several settings of device 100, and user input 564 on (or directed to) icon 422 for a music application opens a user interface for a music application. In some embodiments, when the user navigates to different user interfaces for different applications, the session area 502-7 remains displayed and is continuously updated with the length of time remaining in the current timer session.

[0190] In some embodiments, as will be described in more detail below, the user can initiate additional sessions by interacting with individual applications, and the session area is updated to reflect two or more currently active sessions simultaneously. For example, as described above, user input 564 on icon 422 for the music application causes device 100 to display the user interface 566 for the music application shown in Figure 5Z.

[0191] Figure 5Z shows a user input 568, such as a tap input, that requests the user to start playing a media content item (for example, by selecting the “Listen Now” button). In some embodiments, the user interface 566 of the music application is updated to user interface 570 to display information about the media content item currently playing, as shown in Figure 5AA. In some embodiments, the media content item currently playing (e.g., a song, audiobook, and / or video), or general media content playback from the music application, corresponds to an active session, and therefore, if the user navigates away from browsing the active media playback session in the music application user interface (e.g., navigating away from user interface 570), the session information about the active session is moved to session area 502-7. In some embodiments, the request to navigate away from user interface 570 is a request to display another application user interface, such as the mail application user interface 574, via user input 572. In some embodiments, user input 572 is a swipe gesture along a separate part of the display (e.g., the bottom edge or side).

[0192] Figure 5AB shows that the session area displays information for at least one active session while displaying the user interface of an application such as the mail application user interface 574, which does not correspond to an active session or is different from at least one other application that has an active session. In the example of Figure 5AB, the session area displays information about two currently active sessions, including an active timer session (started as shown in Figures 5R to 5S) and an active media playback session (started, for example, while displaying the music application as shown in Figures 5Z to 5AB). In some embodiments, while two or more sessions are active simultaneously, the session area 502-7 is separated into two parts: a first part for the timer session area 502-8 and a second part for the media playback session area 502-9, and optionally, each part is of a different size and / or at least one part includes one or more sensor cutouts. For example, to fit information for both sessions on the display simultaneously, the timer session information is shifted to the right of the sensors, and the media playback session information is in a separate user interface element to the right of the sensors. In some embodiments, at least one of the parts of the session area includes one or more sensors. For example, in Figure 5AB, the first portion of session area 502-8 encompasses one or more sensor notches. In some embodiments, the second portion of session area 502-9 optionally encompasses one or more sensor notches. In some embodiments, one sensor notch is encompassed by the first portion of session area 502-8, and the other sensor notch is encompassed by the second portion of session area 502-9. In some embodiments, for two active sessions, the session area is divided into three portions, and information about the currently active session is displayed in separate areas for each active session, one to the right and one to the left of the third portion of the session area that encompasses one or more sensor notches.For example, a third portion of the session area that encompasses one or more sensor notches (e.g., the central portion of the session area between the left and right active session portions) will not display active session information.

[0193] In some embodiments, while a session is active in the session area, the session area is displayed with an animation that periodically expands and contracts in size. In some embodiments, while the session area is displayed as a condensed version of the session area, and / or while the session area is displayed as an expanded version of the session area (e.g., one or more activatable control options displayed within an expanded session area such as session areas 502-3 and 502-6), the session area optionally expands and contracts periodically. In some embodiments, as shown in Figure 5AB, the dotted line indicates the outer limit of the animation in which the size of a corresponding individual part of the session area increases before contracting back to the size of the individual part as shown in Figure 5AB. In some embodiments, the animation of expanding and contracting (also referred to herein as vibrating) a separate part of the session area is repeated periodically (e.g., every 0.01, 0.05, 0.1, 0.2, 0.5, 1, 2, or 5 seconds) while the session is active. In some embodiments, a periodic vibration animation expands the background (e.g., background color) of individual parts of the session area without displaying additional information in the expanded area, and optionally without maintaining the information at the same size while the background expands. In some embodiments, there are two or more parts of the session area, which optionally expand and contract at different times, at different speeds, and / or according to different vibration patterns. For example, as shown in Figure 5AC, the first part of session area 502-8 expands to its expanded size (e.g., the size shown by the dashed line in Figure 5AB). In some embodiments, while the first part of session area 502-8 is expanding, the second part of session area 502-9, corresponding to a different session (e.g., a media playback session), is not expanded. In some embodiments, when the first part of session area 502-8 shrinks to its size shown in Figure 5AB, the second part of session area 502-9 expands to its expanded size shown by the dashed line in Figures 5AB to 5AC.In some embodiments, the first portion of session region 502-8 and the second portion of session region 502-9, as well as optionally additional portions of the session region, expand and contract periodically at different times. For example, the session region resizes in a time-shifted manner, as shown in Figures 5AE to 5AF, when the second portion of session region 502-9 is not expanded, the first portion of session region 502-8 is expanded, and vice versa. In some embodiments, each portion of the session region is animated according to the same or different time periods. For example, the first portion of the session region completes its expansion and contraction animation every 0.5 seconds, and the second portion of the session region completes its expansion and contraction animation every 0.6 seconds. In some embodiments, the animations of different portions of the session region may have the same phase or be out of phase (for example, two animations of the same or approximately the same duration may start earlier than the other).

[0194] In some embodiments, the vibration speed of the session area is based on the individual application to which the session area is associated. In some embodiments, the session area is associated with two or more different applications represented in separate parts of the session area, but the vibration speed of the separate parts is based on the individual application to which that part is associated. For example, the vibration speed is based on the speed of the animation of user interface elements displayed in the session area of ​​the individual application. In some embodiments, the vibration speed is based on the urgency of the session. For example, for alerts that are time-sensitive or otherwise considered urgent, such as alerts for incoming communication sessions or alerts for navigation commands for map sessions, the vibration speed is optionally faster than the vibration speed for non-urgent alerts, such as ongoing voice recording sessions. In some embodiments, a first type of session associated with a first application vibrates the session area (e.g., ongoing communication sessions, timers, and / or session mapping), while a second type of session associated with a second application does not vibrate the session area (e.g., low battery alerts and / or silent mode alerts).

[0195] In some embodiments, the vibration velocity while the session region is expanded differs from the vibration velocity while the session region is condensed for an active session. For example, in some embodiments, both the condensed and expanded session regions vibrate. In some embodiments, the expanded session region has a slower vibration velocity than the condensed session region. In some embodiments, the expanded session region does not vibrate selectively, while the condensed session region does. In some embodiments, the session region does not vibrate according to the determination that the session region is minimized (for example, as shown in Figure 5A, there is no active session displayed in the session region, or the state information displayed in the session region fits into a minimized state region, such as between sensor regions). In some embodiments, the session region vibrates if the session region is minimized but contains state information that fits into a minimized state region (for example, a privacy indicator or lock indicator displayed between sensor regions).

[0196] In some embodiments, in response to user input 576 (Figure 5AC), such as a tap input or long-press gesture, detected in the first portion of session area 502-8, the first portion of session area 502-8 expands to session area 502-10, as shown in Figure 5AD, to include additional information about the active session, including one or more controls of the active session. For example, session area 502-10 provides control options for pausing a timer and control options for canceling a timer to end an active timer session. In some embodiments, in response to user input to individual controls (e.g., tap input or other selection input), actions corresponding to individual controls are performed. In some embodiments, animations are provided to display session area 502-10 gradually expanding from the first portion of session area 502-8 through a plurality of intermediate display states between the display of session area 502-8 and the display of session area 502-10. In some embodiments, the size of the expanded session area 502-10 covers the portion of the display where the second portion of session area 502-9 is displayed. In some embodiments, the second portion of session area 502-9 stops displaying until the optionally expanded session area 502-10 shrinks to the size of the first portion of session area 502-8. In some embodiments, as shown in Figures 5AD to 5AE, if no user input is detected in session area 502-10, session area 502-10 automatically reverts to the size of the first portion of session areas 502-8 after a predetermined period (e.g., 1, 3, 5, 15, 30, or 60 seconds).In some embodiments, the session area 502-10 reverts to the size of the first portion of the session area 502-8 in response to detection of user input outside the session area 502-10 (which, in some embodiments, is treated not only as user input to exit the expanded session area 502-10 and return to the condensed session area 502-8, but also as user input interacting with the corresponding portion of the user interface displayed outside the session area (e.g., the mail application user interface 574 in Figure 5AD)).

[0197] Figure 5AD further illustrates user input 578 in response to a request to display the user interface of another application. In some embodiments, user input 578 is a drag gesture along the bottom edge, a swipe gesture, a tap input, or another user input for navigating between user interfaces on device 100. In response to user input 578, device 100 displays the user interface 542 for a web browser application. As described above and further illustrated by Figures 5AE to 5AF, in some embodiments, after changing the display of a user interface displayed outside the session area, the first portion of session area 502-8 and the second portion of session area 502-9 remain displayed. For example, Figure 5AE shows a user input 580, such as a swipe up (or other home navigation input such as a button press), to return to the view of the home screen user interface 501-2 (for example, from the scenario in Figure 5Y, as shown in Figure 5X, the user swipes left to switch from the home screen user interface 501-1 to the home screen user interface 501-2, and then, as shown in Figure 5Z, provides a user input 564 on the music application icon 422 to open the music application user interface 566). In some embodiments, while the home screen user interface 501-2 is displayed in Figure 5AF, the first portion of session area 502-8 and the second portion of session area 502-9 continue to be displayed, with each portion of the session area being animated to periodically expand and contract.

[0198] Figure 5AF shows user input 582 selecting an application icon to open the voice memo application. In response to user input 582, the device displays a user interface 584 (Figure 5AG) for recording voice memos in the voice memo application. In some embodiments, the user interface 584 includes a dark (e.g., black, dark gray, or dark blue) background with bright and / or brighter luminescent colors (relative to the background) for foreground elements such as text, buttons, and status indicators such as time and battery indicators. In some embodiments, one or more visual properties of the respective boundaries of each part of the session area are changed (e.g., compared to when the portion of the display surrounding the session area has a luminance exceeding a threshold amount), without changing the content displayed within the session area (e.g., the inner portion of the session area is maintained with a color that matches the color of one or more sensors), according to the portion of the display surrounding the session area, including the first portion of the session area 502-8 and the second portion of the session area 502-9, which have a color that satisfies a luminance threshold amount (e.g., has a luminance below a threshold amount).

[0199] For example, when the user interface is displayed with dark user interface elements surrounding session areas 502-8 and 502-9 and / or when device 100 is in dark mode, the boundary 585c of session area 502-8 and / or the boundary 585d of session area 502-9 will appear with a more prominent visual effect than when the user interface is displayed with light user interface elements surrounding session area 502 (e.g., including portions 502-8 and 502-9) and / or when the device is in light mode. For example, while the user interface is displayed against a light background, the boundary 585a of session area 502-8 and the boundary 585b of session area 502-9 will appear with a less prominent visual effect than when the user interface is displayed against a dark background, and the visual effect will include a thinner, more transparent and / or fainter boundary.

[0200] In some embodiments, the respective borders of session regions 502-8 and 502-9 are displayed or updated according to the current size of the session region. For example, while the session region is minimized, the border is optionally displayed without a distinct color and / or contrast (for example, instead, the border is displayed as a small number of pixels with a color matching the background color of the session region to display sharp edges), and while the session region is expanded, the border is optionally displayed with a colored and / or more prominent key line.

[0201] In some embodiments, the boundaries of individual session regions change over time. For example, the boundaries of session regions are optionally animated to periodically increase and / or decrease in thickness according to the vibration velocity of the session regions.

[0202] Figure 5AG shows user input 586 selecting a button to start recording a new voice memo. In response, recording a new voice memo is started by an updated voice memo user interface 584 and a privacy indicator 518 displayed in the corner of the display instead of the battery indicator 506, as shown in Figure 5AH. The privacy indicator 518 is displayed to indicate the current or recent use of the camera, microphone, and / or location sensors for capturing information about the device or the device's environment. In the example in Figure 5AH, the privacy indicator 518 indicates at least the current use of the microphone for recording the voice memo. In some embodiments, in situations where the session area is empty while voice recording is in progress, the privacy indicator 518 is displayed between sensor areas within the session area. In some such situations, according to some embodiments, no boundary is displayed around the session area (for example, since a session area where only the privacy indicator 518 is displayed is considered empty), as shown by the session area 5041 (Figure 5BL). On the other hand, according to some embodiments, a boundary is displayed (for example, since a session area where only the privacy indicator 518 is displayed is not considered empty). In some embodiments, while a voice memo is being recorded, the user requests to navigate to another user interface and minimize and / or close the voice memo application's user interface 584. For example, user input 588 in Figure 5AH corresponds to a swipe-up gesture to return to the home screen user interface.

[0203] In some embodiments, in response to user input 588, device 100 displays the home screen user interface 501. While Figure 5AI and other figures below show the home screen user interface 501, it should be noted that in some embodiments, device 100 returns to the home screen user interface 501-2, which was the last home screen user interface displayed (for example, immediately before the voice memo application is displayed, as in Figure 5AF). In some embodiments, the session area displays indications for up to two active sessions simultaneously. For example, after a user starts voice recording, there are three active sessions running on device 100: a timer session, a media playback session, and a voice recording session. In some embodiments, applications associated with active sessions are ranked according to relative priority. In some embodiments, the priority is determined automatically by device 100, for example, communication sessions and map navigation sessions are given higher priority than timer sessions. In some embodiments, the user can change the priority by selecting which application sessions have a higher priority than other application sessions (e.g., via a configuration application, configuration files, assigning higher priority to individual applications, assigning individual applications to one of several predetermined priority levels, or other methods of implementing user-defined priorities). In some embodiments, if the session area displays only up to two active sessions simultaneously, the top two application sessions with the highest priority are displayed in the session area (e.g., as two separate parts of the session area). For example, in Figure 5AI, the audio recording session 502-10 has a higher priority than the timer session 502-8 (e.g., the media playback session 502-9 has a higher priority than the timer session 502-8).Therefore, the portion of the session area that displayed timer session 502-8 is no longer displayed, and instead the session area displays information for audio recording session 502-10 (for example, together with media playback session 502-9). Additionally, the audio recording session area 502-10 displays a privacy indicator 518 (for example, due to the use of a microphone for the recorded voice memo).

[0204] In some embodiments where the assignment of active sessions to separate parts of the session area is for up to two active sessions, a third newly displayed session is placed in place of the lowest-priority session, regardless of whether the newly displayed session has a higher or lower priority than the other displayed sessions, based on priority (e.g., higher-priority sessions are displayed in the left part of the session area, and lower-priority sessions are displayed in the right part of the session area). For example, if media playback session 502-9 is the lowest-priority session and audio recording session 502-10 is the highest-priority session, then to reduce the amount of shuffling in the session area, audio recording session 502-10 is placed in place of media playback session 502-9 in the right part of the session area, while timer session 502-8 continues to be displayed in the left part of the session area (even though audio recording session 502-10 has a higher priority than timer session 502-8).

[0205] In some embodiments, as shown in Figure 5AJ, device 100 is capable of displaying up to three active sessions simultaneously in the session area. In some embodiments, the portion of the session area shrinks while an additional session portion is displayed (for example, the portion when two sessions are displayed is smaller than the portion when one session is displayed, and optionally, the portion when three sessions are displayed is smaller than the portion when two sessions are displayed). For example, Figure 5AJ shows three portions of the session area displaying information for each of three active sessions: timer session 502-13, media playback session 502-12, and audio recording session 502-11. In some embodiments, the arrangement of the three portions is based on session priority (for example, the highest-priority application session is on the left, and the lowest-priority application session is on the right). In some embodiments, the arrangement of the three portions is based on the order in which the sessions became active (for example, the application with the longest-running session is displayed on the far right or far left). In some embodiments, the arrangement of the three parts of the session area is determined to minimize the movement of application sessions that were already visible before the third session became active (for example, a media playback session is kept at the far right because a media playback session had previously been visible there, and / or a timer session is kept at the far left or in the center because a timer session had previously been visible there).

[0206] In some embodiments, one or more status indicators are optionally removed according to the determination that the session area and / or a portion of the session area occupies more space (e.g., horizontally) as additional active sessions are added. For example, the session area requires additional space on the display, causing the display of the time and / or battery indicators to stop, and as a result, the session area can be extended closer to the edges of the display (e.g., in portrait mode, extending closer to the vertical edge while occupying more space along the horizontal edge). For example, as shown in Figure 5S, the cellular network indicator 503 (Figure 5R) is hidden by the addition of a timer session that extends the session area from session area 502-1 to session area 502-7 (e.g., by at least temporarily stopping its display). In another example, as shown in Figure 5AB, the Wi-Fi indicator 505 (Figure 5AA) is hidden by the further addition of a media playback session 502-9 to the session area. In addition, in Figure 5AI, the privacy indicator 518 is displayed in the voice memo session area 502-10, whereas in Figure 5AJ, the privacy indicator 518 is displayed in the corner of the display (e.g., the upper right corner), outside the voice recording session area 502-11 (e.g., due to insufficient space within the voice recording session area 502-11, and / or due to the privacy indicator 518 not being related to the media playback session area 502-12 and therefore not being displayed between the sensor areas within the media playback session area 502-12), replacing the battery indicator 506 (e.g., due to insufficient space at the top of the display outside the session area portion 502, and / or, in some embodiments, due to the privacy indicator 518 being related to two or more displayed session area portions). In some embodiments, as shown in Figure 5AJ, one or more additional status indicators (e.g., current time and / or battery indicator) are hidden by adding a third active session (e.g., a voice recording session) to the status area.In some embodiments, the session area 502-14 of the current media playback session is expanded to occupy all or substantially all of the width of the display, as shown in Figure 5AK. When the session area is further expanded (for example, to include one or more activatable control options), all status indicators, including the time, battery indicator, and privacy indicator 518, are no longer displayed.

[0207] In some embodiments, as shown in Figure 5AK, the session area automatically expands from the sensor notch (e.g., downward and / or outward for a session area located near the top center of the display) in response to an event occurring in one of the active sessions, without receiving one or more user inputs optionally directed at the session area. For example, in a media playback session, the media content item being played (indicated by the "A" cover art in session area 502-9 in Figure 5AI and session area 502-12 in Figure 5AJ) finishes playing, and the next media content item (e.g., "Song B") begins playing. In some embodiments, an event that automatically starts playback of a different media content item automatically expands the session area to display additional information about the session corresponding to the event. For example, session area 502-14 includes additional information about the playback of "Song B" and includes control options for controlling the session (e.g., skip back, pause, and skip forward). In some embodiments, the display of the expanded session area 502-14 is animated as a transition from session area 502-8 expanding outward.

[0208] In some embodiments, after a predetermined amount of time has elapsed, or after user input is detected outside the session area 502-14, the expanded session area 502-14 automatically shrinks to the session area 502-9 shown in Figure 5AL. In some embodiments, as the expanded session area shrinks to a smaller size, the content displayed in the expanded session (e.g., session information and / or controls) fades gradually at a selected rate while optionally shrinking, so that the content disappears before it overlaps with the sensors. In some embodiments, the content displayed in the session area 502-14 is placed within the session area 502-14 to avoid overlapping with one or more sensor notches. More generally, for a session area encompassing one or more sensor notches, the active session content displayed in the session area is placed within the session area to avoid overlapping with one or more sensor notches.

[0209] As illustrated, session area 502-9 is updated to include current playback information about the media playback session (for example, the "B" cover art associated with "Song B" is displayed in session area 502-9 instead of the "A" cover art associated with "Song A" which was displayed in session area 502-8 in Figure 5AI and session area 502-9 in Figure 5AJ). Session area 502-10 also continues to update with current information about the active session, for example, session area 502-10 shows that the audio recording has progressed from 0:32 in Figure 5AI to 0:36 in Figure 5AL.

[0210] Figure 5AL shows a user input 590 directed to the session area 502-9. In some embodiments, depending on the type of user input 590, the device 100 performs different sets of actions. For example, in response to a first type of user input, such as a long-press user input in which the user maintains contact with the display for a threshold time, as shown in Figure 5AM, the session area 502-9 expands to display additional information about the session in the session area 502-14 without opening the application associated with the session, or to display a user interface for the application associated with the session outside the session area, and the expanded session area 502-14 includes one or more selectable controls for performing respective actions associated with media playback in response to user input directed to individual selectable controls (e.g., to play, pause, skip back, skip forward, and / or scrub the currently selected media). In some embodiments, the session area 502-9 expands without providing tactile and / or audio output. In some embodiments, as shown in Figure 5AN, in response to a second type of input such as tap input, the application user interface 570 of the application associated with the session area 502-9 is displayed without optionally displaying the expanded session area 502-14 (for example, without displaying the user interface shown in Figure 5AM) (for example, in response to a second type of user input 590 in Figure 5AL, the device 100 transitions directly from Figure 5AL to Figure 5AN).

[0211] In some embodiments, a second user input 591 (e.g., a tap input or other selection input) detected in an extended version of the session area 502-14 in Figure 5AM causes the device to open the application user interface 570 of the media player application (Figure 5AN) from the extended session area 502-14 (for example, device 100 is configured to transition from Figure 5AL to the intermediate Figure 5AM in response to a first type of user input 590 in Figure 5AL, and then transition from the intermediate Figure 5AM to Figure 5AN in response to user input 591).

[0212] In some embodiments, the session area stops displaying information about the media player application within the session area, while the media player application's user interface 570 is displayed outside the session area. For example, since session information is displayed in the application user interface, session information does not need to be displayed in the session area either. In some embodiments, while displaying the media player application's user interface 570 in Figure 5AN, the session area for the active timer session is redisplayed while continuing to display the session area 502-10 for active audio recording. For example, since device 100 displays up to two active sessions simultaneously, the device replaces the display of the media player session with the display of the timer session 502-15 in the session area.

[0213] Figure 5AN further displays user input 592 selecting the session area 502-10 of an active audio recording session. In some embodiments, in response to user input 592, the application user interface 584 of the audio memory application is displayed as shown in Figure 5AO, according to the determination that the user input is a second type of user input (e.g., tap input or other selection input). In some embodiments, while the user interface 584 of the audio memory application is displayed, the session area stops displaying session information for audio recording (e.g., in Figure 5AO, the audio memo session area 502-10 is no longer displayed). In some embodiments, device 100 optionally redisplays the media player session in the second part of session area 502-9 and moves the display of the active timer session area to the first part of session area 502-8 (e.g., according to the determination that the active timer session was started before the media player session was started).

[0214] Figure 5AO shows user input 594, such as tapping the “Cancel” button in the voice memory application user interface 584. In some embodiments, in accordance with user input requesting to cancel voice recording, the voice recording session is no longer active, as shown in Figure 5AP, where the canceled recording is listed as “Recording 2,” the “Cancel” button is reverted to the “Record” button for starting a new recording, the privacy indicator 518 is no longer displayed, and instead the battery indicator 506 (or another currently active, system-determined, or user-defined status indicator) is redisplayed. Thus, if the user navigates away from the voice memory application user interface while voice recording is not in progress, there is no active session of the voice memory application, and the session area does not display information about the voice memory application (e.g., as in Figures 5AQ and 5AW) until the next active session of the voice memory application is optionally started (e.g., another voice recording is started).

[0215] Figure 5AP shows user input 596 detected while displaying the user interface 584 for the voice memory application. In some embodiments, user input 596 is a swipe gesture, a drag gesture, a press-and-hold gesture, or another gesture for navigating to the control user interface 598. In some embodiments, in response to user input 596, device 100 displays the control user interface 598 as shown in Figure 5AQ.

[0216] In some embodiments, the control user interface 598 allows the user to change one or more settings of device 100 and / or quickly access one or more shortcuts to applications stored in device 100. For example, the user can change volume and / or brightness settings (e.g., using one or more slider bars), turn Bluetooth, Wi-Fi, network connectivity, and flight mode on or off, and further, turn on the flashlight, open the camera application, open the calculator application, and / or select and deselect icons to open the timer application. In some embodiments, the control user interface 598 is customizable by the user to select which settings and / or shortcuts for applications to include in the control user interface 598.

[0217] In some embodiments, as shown in Figure 5AQ, device 100 is configured to receive one or more user inputs, such as user input 5000 and / or user input 5002. In some embodiments, user input 5000 is optionally a user input on a physical or simulated button on the side of the device 100's display (e.g., long press, tap, or other select user input). In some embodiments, user input 5002 is a drag or swipe gesture initiated at the upper edge of the device's display. In some embodiments, each of user input 5000 and user input 5002 causes device 100 to display a wake screen user interface 5004 and / or a cover sheet user interface. For example, wake screen user interface 5004 is a user interface that is displayed after device 100 enters a low-power state in which the device's display is at least partially off, in response to an input prompting device 100 to exit the low-power state. In some embodiments, in the low-power state, the display optionally displays an "always-on" indicator for the time and / or date. For example, in response to user input 5000 in Figure 5AQ, device 100 enters a low-power state (e.g., the display is turned off), and Figure 5AR1 represents the user interface displayed when the device is prompted to exit the low-power state.

[0218] In some embodiments, in response to user input 5000 in Figure 5AQ, the device enters a locked state where a password, passcode, and / or biometric authentication is required to unlock the device, and the device has limited functionality in the locked state and must be unlocked before accessing each application and / or data stored on the device 100. In some embodiments, a wake screen user interface is displayed regardless of whether the device is locked or already unlocked (for example, the wake screen user interface is displayed when the device is started before the user accesses the home screen user interface and / or other application user interfaces). In some embodiments, a cover sheet user interface has the same characteristics as the wake screen user interface, but the user can optionally browse the cover sheet user interface without entering the device's locked state and / or low power state. In some embodiments, in response to optional user input (for example, a swipe gesture upward to the center of the display, such as user input 5010 in Figure 5AR1, or another gesture), one or more alerts (e.g., system alerts and / or notifications) are displayed on the wake screen user interface and / or cover sheet user interface.

[0219] Figure 5AR1 shows that, for one or more applications having an active session, in some embodiments the active session is not displayed in a session area that includes one or more sensor notches. For example, in Figure 5AR1, media playback sessions 502-9 are no longer displayed in the session area and are instead displayed as a separate user interface element 5006 on the wake screen user interface 5004. In some embodiments, the session is animated to move out of and / or away from the user interface element 5006 from the session area. In some embodiments, the user interface element 5006 includes information about the currently playing media item (e.g., title, artist, and / or cover art associated with the media item) and / or a number of controls (e.g., skip back, pause and / or play, skip forward, and optionally a volume control slider) for controlling the media playback session, which can be invoked by user input to perform individual actions related to media playback. In some embodiments, the user interface element 5006 is displayed in a predetermined portion of the display area (e.g., below the date, or at the bottom of the display area above the flashlight and / or camera icons). In some embodiments, one or more applications continue to be represented in the session area (for example, the active timer session 502-16, since the wake screen user interface 5004 is not optionally configured to display separate user interface elements for the timer application).

[0220] Figures 5AR2 to 5AR3 illustrate an example of moving two or more sessions from the session area to the cover sheet user interface 5004a. In some embodiments, while there are three active sessions, including media playback session 502-9a, timer session 502-8a, and sports session 599, in response to user input 5002b requesting the display of the cover sheet user interface 5004a, device 100 displays the cover sheet user interface 5004a (Figure 5AR3), and two or more of the active sessions are displayed outside the session area as user interface elements, including user interface element 5006 (for, for, the media playback session) and user interface element 5007 (for, for, the sports session). In some embodiments, one or more sessions may optionally remain displayed in the session area (e.g., timer session 502-16a).

[0221] In some embodiments, the device maintains current information about the active session even when the device enters a low-power state. For example, when the low-power state ends, displaying the wake screen user interface 5004 includes displaying the session area 502-16 for the active timer session (e.g., if the remaining time is 01:21). In some embodiments, a lock indicator 5008 is optionally displayed between sensor notches in the session area to indicate that the device is in a locked state. In some embodiments, the lock indicator is displayed on a display outside the session area (e.g., above the time "9:41" displayed on the wake screen user interface 5004).

[0222] Figure 5AR1 shows a user input 5010 for requesting to view one or more alerts, which is detected by the wake screen user interface 5004. In some embodiments, the user input 5010 is a swipe gesture in a first direction (e.g., up, down, right, or left) within a given area of ​​the wake screen user interface 5004 (e.g., away from the edge of the display). In some embodiments, in response to the user input 5010, the device 100 displays several alerts received by or occurring on the device 100 (e.g., system alerts and / or notifications associated with their respective applications), as shown in Figure 5AS. In some embodiments, while the notification view is displayed in the wake screen user interface, the user interface elements displayed in the wake screen user interface 5004 appear to scroll up and down the display. For example, as shown in wake screen user interface 5004-1 in Figure 5AS, the user interface element 5006 is shifted upward within the user interface.

[0223] Figure 5AS further illustrates that, in accordance with the determination that device 100 has been unlocked (e.g., by using a passcode, fingerprint authentication, facial recognition, iris authentication, or another biometric or non-biometric authentication), the lock indicator 5008 displayed between the sensor notches is updated to an unlocked appearance to indicate that device 100 is in an unlocked state. In some embodiments, the session area 502-16 continues to update as the timer session counts down the remaining time of the timer.

[0224] In some embodiments, a user can interact with multiple alerts, including opening applications associated with individual alerts and / or discarding and / or muting individual alerts. For example, Figure 5AS shows user input 5012, such as a right-to-left swipe gesture on a notification from a messaging application, and as shown in Figure 5AT, subsequent user input 5014 selects the option to clear the notification. Thus, in Figure 5AU, the notification from the messaging application has been removed from the notification view, and in place of the removed messaging application notification, the next notification in the list of alerts (e.g., a news article) is displayed. In some embodiments, a user can scroll to view additional alerts. For example, user input 5016 in Figure 5AU is an upward swipe gesture, and in response to user input 5016, additional notifications are displayed in user interface 5004-1, as shown in Figure 5AV. According to some embodiments, the session area 502-16 is continuously updated as the timer session counts down the remaining time on the timer through various interactions with the wake screen user interface described with respect to Figures 5AR1 to 5AU.

[0225] In some embodiments, as shown in Figure 5AV, when the user scrolls to additional notifications, the user interface element 5006 of the media playback session is scrolled out of view on the display. In some embodiments, upon determination that the user interface element 5006 is no longer visible in the user interface 5004-1, the media playback session is optionally displayed (e.g., redisplayed) in session area 502-9 (e.g., second part of the session area) simultaneously with another active session (e.g., active timer session 502-16b) in the first part of the session area.

[0226] Figure 5AV shows a user input 5018 requesting to exit the wake screen user interface 5004 and display the home screen user interface 501. In some embodiments, the user input 5018 is a swipe up initiated in a predetermined area of ​​the display near the bottom edge. In some embodiments, as the user input 5018 progresses, the wake screen user interface 5004 is displayed shifting on the display in the same direction as the user input 5018 (for example, the wake screen user interface 5004 is displayed sliding up and down the display).

[0227] In some embodiments, after navigating away from the wake screen user interface 5004, the lock indicator 5008 is no longer displayed in the session area. In some embodiments, the lock indicator 5008 is displayed in an unlocked state (e.g., as unlocked) for a predetermined time (e.g., from the time the device 100 is unlocked), and then the display is automatically stopped after a predetermined time has elapsed. In some embodiments, the active session remains displayed in the session area (e.g., in the first and second portions of the session area) when the user navigates between the wake screen user interface 5004 and other system user interfaces (e.g., the home screen user interface 501) and / or application user interfaces for applications running on the device 100. In some embodiments, user input 5020 is an input that activates a physical or virtual button on the device side to put the device 100 into a low-power state.

[0228] In some embodiments, device 100 is brought out of the low-power state in response to user input (e.g., the user touching the screen, the user picking up the device, and / or the user touching a button on the device). In some embodiments, device 100 automatically leaves the low-power state in response to an event (e.g., in response to receiving an alert, and / or an event from an application). Figure 5AX, similar to Figure 5AR1, shows the wake screen user interface 5004 that is initially displayed in response to the device leaving the low-power state (e.g., transitioning to the wake state), the media playback session 502-9 transitioning from the session area to the user interface element 5006 (e.g., the media playback widget for the wake screen user interface 5004), and the timer session 502-16 being displayed in the session area. In some embodiments, the device automatically enters lock mode while in the low-power state.

[0229] In some embodiments, the session region automatically expands in response to events in the current session without detecting user input on the session region. For example, in Figure 5AY, the timer ends and session region 502-16 (Figure 5AX) expands to session region 502-17 (Figure 5AY). In some embodiments, device 100 optionally provides audio and / or haptic output in addition to expanding the session region in response to events in the ongoing session. In some embodiments, device 100 provides audio and / or haptic output while expanding the session region in response to a determination that the session region has been expanded in response to user input, but does not provide audio and / or haptic output according to the session region that expands as a result of the session events (or optionally provides different audio and / or haptic output). Examples of events in an ongoing session that optionally extend the session area to a device include when a new media item starts playing (Figure AK), when a timer ends (Figure 5AY), when a new navigation command is imminent, when a connection is made (e.g., to share a media item over a device-to-device network), or when a new media item is being shared, and / or when the device is imminent to access a payment method stored in device 100.

[0230] Figure 5AZ shows that after the timer has finished and the timer session is no longer active, the session area is optionally displayed as session area 502-1 for a predetermined period of time, including the display of the lock indicator 5008 as open lock, according to the fact that there is no active session and device 100 is unlocked.

[0231] Figures 5BA1 to 5BA3 illustrate examples of receiving alerts (e.g., system alerts or notifications) while displaying session regions in different states. For example, in Figure 5BA1, an empty session region 502-18 does not contain any active sessions. Figure 5BA2 shows an example of an incoming alert while there is one active session (e.g., a timer session) displayed in a condensed size in session region 502-19. Figure 5BA3 shows an active session region 502-20 for an ongoing communication session containing multiple controls for the session. In some embodiments, session regions 502-19 and / or 502-20 are outlined with colored borders and optionally have colors based on the applications associated with the active sessions within the session regions.

[0232] In some embodiments, incoming alerts (e.g., notifications 5024-1, 5024-2, and / or 5024-3) are animated as they enter the device 100's display from the top edge of the display. In some embodiments, the incoming alert slides down from the top of the display to a lower position within the display. In some embodiments, the incoming alert is initially displayed with visual de-emphasis (e.g., blur effect, low opacity, and / or low brightness). In some embodiments, the incoming alert is displayed with visual characteristics that make it appear as if it is sliding behind the displayed session area. In some embodiments, the size of the alert changes as it slides across the display (e.g., from smaller to larger) to make it appear as if the alert is getting closer.

[0233] As shown in Figures 5BA1 to 5BA3, in some embodiments, the alert does not change its animation based on the size of the session area 502 (e.g., the same animation of the alert is displayed for session areas 502 of different sizes). In some embodiments, after the alert has moved beyond the session area in which it was displayed, so that the alert is no longer obstructed by the session area, the alert is displayed with less visual de-emphasis (or, optionally, without visual de-emphasis), and / or the alert is enlarged in size (e.g., to provide a visual effect that simulates an alert sliding behind the session area and then moving toward the user). In some embodiments, the alert is displayed below and / or near the session area and changes its final position based on the current size of the session area (e.g., alerts 5024-1 and 5024-2 are displayed in separate vertical positions due to the respective heights of session areas 502-18 and 502-19, while alert 5024-3 is displayed in a lower vertical position on the display due to the increased height of session area 5024-3).

[0234] Figure 5BB shows the final state alert 5024-1 of Figure 5BA1 displayed on the home user interface 501, and user input 5026 (e.g., tap input) requesting to open the map application associated with icon 436 on the home user interface 501. In response to user input 5026, the user interface 5028 of the map application is displayed, as shown in Figure 5BC. In some embodiments, the user interface 5028 of the map application surrounds the session area on all sides. Figure 5BC further shows user input 5030 for initiating navigation to a destination using the map application.

[0235] Figure 5BD shows a user interface 5028-1 of a map application that includes navigation commands. In some embodiments, the session area 502-21 is optionally displayed with a boundary according to a determination that the user interface surrounding the session area meets a brightness threshold (for example, if the content around the session area is dark, a prominent boundary is displayed, and if the content around the session area is bright, a second boundary that is less prominent than the first boundary is displayed). In some embodiments, while the map application is providing navigation commands, user input 5032 is detected that corresponds to a request to leave the map application user interface 5028-1.

[0236] Figures 5BE1 to 5BE2 show animated transitions to minimize the display of the map application user interface 5028-1 and display the home screen user interface 501 in response to user input 5032. In some embodiments, the animated transitions include shrinking the map application user interface 5028-1, shrinking the content within the user interface 5028-1 (while optionally maintaining the display of the content as it would be when displayed in full-size user interface 5028-1), and moving the shrink version of user interface 5028-1 upward toward the session area. In some embodiments, user interface 5028-1 is shrinked at a respective rate and moved upward toward the display so that the width of the shrink user interface 5028-1 matches the current width of the session area 502-21 (for example, when the shrink user interface 5028-1 reaches a vertical position on the display in or near the vertical position of the session area 502-21, as indicated by the dashed line in Figure 5BE1). For example, the user interface 5028-1 appears to slide below and / or behind the session area 502-21 (for example, as shown in Figure 5BE2). In some embodiments, the animated transition optionally includes shrinking the map application user interface 5028-1 while simultaneously expanding the session area 502-21 (for example, the session area 502-21 is wider in Figure 5BE2 than in Figure 5BE1). In some embodiments, during the animated transition, the home screen user interface 501 is not visually highlighted (for example, it appears blurred and / or with reduced opacity and / or reduced brightness). In some embodiments, the user interface 5028-1 optionally continues to move upward on the display until it touches the upper edge of the display of the device 100 displayed above the session area 502-21.

[0237] In some embodiments, as shown in Figure 5BF, after the user interface 5028-1 is reduced to a width matching the width of the session area, the device 100 stops displaying the reduced version of the user interface 5028-1 and instead displays information about the navigation session in the session area 502-22. For example, instead of continuing to display the map with step-by-step instructions, the session area 502-22 includes an indication of the next step in the navigation (e.g., turn right at 2 miles). In some embodiments, the session area 502-22 is maintained for a threshold time (e.g., 0.1, 0.2, 0.5, 1, 2, 5, 15, 30, or 60 seconds) at the end of an animated transition before automatically minimizing to the session area 502-23 shown in Figure 5BG. In some embodiments, the session area 502-23 automatically disappears without user input according to the determination that the navigation instructions have ended (e.g., the user has arrived at the destination). In some embodiments, the session area 502-23 is no longer displayed in accordance with the determination that the user has requested the end of the navigation session.

[0238] Figure 5BG shows user input 5034 requesting to open a web browser application associated with an icon 420 in the home screen user interface 501. Figure 5BH shows the user interface of the web browser application while the device 100 is rotated from portrait (e.g., vertical as in Figure 5BG) to landscape (e.g., horizontal). In some embodiments, the device 100 can be rotated such that the downward direction is defined relative to the user's view of the device. For example, when the user rotates the device from vertical to horizontal, in some embodiments, the display of the device 100 is updated to change the orientation of text, images, and / or other content to match the current orientation of the device, as described with reference to Figure 5BK.

[0239] In some embodiments, while device 100 is oriented horizontally as shown in Figure 5BH, a session for a single application is displayed in separate portions of the session area (e.g., portions 502-24, 502-25, and 502-26). For example, in Figure 5BH, a session for wireless headphone connection is initiated. In some embodiments, portions of the session area include separate user interface elements (e.g., bubbles) that appear to animate from session area 502-24, which optionally includes one or more sensor notches in the vertical direction (e.g., above and below the session). For example, because device 100 is oriented horizontally, to minimize the intrusion of the session area that blocks user interface elements within the user interface of a web browser application, the session area displays session information above and below session area 502-24 that encompasses the sensor notches. In some embodiments, if there are two active sessions, portions 502-25 of the session area optionally display information for the first of the two active sessions, and portions 502-26 of the session area optionally display information for the second of the two active sessions.

[0240] Figure 5BI shows that, after the animation shown in Figure 5BH, the user interface elements of the session area are optionally maintained as separate user interface elements at a predetermined distance from the session area. In some embodiments, at least one portion of the session area includes one or more sensor areas (e.g., at least one of the sensor area portions 502-25 or 502-26 extends and remains connected to the session area 502-24, rather than both being separated from the session area 502-24). In some embodiments, user input 5036 (e.g., tap input or other selection input) is detected in a portion of the session area (e.g., portion 502-25). In some embodiments, portion 502-25 of the session area displays an indication of headphones currently connected to and / or paired with device 100. In some embodiments, portion 502-26 includes an indication of the headphones' current battery level (e.g., "82%"). In some embodiments, the session area portion 502-24 is maintained with a background color that matches the color of the sensor within the sensor cutout, without displaying any information in the session area portion 502-24.

[0241] Figure 5BJ shows that in response to user input 5036, portions 502-26 of the session area expand to display additional information about the device 100 and the headphones currently connected and / or paired. In some embodiments, portions 502-24 and / or 502-25 remain displayed without expanding or displaying additional information. In some embodiments, portions 502-24 and / or 502-25 of the session area also expand and / or display additional information in response to user input 5036. Figure 5BJ further shows user input 5038 in a web browser application selecting the option to start video playback.

[0242] Figure 5BK shows multiple orientations of device 100 (e.g., device 100a, device 100b, device 100c, and device 100d) while displaying a video in playback (e.g., movie A). In some embodiments, devices 100a and 100b are shown in landscape orientation. Device 100a shows a sensor notch, indicated by a dotted line, on the left side of device 100a. In some embodiments, while the user interface is dark, the sensor located in the sensor notch is optionally not visible, or otherwise not displayed with any user interface elements to draw attention to the sensor notch so that the sensor merges with the user interface. In some embodiments, the user interface (e.g., or a portion of the user interface surrounding portions of session areas 502-25 and 502-26) is displayed by a dark background (e.g., having a luminance threshold amount), while edge processing is displayed as a boundary surrounding each of the portions of session areas 502-25a and 502-26a. In some embodiments, portion 502-25a is displayed and portion 502-26a is not displayed. In some embodiments, portions of session areas 502-25a and 502-26a are optionally not displayed while content is being played in full-screen mode. In some embodiments, the border color of portion 502-25a is the same as the border color of portion 502-26a, according to the determination that portions 502-25a and 502-26a display information about the same session (for example, both portions display information about a headphone connection session). In some embodiments, if portions of a session area display information about different sessions associated with separate applications, the border color of the individual portions is selected based on the application associated with the individual session area (for example, a timer session optionally displays an orange border, a communication session optionally displays a green border, and / or a map navigation session optionally displays a blue border).For example, in Figure 5BN, the boundary of portion 5042-1 of the session area for the audio recording session is shown in a first color, and the boundary of portion 5042-2 of the session area for the navigation session is shown in a second color that is different from the first color.

[0243] In some embodiments, when the user rotates the device 100b 180 degrees such that the sensor notch is on the right side of the device 100b while the device 100b is in a landscape orientation, portions of session regions 502-25b and 502-26b remain visible close to the sensor notch (e.g., on the right side). In some embodiments, the relative positions of the session region portions change at will. For example, portion 502-26b is displayed above portion 502-25b of device 100b, and portion 502-25a is displayed above portion 502-26a of device 100a. In some embodiments, portion 502-25b is displayed and portion 502-26b is not displayed. In some embodiments, while device 100 is in a landscape orientation (for example, as devices 100a and 100b in Figure 5BK), the session area does not display information that is updated over time for the active session (for example, a minimized session area may be displayed to minimize the appearance of one or more sensor areas, and the session area is not expanded to continuously display state information that is updated over time, such as timers, media playback, or other sessions described herein, although in some embodiments the session area may be temporarily expanded to display alerts and other session events, as shown in Figure 5BK).

[0244] In some embodiments, according to the determination that the devices (e.g., devices 100c and 100d) are in portrait orientation, the session area is displayed as a single session area 502-24a (rather than, for example, as a separate user interface element adjacent to the sensor notch). In some embodiments, the session area 502-24a optionally displays a battery icon and / or a percentage of battery remaining to indicate the battery level of the connected headphones. In some embodiments, according to the determination that the background is sufficiently dark (e.g., meeting a brightness threshold), a border is displayed around the session area 502-24a.

[0245] In some embodiments, when the device is rotated 180 degrees from the orientation of device 100c to device 100d, the session area 502-24b is displayed in a portion of the display that encompasses the sensor notch and / or sensor of device 100. For example, the session area 502-24b is displayed at the bottom of the display to encompass the sensor located at the bottom of the display while the device is in the orientation of device 100d.

[0246] In some embodiments, depending on whether the device is displaying content in full-screen mode, for example in Figure BK, the video content is played to occupy the entire user interface area (excluding the session area), and the boundaries of each part of the session area are optionally not displayed. For example, the session area is displayed inconspicuously to better blend with the content displayed in full-screen mode.

[0247] Figure 5BL shows an example of information displayed in the session areas of multiple applications. The example shown in Figure 5BL represents a snapshot of an individual session area in time, and it will be understood that as the application state changes, the state information is updated within the session area. In some embodiments, session areas 5041 and 5040a-5040q are displayed in the session area simultaneously with individual user interfaces displayed outside the session area on device 100. For example, in Figures 5E-5F, session areas 502-4 are optionally displayed along with either session area 5041 or one of 5040a-5040q, depending on the type of session that is active at a given time, if present. For example, Figures 5E-5F show a communication session, where either session area 5041 or 5040a-5040q could be displayed in the location of session area 502-4 (Figure 5E). In some embodiments, session area 5041 includes a privacy indicator (e.g., privacy indicator 518) without displaying state information about an active session (e.g., if there is no active session). In some embodiments, the session area 5040a corresponds to a communication session (e.g., telephone and / or video conference) that includes an icon of the application associated with the session (e.g., a phone icon), the length of time the communication session is in progress (e.g., "0:08"), optionally a privacy indicator displayed between sensor notches to indicate that the microphone of device 100 is currently being accessed by the application (e.g., dots between sensor notches in the sensor area 5040a), and an audio waveform indicating incoming and / or outgoing audio data during the communication session.

[0248] In some embodiments, session area 5040b corresponds to a timer session and includes an icon for the timer application and the remaining time of the timer. In some embodiments, session area 5040c corresponds to a media playback session and includes a representation of the media content item currently being played (e.g., cover art for the media content item) and a playback icon indicating that playback is in progress. In some embodiments, session areas 5040c-2 correspond to a media playback session (e.g., for a music application or other media playback application) that includes a representation of the media content item currently being played (e.g., cover art for the media content item) and an audio waveform indicator. In some embodiments, the audio waveform indicator is generated based on the current audio output of the media content item currently being played so that the audio waveform is updated as the audio output changes over time. In some embodiments, the audio waveform indicator is displayed in one or more colors selected based on the representation of the media content item currently being played. For example, one or more prominent colors are sampled from the cover art of the media content item or otherwise identified, and the waveform is displayed in one or more prominent colors. In some embodiments, the sound wave indicator is superimposed on at least a portion of the cover technology, optionally on at least a portion of a blurred or visually modified version of the cover technology.

[0249] In some embodiments, the session area 5040d corresponds to an accessibility session and includes icons for accessibility applications and indications for connected devices for the accessibility session (e.g., connected hearing aids).

[0250] In some embodiments, the session area 5040e corresponds to a navigation session for a map application and includes indications of the next step in the navigation (e.g., turn right) and indications of when the next step will occur (e.g., in 2 miles).

[0251] In some embodiments, the session area 5040f indicates a health and / or health session and includes an icon for the training application and an indication of the current training (e.g., distance covered, amount of time spent on training, and / or calories burned during training).

[0252] In some embodiments, the session area 5040g indicates a session for sharing content (for example, over inter-device Wi-Fi), which includes a share icon and a progress indicator that updates as the sharing process progresses until it is complete.

[0253] In some embodiments, session area 5040h represents a session area for an audio recording session, which includes an audio waveform indicating the audio data being recorded in the audio recording, a recording indicator, and the amount of time the audio recording is in progress. Session area 5040h optionally includes a privacy indicator indicating that the microphone of device 100 is being accessed by an application.

[0254] In some embodiments, the session area 5040i indicates a session area for remotely viewing and / or sharing content from device 100 on a separate display device, which includes an indication that device 100 is sharing its display, and optionally an indication of an application displayed on a separate display device (for example, an application for streaming video content, where the video content is being played on the separate display device).

[0255] In some embodiments, the session area 5040j represents a session area for a sports application, including an indication of the current score of each team participating in the active sports session (e.g., during an individual game).

[0256] In some embodiments, the session area 5040k represents a session area for a ride-sharing session, including an icon representing the ride-sharing application, as well as the time until the ride-sharing arrives and / or the user is dropped off.

[0257] In some embodiments, the session area 5040l represents a session area that is displayed while the device is in locked mode and the device is authenticating a user to unlock the device. In some embodiments, the session area 5040l includes a lock icon and a progress animation to indicate that the device is in the process of authenticating a user.

[0258] In some embodiments, the session area 5040m represents a session area for biometric authentication, which includes an icon representing an authentication prompt to prompt the user to perform biometric authentication (e.g., by looking at a camera and / or providing a fingerprint).

[0259] In some embodiments, session region 5040n indicates a low-battery session provided by the system software of device 100 according to a determination that the remaining battery level of device 100 is below a threshold amount.

[0260] In some embodiments, the session area 5040o represents a session area for a connected device (e.g., connected headphones and / or speakers(s)) including indications for connected devices and optionally indications for the battery level of connected devices.

[0261] In some embodiments, session area 5040p displays a session area for silent mode, including an indication (e.g., a bell icon with a line drawn) that silent mode has been started or enabled (e.g., ringer mode has been disabled), and optionally a description of “silent mode”. In some embodiments, session area 5040p-2 displays a session area for ringer mode, including an indication (e.g., a bell) that ringer mode has been started or enabled (e.g., silent mode has been disabled), and optionally a description of “ringer”, which optionally includes an animation of the bell indication as a shake. In some embodiments, the user can switch the device 100 between silent mode and ringer mode. In some embodiments, in response to detection that the mode has been switched between silent mode and ringer mode, session regions 5040p and / or 5040p-2 optionally include shaking a bell indicator within the session region while the session region is shaking (for example, repeatedly from right to left, as illustrated in the shake animations in Figures 6H-6J).

[0262] In some embodiments, the session area 5040q indicates a session area for an active hotspot session, which includes an icon indicating that at least one device is currently linked via the hotspot connection, and optionally an indication of the signal strength of the hotspot connection.

[0263] In some embodiments, one or more of the session areas 5040a to 5040q are displayed while the corresponding session is in progress. In some embodiments, session areas associated with individual applications and / or software are displayed for a predetermined period of time before the display of the session areas is automatically stopped. For example, the session area 5040p in silent mode is optionally displayed for 2 or 10 seconds after device 100 enters silent mode, but is not maintained on the display while device 100 remains in silent mode.

[0264] Figures 5BM1 and 5BM2 illustrate how the session area of ​​the same application is updated to display two or more sets of information about the application. For example, during an active session for a sports application, two or more games that are simultaneously active may have different sets of information for the individual games (e.g., different scores and / or different teams). In some embodiments, while two or more sets of information are available, the session area rotates periodically between the display of the two or more sets of information. For example, in Figure 5BM1, session area 5040j-1 shows the current score for GS vs. CHI, and in Figure 5BM2, session area 5040j-2 shows the current score for LA vs. NY. In some embodiments, the games do not need to be of the same type (e.g., basketball, baseball, football, and / or hockey). Instead, according to the determination that the individual applications associated with the session area contain two or more sets of information, each set of information is displayed for a predetermined amount of time (e.g., 1, 2, 5, 10, 15, 30, 60, 90, or 120 seconds) before replacing the display with another set of information during that predetermined time period. In some embodiments, the set of information automatically changes from a first set of information to a second set of information in accordance with an event occurring for an individual set. For example, while the first set of information is displayed, the team scores in a game related to the second set of information, and device 100 automatically replaces the display of the first set of information with the second set of information to indicate the occurrence of the event (optionally, regardless of a predetermined amount of time). In some embodiments, in response to user input directed to session area 5040j-1 in its condensed form as shown in Figure 5BM1 (e.g., certain types such as a long press), device 100 expands the session area to session area 5040-j3 to display additional information such as the current playback period (e.g., a quarter or half) and / or the remaining time of the current playback period, as shown in Figure 5BM3. A similar user input directed to session area 5040j-2 in Figure 5BM2 similarly results in a session area that expands to session area 5040-j4 in Figure 5BM4.

[0265] Figures 5BM3 to 5BM4 illustrate the display of two sets of information in the extended session area in response to user input 5999b, which can be optionally selected. For example, user input 5999b is detected while the extended session area 5040j-3 is displayed, which includes a display of a first set of information about a first event (e.g., a first game about a sports application). In some embodiments, user input 5999b is, for example, a left-to-right swipe input, and in response to user input 5999b, device 100 updates the display in the extended session area to display a second set of information about a second event (e.g., a second game about a sports application).

[0266] Figure 5BN shows an example of multiple active sessions displayed simultaneously in a session area. While the session area shown in Figure 5BN is an example of simultaneous sessions, it will be understood that alternative and / or additional arrangements of active sessions may also be used depending on which application has the current active session and / or relative priority of the active session. For this purpose, the session area includes a portion 5042-1 for the current audio recording encompassing a sensor notch (e.g., a sensor area), optionally including a privacy indicator displayed between the sensor notches, and simultaneously displaying a portion 5042-2 for the navigation session. In some embodiments, simultaneous sessions are displayed in different sizes, and the information displayed in each portion of the session area is based on the size allocated to individual applications within the portion of the session area.

[0267] In some embodiments, device 100 displays an animation while transitioning the session area from a first state to a second state, including expanding and / or minimizing the session area and / or updating the session area with information from a first session of a first type to a second session of a second type (for example, as described with reference to Figures 5BT to 5BU). In some embodiments, the type of animation displayed during the transition between session states is based at least in part on the current size of the session area. For example, device 100 displays a first animated transition from an expanded timer session to another session state (for example, another session such as a condensed timer session or a music session), including animating numbers to selectively shift numbers up and down while decreasing the text size of the numbers (for example, representing the remaining time on a timer). In some embodiments, if the timer session is a condensed timer session (e.g., smaller than an expanded timer session), device 100 displays a second animated transition from the condensed timer session to another session state, distinct from a first animated transition, and includes crossfading the numbers of the condensed timer session with the text and / or icons displayed in the other session, without shifting the numbers up or down as they change. Thus, the animated transitions between sessions displayed in the session area depend on the current size of the session area (e.g., condensed or expanded) when the transition is initiated.

[0268] In some embodiments, different types of animation are used between different display states of the session area. For example, animating timer numbers to shift up and down while changing the displayed value (e.g., mimicking the mechanical rotation of a clock or the reversal of numbers) is optionally performed while the session area is in an expanded state (e.g., having more space for animation), while crossfading numbers without shifting up and down while changing the displayed value is optionally performed while the session area is in a condensed timer session (e.g., having less space for animation).

[0269] As another example, portion 5043-1 of the session area includes a timer session and optionally includes a sensor notch, and portion 5043-2 includes a session about biometric authentication. In some embodiments, portion 5044-1 of the session area includes an indication of the set of headphones currently connected, and portion 5044-2 includes session information about a training session and optionally includes a sensor notch. In some embodiments, portion 5045-1 of the session area indicates a session about silent mode and optionally includes a sensor notch, and portion 5045-2 includes an indication of a media streaming application session simultaneously with portion 5045-3 which includes an indication of the set of headphones currently connected. Thus, in some embodiments, three or more sessions are displayed simultaneously in portions of the session area, and optionally, each session is displayed as a separate user interface element as a distinct portion of the session area.

[0270] In some embodiments, portion 5046-1 of the session area displays an active hotspot connection session, and portion 5046-2 of the session area displays a media playback session. In some embodiments, portion 5047-1 displays a sharing session for sharing media content with other devices (e.g., via inter-device Wi-Fi, NFC, and / or Bluetooth), and portion 5047-2 displays a session about audio recording. In some embodiments, a separate user interface element is displayed as encompassing a sensor notch, and the portion of the session area displaying information about the active session is displayed next to (e.g., to its right and / or left) the user interface element encompassing the sensor notch.

[0271] In some embodiments, a portion 5048-1 of the session area displays an indication of a set of connected headphones, a portion 5048-2 of the session area displays a silent mode indication and optionally includes one or more sensor notches, and a portion 5048-3 displays a screen recording session.

[0272] Figure 5BO shows several examples of extended session areas. As described above, in some embodiments, the session area expands in response to user input (e.g., tap input and / or long press input) directed at the session area (e.g., while minimized, e.g., as session areas 5040a-5040q). In some embodiments, the session area expands in response to events and / or updates occurring in the application associated with the session without detecting user input (e.g., timer off, song change, or next navigation command). In some embodiments, each extended session area 5050 includes one or more controls for interacting with the application associated with the session. For example, extended session area 5050a represents an extended timer session and optionally includes control 5050a-1 for aborting (e.g., pausing) and / or control 5050a-2 for canceling the timer. In some embodiments, session area 5050b represents an extended communication session with optionally included controls, including a control 5050b-1 for muting, a control 5050b-2 for accessing the keypad, a control 5050b-3 for starting the use of the speakerphone, a control 5050b-4 for starting a video call, and / or a control 5050b-5 for ending the communication session. In some embodiments, session area 5050c represents an extended screen recording session with optionally included a control 5050c-1 for optionally stopping screen recording. In some embodiments, session area 5050d represents an extended ride-sharing session with optionally included a control 5050d-1 for contacting the ride-sharing driver. In some embodiments, session area 5050e represents an extended media playback session with optionally included controls, including a control 5050e-1 for skipping to the previous content item, a control 5050e-2 for pausing the content item, and / or a control 5050e-3 for skipping to the next content item.In some embodiments, session area 5050f represents an extended navigation session that optionally includes controls including a control 5050f-1 for pausing and / or a control 5050f-2 for ending a navigation command. In some embodiments, the user can select any of the above controls to cause device 100 to perform one or more actions associated with the individual controls (for example, in response to a user input selection control 5050c-1, device 100 may stop screen recording).

[0273] Figures 5BP to 5CY illustrate transitions between application user interfaces. In some embodiments, an active media playback session is displayed in session areas 502-30 while a user interface 5060 for a first application, such as a map application, is displayed. In some embodiments, session areas 502-30 include a boundary displayed around the edges of session areas 502-30 (e.g., boundary 552 (Figure 5W)). In some embodiments, session areas 502-30 are displayed with a background of a first color optionally selected to match the color of one or more sensors placed within the area, as described above with reference to Figure 5A. In some embodiments, device 100 detects a user input 5062 that responds to a request to switch applications, such as an edge swipe gesture or other application switching user input to switch the display of user interface 5060 to another application user interface. For example, in response to user input 5062, device 100 displays an animated transition that includes optionally sliding user interface 5060 in a first direction (optionally corresponding to the direction of user input 5062) and simultaneously sliding the user interface 5066 of a music application onto the user interface.

[0274] Figure 5BQ shows an example of an animated transition displayed in user interface 5064, which includes a portion of user interface 5060 sliding out of the display area and a portion of user interface 5066 sliding on the display area. In some embodiments, while displaying an animated transition for switching application user interfaces, session region 502-30 is optionally minimized as session region 502-32, and session region 502-32 does not display state information about active sessions, even if one or more sessions remain active (for example, a media playback session and / or navigation session are active but are not displayed in the session region during the animated transition). In some embodiments, the boundary of session region 502-30 is displayed gradually shrinking or closing to indicate the edge of session region 502-30 surrounding a smaller display area as session region 502-30 decreases in size to the minimized session region 502-32. For example, as the size of session area 502-30 decreases, the boundaries of session area 502-30 are maintained along the decreasing edges of session area 502-30. In some embodiments, the background color of session area 502-30 is maintained as the size of the session area changes, including displaying the same background color as the minimized session area 502-32.

[0275] In some embodiments, as shown in Figure 5BR, while the user interface 5066 of a music application corresponding to a media playback session is displayed in session area 502-30 (Figure 5BP), device 100 does not display an indication of the media playback session in the session area. Instead, it displays the active navigation session in session area 502-34, which is associated with the continued operation of a map application whose user interface has been removed by device 100. For example, since user interface 5066 is for an application associated with a media playback session, the media playback session is not displayed in the session area.

[0276] In some embodiments, device 100 detects another user input 5068, such as an edge swipe gesture or other user input, to switch the display of user interface 5066 to a different application user interface. In some embodiments, as shown in Figure 5BS1, in response to user input 5068, device 100 optionally displays an animated transition (e.g., an animated transition described with reference to Figure 5BQ) and displays the user interface 5070 of a messaging application (or another application different from the applications corresponding to user interfaces 5060 and 5062). In some embodiments, while displaying the user interface 5070 for a messaging application, device 100 displays indications for an active media playback session and an active navigation session in the session area. For example, the media playback session is displayed in portion 502-36 of the session area and, simultaneously with the navigation session displayed in portion 502-38 of the session area, optionally includes one or more sensor notches.

[0277] In some embodiments, device 100 may optionally refrain from expanding session region 502-32 to session region 502-34 (or completely refrain from expanding it) according to a determination that user input 5068 satisfies a quick switching criterion and / or that user interface 5066 is displayed for less than a threshold time before detecting user input 5068. For example, if the user navigates between the displays of user interface 5060, user interface 5066, and user interface 5070 for less than a threshold time, device 100 refrains from expanding session region to session region 502-34 and maintains a minimized (or nearly minimized) session region 502-32 without displaying the active session state information.

[0278] Figure 5BS1 further illustrates the detection of user input 5102, such as an edge swipe gesture (optionally corresponding to an edge of the display area, such as the bottom edge) or other user input, for switching the display of user interface 5070 to another application user interface. In some embodiments, in response to user input 5102, device 100 optionally displays an animated transition of the user interface in a display area outside the session area (e.g., an animated transition described with reference to Figure 5BQ), as shown in user interface 5104 of Figure 5BS2. In some embodiments, the animated transition in the display area outside the session area optionally displays the previously displayed user interface 5070 sliding off the display in a direction corresponding to the direction of user input 5102 (e.g., gradually sliding the display off at a rate based on the progression of user input, such as a swipe gesture over time and / or the rate of user input, such as a swipe gesture), and a user interface for another application sliding across the display (e.g., user interface 5106 for a web browsing application in Figure 5BS2). In some embodiments, device 100 further displays an animated transition that includes optionally minimizing the session area, which includes optionally merging and / or reducing the size of portions 502-36 and 502-38 of the session area to a minimum size indicated as the minimized session area 502-40. In some embodiments, as described with reference to Figure 5BY, the animated transition includes gradually minimizing and / or expanding the session area as the content displayed in the session area is optionally updated (for example, from displaying session information for one or more first applications to displaying session information for one or more second applications).More generally, one or more aspects of animated transitions in the display area outside the session area (including, for example, sliding a previously displayed user interface off the display and sliding a user interface for another application onto the display) should be noted as also applying to any of the examples of switching application user interfaces described herein, optionally with respect to the examples in Figures 5BP to 5BS7, 5BV to 5BW, and / or Figures 5CE to 5CG, and optionally with respect to Figures 5R to 5T.

[0279] In some embodiments, after displaying animated transitions (one or more) (e.g., in the display area outside the session area and / or the session area), device 100 displays a user interface 5106 for a web browsing application (Figure 5BS3) or a user interface for another application in the display area outside the session area. In some embodiments, as shown in Figure 5BS3, the session area is updated from a minimized session area 502-40 to redisplay the active session that was displayed before detecting user input 5102. For example, in Figure 5BS3, the portion 502-36 corresponding to the music session and the portion 502-38 corresponding to the navigation session are redisplayed simultaneously with the display of the user interface 5106 for the web browsing application. Thus, device 100 maintains and / or redisplays the active session while the user interface displayed outside the session area is updated to a different application user interface.

[0280] Figure 5BS3 further illustrates the detection of user input 5108 directed to portion 502-36 corresponding to a music session. In some embodiments, user input 5108 is a tap input or other user input corresponding to a request to open an application user interface for an application associated with the music session displayed in portion 502-36. In response to user input 5108, device 100 displays a user interface 5066 for the music application, as shown in Figure 5BS4. In some embodiments, in response to user input 5108, device 100 displays an animated transition of the session area. For example, as shown in Figures 5BS3 to 5BS5, during the animated transition, the first portion 502-36 and the second portion 502-38 of the session area (Figure 5BS3) appear to decrease in size and merge into a minimized session area 502-40 (Figure 5BS4), and then the minimized session area 502-40 expands to display session area 502-42 (Figure 5BS5).

[0281] In some embodiments, as shown in Figure 5BS5, while the user interface 5066 for the music application is displayed, the device 100 optionally displays session areas 502-42 corresponding to the navigation session without displaying the active session corresponding to the music session in the session area. For example, as described above with reference to Figure 5AN, while the user interface for the application corresponding to an individual session is currently displayed in a display area outside the session area, the individual session information is not displayed.

[0282] Figure 5BS5 illustrates the detection of user input 5110, optional swipe user input in individual directions such as right to left, and swipe user input corresponding to an edge of the display area, such as the bottom edge, in response to a request to optionally display the user interface of another application. In response to the detection of user input 5110, device 100 displays an animated transition of the display area outside the session area, as shown by the user interface 5116 in Figure 5BS6. For example, the user interface of a music application may optionally slide off the display in the direction of the user input, such as right to left, and the user interface of a map application may optionally slide on the display from the opposite edge of the display in the direction of the user input. In some embodiments, the animated transition is analogous to the animation (e.g., its reversal) described with reference to Figure 5BQ. As described above, in some embodiments, device 100 displays an animated transition of the session area, which includes displaying a minimized session area 502-44 during the animated transition.

[0283] In response to the detection of user input 5110, the user interface 5060 for the map application is displayed, as shown in Figure 5BS7. In some embodiments, the session area gradually increases in size from a minimized session area 502-44 to display one or more active sessions, including replacing the display of navigation session 502-42 with the display of music session 502-46. For example, device 100 displays session area 502-46 corresponding to a music session that was not previously displayed while the user interface for the music application was displayed in a display area outside the session area. Similarly, in some embodiments, the user interface 5060 for the map application is displayed in a display area outside the session area (Figure 5BS7), but the session area does not display the session corresponding to the navigation session associated with the map application.

[0284] Figures 5BT and 5BU illustrate examples of session regions that optionally expand during transitions between user interfaces for each application. Figure 5BT shows a session region (e.g., within user interfaces 5080a-5080e) based on the determination that user input (e.g., user input 5068) does not satisfy a quick-switching criterion (e.g., user input slower than the user input detected in Figure 5BU). For example, if a user navigates between the user interface for map application 5080a and the user interface for music application 5080c, and views the user interface for music application 5080c for at least a threshold time, the user input does not satisfy the quick-switching criterion. In some embodiments, while the user is viewing the user interface for music application 5080c, the session region expands to display navigation session state information (e.g., to session regions 502-34). In some embodiments, in response to detection of user input for navigating to a messaging application, the session region is minimized during a transition (e.g., in user interface 5082d) before displaying the state information of the active session within the session region (e.g., in portions 502-36 and 502-38 of the session region). In some embodiments, the session region is minimized during animated transitions between user interfaces (e.g., animated transitions in user interfaces 5080b and 5080d).

[0285] Figure 5BU shows a session region (e.g., within user interfaces 5082a-5082e) based on a determination that a user input (e.g., user input 5068) satisfies a quick switching criterion (e.g., a user input faster than the user input detected in Figure 5BT). For example, according to a determination that the user navigates from the user interface for the map application 5082a to the user interface for the music application 5082c, and then to the user interface for the messaging application 5082e, without pausing for a threshold time while the user interface for the music application 5082c is displayed, the session region remains displayed in its minimized state without expanding the session region to display state information for the navigation session. In some embodiments, the session region is minimized during animated transitions between user interfaces (e.g., animated transitions in user interfaces 5082b and 5082d).

[0286] Figure 5BV shows the display of the user interface 5060 for the map application and the session areas 502-48 for the timer session. In response to detection of user input to switch to another application user interface (e.g., user input 5118-1), device 100 displays animated transitions between application user interfaces without optionally displaying animated transitions of session areas 502-48. For example, device 100 displays animated transitions of display areas outside the session areas without optionally displaying animated transitions of the session areas.

[0287] In some embodiments, the device 100 displays an animated transition of session region 502-48, including minimizing the session region, after detecting that user input has stopped (e.g., detection of liftoff of user input 5118-1). For example, Figure 5BW shows the end of user input 5118-1 as liftoff 5118-2. Figure 5BW further shows that session region 502-48 is maintained without animating the session region before detecting liftoff 5118-2. In some embodiments, an animated transition of the display area outside the session region shown in the user interface 5120 of Figure 5BW is initiated in response to detection of user input 5118-1 and displayed while user input 5118-1 is in progress (e.g., without requiring liftoff 5118-2 to occur, as opposed to what is required to optionally display an animated transition of session region 502-48).

[0288] Figure 5BX shows that, in accordance with the determination that user input has stopped (for example by detecting lift-off 5118-2), an animated transition of the session area is initiated, which includes gradually reducing the size of session area 502-48 to a minimized session area 502-50, and optionally, while continuing the animated transition of the display area outside the session area, the user interface for the map application is simultaneously slid off the display and the user interface for the music application is slid onto the display.

[0289] In some embodiments, the animated transition of the session area further includes displaying a minimized session area 502-50 as its size increases to display a first portion 502-52 and a second portion 502-54 of the session area, as shown in Figure 5BY. For example, an animated transition of the session area, optionally initiated upon detection of a liftoff 5118-2, includes shrinking the session area 502-48 by gradually decreasing the size of the text and / or icons displayed in the session area and / or removing the text and / or icons, and shrinking the area of ​​the session area by gradually displaying the boundary of the session area as it moves inward until the session area is displayed as a minimized session area 502-50 (Figure 5BX). In some embodiments, the animated transition of the session area 502-48 is initiated upon initial detection of user input 5118-1 (without requiring, for example, a liftoff 5118-2 to occur and without requiring user input 5118-1 to meet additional criteria). In some embodiments, animated transitions in session regions 502-48 are optionally initiated upon the satisfaction of additional criteria, such as detection of user input 5118-1 and detection of user input 5118-1 movement, according to the determination that user input 5118-1 has moved by a threshold amount, is moving at a threshold velocity, and / or is moving in a specific range of directions (e.g., within a threshold angular distance of a reference direction). In some embodiments, minimized session regions 502-50 remain displayed until the animated transitions between application user interfaces in the display area outside the session region are complete. For example, in Figure 5BY, the user interface for the music application 5124 is displayed at the end of the animated transitions shown to user interface 5122 in the display area outside the session region.

[0290] In some embodiments, the animated transition of the session area continues by gradually increasing the size of the minimized session area 502-50 to display the first portion 502-52 and the second portion 502-54 of the session area, including gradually displaying text and / or icons in the session area (e.g., by fading in, increasing in size, focusing or sharpening, and / or animating in other ways). In some embodiments, the portions are displayed so as to extend outward from the minimized session area. For example, in Figure 5BY, as shown in the first portion 502-56 of Figure 5BZ, the first portion 502-52 displays an icon for the timer application, and as the first portion 502-52 expands, the first portion 502-52 displays the icon as a larger icon and / or displays additional text (e.g., remaining time for the timer, 14:43). In some embodiments, one or more portions of the session area are optionally separated to the right and / or left of the area corresponding to the minimized session area. For example, as shown in the second portion 502-58 of Figure 5BZ, the second portion 502-54 is animated to gradually separate from the minimized session area (for example, as a separate user interface element similar to a bubble), and the size of the navigation icon gradually increases. More generally, it should be noted that one or more embodiments of the animated transitions of the session area described herein may optionally be applied similarly to any of the examples of session area transitions described herein with respect to Figures 5BP to 5BZ and / or Figures 5CE to 5CH, etc.

[0291] Figure 5BZ shows that while displaying the user interface for the music application 5124, device 100 detects a user input 5126 directed to a first portion 502-56 of the session area corresponding to a timer session. In some embodiments, the user input 5126 includes a first type of user input, such as a long press input, also referred to herein as a long press input. In some embodiments, the user input 5126 includes a second type of user input, such as a tap input. In some embodiments, device 100 updates the display based on the type of input 5126 detected in response to the user input 5126.

[0292] For example, upon determination that user input 5126 is a first type of user input, such as a long press input, device 100 expands the session area corresponding to the session displayed in the first section 502-56 (e.g., via a transition to Figure 5CC). In some embodiments, upon determination that user input 5126 is a second type of user input, such as a tap input, device 100 displays a user interface for an application (e.g., a clock application) associated with the session displayed in the first section 502-56 corresponding to a timer session (e.g., via a transition to Figure 5CE).

[0293] In some embodiments, in response to the detection of user input 5126 (e.g., input of a first type or input of a second type), device 100 animates a transition of the session area, which includes shrinking, merging, and / or otherwise reducing the size of the first portion 502-56 and / or the second portion 502-58 of the session area, including shrinking the text and / or icons displayed within the separate portions, as shown in Figure 5CA. In some embodiments, the transition of the session area further includes bringing the first portion 502-56 and / or the second portion 502-58 closer to a midpoint (e.g., optionally, a point between two or more sensors in the session area (e.g., speaker 111 and light sensor 164), or another point substantially at the center of the display width, predefined). For example, as shown in Figure 5CA, the distance between the separate user interface elements of the first portion 502-56 and the second portion 502-58 gradually decreases.

[0294] In some embodiments, the animated transition of the session region gradually reduces the size of a portion of the session region until the session region is minimized to session region 502-60 (Figure 5CB) or session region 502-64 (Figure 5CD). For example, before the device stops displaying the text and / or icons, the text and / or icons decrease in size and optionally approach a midpoint.

[0295] In some embodiments, device 100 displays transitions between displaying text and / or icons in a session area by crossfading, or otherwise partially overlapping, the display of text and / or icons in the session area before and after the animated transition. For example, while decreasing the size of the text and / or icons displayed in the first portion 502-56 and / or second portion 502-58 of the session area, device 100 begins displaying text and / or icons in session area 502-62, including fading the text and / or icons on the display and / or gradually increasing the size of the text and / or icons. In some embodiments, it will be understood that the device refrains from minimizing the session area to a completely minimized session area 502-60 in order to display the text and / or icons in the session area with crossfading. For example, the animated transition includes shrinking the text and / or icons in the first portion 502-56 and the second portion 502-58 without displaying a minimized session area 502-60 that does not contain text and / or icons. In other words, during an animated transition, the text and / or icons displayed in session areas 502-62 overlap at least partially with the display of text and / or icons in the first portion 502-56 and the second portion 502-58 of the session area. In some embodiments, the display of the minimized session area 502-60 occurs during a crossfade of the session areas displayed before and after the animated transition. For example, some of the partial overlap between the text and / or icons of the session area displayed before the animated transition and the text and / or icons of the session area displayed after the animated transition is displayed before the minimized session area 502-60 is displayed (e.g., as the size of the session area decreases), and some of the partial overlap (e.g., the rest) is displayed after the minimized session area 502-60 is displayed (e.g., as the size of the session area increases).

[0296] Figure 5CC shows that, in response to the detection of a first type of user input as user input 5126, the device displays an expanded session area 502-62 that includes one or more additional controls for a timer session, while continuing to display a user interface for a music application 5124. In some embodiments, the expanded session area 502-62 is displayed by the device 100 for a threshold time without additional user input, and redisplays the session area that was displayed before the detection of user input 5126, including the display of a first portion 502-56 and a second portion 502-58. In some embodiments, stopping the display of the expanded session area includes performing an animated transition in reverse. For example, the expanded session area is animated to shrink in size to a minimized session area 502-60, and the first portion 502-56 and the second portion 502-58 are animated as they enlarge and move outward from the minimized session area 502-60.

[0297] Figure 5CD shows that, in response to the detection of a second type of user input, such as a tap input as user input 5126, device 100 displays a user interface for the clock application 5128 associated with the session of the first portion 502-56 in a display area outside the session area. In some embodiments, while replacing the display of the user interface for the music application 5124 with the display of the user interface for the clock application 5128, device 100 displays an animated transition of the session area as described with reference to Figure 5CA (for example, by gradually decreasing the size of the session are...

Claims

1. It is a method, In a computer system that communicates with a display generation component having a display area that at least partially encompasses a state region, To detect user input corresponding to the aforementioned state region, In response to detecting the user input corresponding to the state region, In accordance with the determination that the state area is associated with the active session of an individual application, Performing the actions associated with the aforementioned individual applications, In accordance with the determination that the aforementioned state area is not associated with an active session of an individual application, A method comprising providing feedback associated with user input without displaying information about individual applications in a state area and without performing actions associated with individual applications.

2. A method according to claim 1, wherein one or more sensors are arranged within one or more sensor regions surrounded by the display area, and the state region encompasses one or more sensor regions.

3. The method according to claim 1, A first user interface is displayed in the display area outside the state area. The system detects user input that navigates from the first user interface to the second user interface, A method comprising detecting user input that navigates from the first user interface to the second user interface, and displaying the second user interface in the display area outside the state area, while maintaining the display of the state area.

4. A method according to claim 1, wherein performing the operation associated with the individual application includes displaying the user interface of the individual application in a display area outside the state area.

5. A method according to claim 1, wherein performing the operation associated with the individual application includes extending the state area and displaying information about the individual application in the extended state area.

6. A method according to claim 5, comprising displaying content in a display area outside the state area before expanding the state area, wherein expanding the state area obscures a portion of the content that was displayed in the display area outside the state area.

7. A method according to claim 1, wherein providing the feedback associated with the user input in accordance with the determination that the state region is not associated with an active session of an individual application includes extending the state region.

8. A method according to claim 1, wherein providing the feedback associated with the user input comprises expanding the state region, the method comprising shrinking the state region in accordance with a determination that a first threshold time has elapsed since detecting the end of the user input corresponding to the state region.

9. A method according to claim 8, wherein detecting the end of the user input includes detecting the lift-off of a contact that was part of the input from the touch-sensing surface.

10. A method according to claim 1, wherein providing the feedback associated with the user input comprises expanding the state region, the method comprising shrinking the state region in accordance with a determination that the user input has been maintained for a second threshold time with respect to the state region while continuing to detect the user input corresponding to the state region.

11. A method according to claim 1, wherein providing the feedback associated with the user input comprises generating a first tactile output in accordance with a determination that the user input is maintained with respect to the state region for a third threshold time while the state region is not associated with an active session of a separate application.

12. A method according to claim 11, wherein the operation associated with the individual application is performed in accordance with a determination that the user input is maintained with respect to the state region for a third threshold time while the state region is associated with an active session of the individual application, and performing the operation associated with the individual application includes extending the state region and displaying information about the individual application in the extended state region without generating the first tactile output.

13. The method according to claim 11, While continuing to detect the user input corresponding to the state region over the third threshold time, A method for reducing a state region in accordance with the determination that the user input is maintained with respect to the state region for a fourth threshold time.

14. The method according to claim 1, wherein while a user input corresponding to the state area has not been detected, In accordance with the determination that the state area is associated with the active session of an individual application, A method comprising displaying information about the individual application in the state area, and displaying an animation of the state area, which includes changing the size of the state area and then reversing at least a portion of the change in the size of the state area.

15. The method according to claim 14, wherein while a user input corresponding to the state area has not been detected, A method comprising maintaining the size of the state region in accordance with the determination that the state region is not associated with an active session of an individual application.

16. A method according to claim 14, comprising displaying the animation of the state region, regardless of whether the state region is displayed in a first mode in which the state region includes a first amount of information about the individual application, or whether the state region is expanded and displayed in a second mode in which a second amount of information about the individual application is included, wherein the second amount of information is greater than the first amount of information.

17. The method according to claim 14, wherein the animation of the state region is displayed while the state region is displayed in a first mode in which the state region includes a first amount of information about the individual application, and the method is performed while no user input corresponding to the state region is detected. According to the determination that the state area is associated with an active session of an individual application, A method comprising animating the state region less than when it is displayed in the first mode, while the state region is expanded and displayed in a second mode containing a second amount of information less than a first amount of information about individual applications.

18. The method according to claim 14, In accordance with the determination that the state region is associated with the active session of the individual application, In accordance with the determination that the active session is a session of the first type, the animation of the state region changes the size of the state region at a first rate of change. A method in which, according to the determination that the state region is a second type of session, the animation of the state region changes the size of the state region at a second rate of change different from the first rate of change.

19. A method according to claim 18, wherein displaying the information for the individual application includes displaying one or more user interface elements associated with the individual application, and the animation of the state area changes the size of the state area at a rate corresponding to the rate of change of each of the displayed user interface elements associated with the individual application.

20. A method according to claim 18, wherein the animation of the state area changes the size of the state area at a rate based on the urgency of the information about the individual applications displayed in the state area.

21. The method according to claim 1, wherein while a user input corresponding to the state area has not been detected, The state region is associated with an active session of an individual application, and according to the determination that the active session is a first type of session, Displaying information about the individual applications in the state area, and displaying an animation of the state area, which includes changing the size of the state area and then reversing at least a portion of the change in the size of the state area. A method for displaying information about an individual application in a state area without displaying the animation of the state area, based on the determination that the state area is associated with the active session of the individual application and that the active session is a second type of session.

22. The method according to claim 1, wherein while a user input corresponding to the state area has not been detected, In accordance with the determination that the state area is associated with the active session of the first application and the active session of the second application, Displaying information about the first application in a first portion of the state area, and displaying a first animation of the first portion of the state area, which includes changing the size of the first portion of the state area and then reversing at least a portion of the change in the size of the first portion of the state area. A method comprising displaying information about the second application in a second portion of the state area, and displaying a second animation of the second portion of the state area, which includes changing the size of the second portion of the state area and then reversing at least a portion of the change in the size of the second portion of the state area.

23. A method according to claim 22, wherein the first animation resizes the first portion of the state region in a manner different from the second animation resizing the second portion of the state region.

24. A computer system, A display generation component having a display area that at least partially encompasses the state region, One or more processors, The system includes memory for storing one or more programs, and the one or more programs are configured to be executed by the one or more processors, To detect user input corresponding to the aforementioned state region, In response to detecting the user input corresponding to the state region, In accordance with the determination that the state area is associated with the active session of an individual application, Performing the actions associated with the aforementioned individual applications, In accordance with the determination that the aforementioned state area is not associated with an active session of an individual application, A computer system including instructions for providing feedback associated with user input without displaying information about individual applications in a state area and without performing actions associated with individual applications.

25. A computer system according to claim 24, wherein one or more programs include instructions for performing the method according to any one of claims 2 to 23.

26. A computer program including instructions, wherein the instructions are executed by a computer system having a display generating component having a display area that at least partially includes a state area, To detect user input corresponding to the aforementioned state region, In response to detecting the user input corresponding to the state region, In accordance with the determination that the state area is associated with the active session of an individual application, Performing the actions associated with the aforementioned individual applications, In accordance with the determination that the aforementioned state area is not associated with an active session of an individual application, A computer program that provides feedback associated with user input without displaying information about individual applications in the state area and without performing actions associated with individual applications.

27. A computer program according to claim 26, wherein one or more programs, when executed by the computer system, include instructions causing the computer system to perform the method described in any one of claims 2 to 23.

Images