Multi-participant live communication user interface

Abstract

To provide electronic devices with faster and more efficient methods and interfaces for live communication involving multiple participants. [Solution] The method displays a messaging user interface of a messaging application, which includes a message area containing multiple messages in a message conversation between three or more participants. Upon receiving first data indicating that a live communication session is active and available to three or more participants, a notification indicating that the live communication session is active is displayed. While the notification is displayed, upon receiving second data indicating that the live communication session between three or more participants has become inactive, the display of the notification indicating that the live communication session is active is canceled or updated to indicate that the live communication session has become inactive.

Description

Technical Field

[0001] This application relates to U.S. Provisional Patent Application No. 62 / 668,229, filed on May 7, 2018, entitled "MULTI-PARTICIPANT LIVE COMMUNICATION USER INTERFACE", which is hereby incorporated by reference in its entirety.

[0002] The present disclosure generally relates to computer user interfaces, and more specifically to technologies for live communication involving multiple participants.

Background Art

[0003] Users are increasingly using electronic devices to communicate virtually with others for personal and business reasons. Most electronic devices can provide live communication between users and among multiple participants. As an example, some electronic devices provide an interactive interface for displaying and controlling live communication.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, some technologies for live communication involving multiple participants using electronic devices are generally cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may involve multiple key presses or keystrokes. Existing technologies require more time than necessary, wasting the user's time and the device's power. This latter problem is particularly significant in battery-operated devices.

Means for Solving the Problems

[0005] Therefore, this technology provides electronic devices with faster and more efficient methods and interfaces for live communication involving multiple participants. Such methods and interfaces optionally complement or replace other methods for live communication involving multiple participants. Such methods and interfaces reduce the cognitive burden on the user and enable a more efficient human-machine interface. In the case of battery-operated computing devices, such methods and interfaces save power and extend the battery charging interval.

[0006] In some embodiments, the method is performed on an electronic device equipped with a display. The method may include displaying a messaging user interface for a messaging application, the messaging user interface including a message area containing multiple messages in a message conversation between three or more participants; receiving first data indicating that a live communication session available to three or more participants is active; displaying a notification indicating that the live communication session is active in response to receiving the first data indicating that the live communication session is active in response to receiving the first data indicating that the live communication session is active in response to receiving the first data indicating that the live communication session is active in response to receiving the first data indicating that the live communication session is active in response to receiving the first data indicating that the live communication session is active in response to receiving the second data indicating that the live communication session is no longer active in response to receiving the second data indicating that the live communication session is no longer active in response to receiving the second data indicating that the live communication session is no longer active in response to receiving the first data.

[0007] In some embodiments, a non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a messaging user interface for a messaging application, which includes a message area containing multiple messages in a message conversation between three or more participants; receiving first data indicating that a live communication session available to three or more participants is active; displaying a notification indicating that the live communication session is active in response to the receipt of the first data indicating that the live communication session is active among three or more participants; receiving second data indicating that the live communication session between three or more participants has become inactive while the notification indicating that the live communication session is active is being displayed; and stopping the display of the notification or updating the notification to indicate that the live communication session has become inactive in response to the receipt of the second data indicating that the live communication session between three or more participants has become inactive.

[0008] In some embodiments, a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a messaging user interface of a messaging application, which includes a message area containing multiple messages in a message conversation between three or more participants, and for receiving first data indicating that a live communication session available to three or more participants is active. In response to receiving the first data indicating that a live communication session between three or more participants is active, the program may include displaying a notification indicating that the live communication session is active, receiving second data indicating that the live communication session between three or more participants has become inactive while the notification indicating that the live communication session is active is being displayed, and in response to receiving the second data indicating that the live communication session between three or more participants has become inactive, the program may include ceasing to display the notification or updating the notification to indicate that the live communication session has become inactive.

[0009] In some embodiments, the electronic device comprises a display, one or more processors, and memory. The memory stores one or more programs configured to be executed by one or more processors. One or more programs include instructions for displaying a messaging user interface for a messaging application, which includes a message area containing multiple messages in a message conversation between three or more participants, and for receiving first data indicating that a live communication session is active and available to three or more participants. In response to receiving the first data indicating that a live communication session between three or more participants is active, the program may include displaying a notification indicating that the live communication session is active, receiving second data indicating that the live communication session between three or more participants has become inactive while the notification indicating that the live communication session is active is being displayed, and in response to receiving the second data indicating that the live communication session between three or more participants has become inactive, the program may include ceasing to display the notification or updating the notification to indicate that the live communication session has become inactive.

[0010] In some embodiments, the electronic device includes a display, means for displaying a messaging user interface of a messaging application, which includes a message area containing multiple messages in a message conversation between three or more participants, means for receiving first data indicating that a live communication session available to three or more participants is active, means for displaying a notification indicating that the live communication session is active in response to the reception of the first data indicating that the live communication session between three or more participants is active, means for receiving second data indicating that the live communication session between three or more participants has become inactive while the notification indicating that the live communication session is active is being displayed, and means for stopping the display of the notification or updating the notification to indicate that the live communication session has become inactive in response to the reception of the second data indicating that the live communication session between three or more participants has become inactive.

[0011] In some embodiments, the method is performed on an electronic device equipped with a display. The method includes receiving a request to display a user interface for a live communication session between two or more participants, and displaying a live communication user interface, which includes simultaneously displaying representations of multiple participants in a live communication session in response to receiving a request to display a user interface for a live communication session between two or more participants, wherein the display of representations of multiple participants in a live communication session may include, upon determination that the number of participants in the live communication session is greater than the threshold number of participants, simultaneously displaying representations of a first group of participants other than the user of the electronic device in one or more sizes larger than their respective threshold size, while displaying representations of a second group of participants, including one or more participants not included in the first group of participants, in an overflow area, with the representations of one or more other participants displayed in the overflow area being displayed in a size smaller than their respective threshold size, and upon determination that the number of participants in the live communication session is less than or equal to the threshold number of participants, displaying representations of participants other than the user of the electronic device in one or more sizes larger than their respective threshold size, without displaying them in one or more sizes smaller than their respective threshold size.

[0012] In some embodiments, a non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. One or more programs receive a request to display a user interface for a live communication session between two or more participants, and in response to receiving a request to display a user interface for a live communication session between two or more participants, the program includes a command to display a live communication user interface, which includes simultaneously displaying representations of multiple participants in a live communication session, wherein, upon determination that the number of participants in the live communication session is greater than the threshold number of participants, the representations of a first group of participants other than the user of the electronic device are simultaneously displayed in one or more sizes larger than their respective threshold sizes, while the representations of a second group of participants, including one or more participants not included in the first group of participants, are displayed in an overflow area, and the representations of one or more other participants displayed in the overflow area are displayed in sizes smaller than their respective threshold sizes, and upon determination that the number of participants in the live communication session is less than or equal to the threshold number of participants, the representations of participants other than the user of the electronic device are displayed in one or more sizes larger than their respective threshold sizes, without displaying them in one or more sizes smaller than their respective threshold sizes.

[0013] In some embodiments, a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. One or more programs receive a request to display a user interface for a live communication session between two or more participants, and in response to receiving a request to display a user interface for a live communication session between two or more participants, the program includes instructions for displaying a live communication user interface, which includes simultaneously displaying representations of multiple participants in a live communication session, wherein, upon determination that the number of participants in the live communication session is greater than the threshold number of participants, the program simultaneously displays representations of a first group of participants other than the user of the electronic device in one or more sizes larger than their respective threshold size, while displaying representations of a second group of participants, including one or more participants not included in the first group of participants, in an overflow area, with the representations of one or more other participants displayed in the overflow area being smaller than their respective threshold size, and upon determination that the number of participants in the live communication session is less than or equal to the threshold number of participants, the program displays representations of participants other than the user of the electronic device in one or more sizes larger than their respective threshold size, without displaying them in one or more sizes smaller than their respective threshold size.

[0014] In some embodiments, the electronic device comprises a display, one or more processors, and memory. The memory stores one or more programs configured to be executed by one or more processors. One or more programs receive a request to display a user interface for a live communication session between two or more participants, and in response to receiving a request to display a user interface for a live communication session between two or more participants, the program includes a command to display a live communication user interface, which includes simultaneously displaying representations of multiple participants in a live communication session, wherein, upon determination that the number of participants in the live communication session is greater than the threshold number of participants, the representations of a first group of participants other than the user of the electronic device are simultaneously displayed in one or more sizes larger than their respective threshold sizes, while the representations of a second group of participants, including one or more participants not included in the first group of participants, are displayed in an overflow area, and the representations of one or more other participants displayed in the overflow area are displayed in sizes smaller than their respective threshold sizes, and upon determination that the number of participants in the live communication session is less than or equal to the threshold number of participants, the representations of participants other than the user of the electronic device are displayed in one or more sizes larger than their respective threshold sizes, without displaying them in one or more sizes smaller than their respective threshold sizes.

[0015] In some embodiments, the electronic device includes a display, means for receiving a request to display a user interface for a live communication session between two or more participants, and means for displaying a live communication user interface, which includes simultaneously displaying representations of multiple participants in a live communication session in response to receiving a request to display a user interface for a live communication session between two or more participants, wherein the means for displaying representations of multiple participants in a live communication session includes, upon determination that the number of participants in the live communication session is greater than the threshold number of participants, simultaneously displaying representations of a first group of participants other than the user of the electronic device in one or more sizes larger than their respective threshold sizes, while displaying representations of a second group of participants, including one or more participants not included in the first group of participants, in an overflow area, and displaying the representations of one or more other participants displayed in the overflow area in a size smaller than their respective threshold sizes, and upon determination that the number of participants in the live communication session is less than or equal to the threshold number of participants, displaying representations of participants other than the user of the electronic device in one or more sizes larger than their respective threshold sizes, without displaying them in one or more sizes smaller than their respective threshold sizes.

[0016] In some embodiments, the method is performed on an electronic device equipped with a display. The method may include: displaying a live video communication user interface that simultaneously displays representations of a first participant in a live video communication session, representations of a second participant in a live video communication session, and representations of a third participant in a live video communication session; receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session while the live video communication user interface is being displayed; changing the size and / or position of the representation of the first participant in the live video communication user interface in response to receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session; changing the size and / or position of the representation of the second participant in the live video communication user interface; and changing the size and / or position of the representation of the third participant in the live video communication user interface.

[0017] In some embodiments, a non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a live video communication user interface that simultaneously displays a representation of a first participant in a live video communication session, a representation of a second participant in a live video communication session, and a representation of a third participant in a live video communication session; receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session while the live video communication user interface is being displayed; and, in response to receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session, changing the size and / or position of the representation of the first participant in the live video communication user interface, changing the size and / or position of the representation of the second participant in the live video communication user interface, and changing the size and / or position of the representation of the third participant in the live video communication user interface.

[0018] In some embodiments, a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a live video communication user interface that simultaneously displays a representation of a first participant in a live video communication session, a representation of a second participant in a live video communication session, and a representation of a third participant in a live video communication session; receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session while the live video communication user interface is being displayed; and, in response to receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session, changing the size and / or position of the representation of the first participant in the live video communication user interface, changing the size and / or position of the representation of the second participant in the live video communication user interface, and changing the size and / or position of the representation of the third participant in the live video communication user interface.

[0019] In some embodiments, the electronic device comprises a display, one or more processors, and memory. The memory stores one or more programs configured to be executed by one or more processors. The one or more programs include instructions for displaying a live video communication user interface that simultaneously displays a representation of a first participant in a live video communication session, a representation of a second participant in a live video communication session, and a representation of a third participant in a live video communication session; receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session while the live video communication user interface is being displayed; and, in response to receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session, changing the size and / or position of the representation of the first participant in the live video communication user interface, changing the size and / or position of the representation of the second participant in the live video communication user interface, and changing the size and / or position of the representation of the third participant in the live video communication user interface.

[0020] In some embodiments, the electronic device includes a display, means for displaying a live video communication user interface that simultaneously displays a representation of a first participant in a live video communication session, a representation of a second participant in a live video communication session, and a representation of a third participant in a live video communication session, means for receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session while the live video communication user interface is being displayed, and means for changing the size and / or position of the representation of the first participant in the live video communication user interface, the size and / or position of the representation of the second participant in the live video communication user interface, and the size and / or position of the representation of the third participant in the live video communication user interface in response to receiving data indicating that the activity level of the second participant in a live video communication session has risen above the activity level of the first participant in a live video communication session.

[0021] In some embodiments, the method is performed on an electronic device equipped with a display. The method may include displaying a messaging user interface for a messaging application, which includes a message area containing multiple messages between participants in a message conversation involving three or more participants, and an affordance separate from the message area for starting a live video communication session in a live video communication application; detecting an input corresponding to the selection of an affordance; and, in response to the detection of the input corresponding to the selection of an affordance, starting a process to start a live video communication session in a live video communication application, the live video communication session including the participants of the message conversation.

[0022] In some embodiments, a non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a messaging user interface for a messaging application, which includes a message area containing multiple messages between participants in a message conversation involving three or more participants, and an affordance separate from the message area for starting a live video communication session in a live video communication application, detecting input corresponding to the selection of an affordance, and, in response to the detection of input corresponding to the selection of an affordance, initiating a process to start a live video communication session including the participants of the message conversation in the live video communication application.

[0023] In some embodiments, a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a messaging user interface for a messaging application, which includes a message area containing multiple messages between participants in a message conversation involving three or more participants, and an affordance separate from the message area for starting a live video communication session in a live video communication application, and in response to the detection of the input corresponding to the affordance selection, for starting a process in the live video communication application to start a live video communication session including the participants in the message conversation.

[0024] In some embodiments, an electronic device includes a display, one or more processors, and memory. The memory stores one or more programs configured to be executed by the one or more processors. The one or more programs include a messaging user interface of a messaging application that includes a message area containing a plurality of messages among participants in a message conversation including three or more participants, and an affordance separate from the message area for initiating a live video communication session in a live video communication application. The one or more programs display the messaging user interface, detect an input corresponding to a selection of the affordance, and in response to detecting the input corresponding to the selection of the affordance, include instructions for initiating a process of initiating a live video communication session including participants in the message conversation in the live video communication application.

[0025] In some embodiments, an electronic device includes means for displaying a messaging user interface of a messaging application, the messaging user interface including a message area containing a plurality of messages among participants in a message conversation including three or more participants, and an affordance separate from the message area for initiating a live video communication session in a live video communication application, means for detecting an input corresponding to a selection of the affordance, and means for initiating a process of initiating a live video communication session including participants in the message conversation in the live video communication application in response to detecting the input corresponding to the selection of the affordance.

[0026] In some embodiments, the method is performed on an electronic device equipped with a display. The method may include displaying a user interface for a live video communication application, which includes a plurality of participant affordances corresponding to one or more participants, each including a first participant affordance corresponding to a group of three or more participants, and a new session affordance for starting a new live video communication session; detecting an input corresponding to a request to start a process for starting a new live video communication session; in response to the detection of the input, and upon receiving that the input corresponds to the selection of the first participant affordance, initiating a process for starting a new live video communication session including a group of three or more participants; and upon receiving that the input corresponds to the selection of a new session affordance, initiating a process for selecting two or more participants, and starting a new live video communication session including participants associated with the device and the two or more selected participants.

[0027] In some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device including a display. The one or more programs include a live video communication user interface of a live video communication application, including a plurality of participant affordances corresponding to one or more participants respectively, including a first participant affordance corresponding to a group of three or more participants, a new session affordance for starting a new live video communication session, display the live video communication user interface, detect an input corresponding to a request to start a process of starting a new live video communication session, and in response to the detection of the input, upon receiving that the input corresponds to a selection of the first participant affordance, start a process of starting a new live video communication session including a group of three or more participants, and upon receiving that the input corresponds to a selection of the new session affordance, start a process of selecting two or more participants, and include instructions for starting a new live video communication session including participants associated with the device and the two or more selected participants.

[0028] In some embodiments, a temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display. The one or more programs include instructions for displaying a live video communication user interface for a live video communication application, which includes a plurality of participant affordances corresponding to one or more participants, each including a first participant affordance corresponding to a group of three or more participants, and a new session affordance for starting a new live video communication session; detecting an input corresponding to a request to start the process of starting a new live video communication session; in response to the detection of the input, upon receiving that the input corresponds to the selection of the first participant affordance, initiating the process of starting a new live video communication session including a group of three or more participants; upon receiving that the input corresponds to the selection of the new session affordance, initiating the process of selecting two or more participants; and starting a new live video communication session including participants associated with the device and the two or more selected participants.

[0029] In some embodiments, the electronic device comprises a display, one or more processors, and memory. The memory stores one or more programs configured to be executed by one or more processors. The one or more programs include instructions for displaying a live video communication user interface for a live video communication application, which includes a plurality of participant affordances corresponding to one or more participants, each including a first participant affordance corresponding to a group of three or more participants, and a new session affordance for starting a new live video communication session; detecting an input corresponding to a request to start processing to start a new live video communication session; in response to the detection of the input, upon receiving that the input corresponds to the selection of the first participant affordance, initiating processing to start processing to start a new live video communication session including a group of three or more participants; upon receiving that the input corresponds to the selection of the new session affordance, initiating processing to select two or more participants; and starting a new live video communication session including participants associated with the device and two or more selected participants.

[0030] In some embodiments, the electronic device includes a display and means for displaying a live video communication user interface for a live video communication application, the means for displaying a plurality of participant affordances corresponding to one or more participants, each including a first participant affordance corresponding to a group of three or more participants, and a new session affordance for starting a new live video communication session; means for detecting an input corresponding to a request to start a process for starting a new live video communication session; and means for, in response to the detection of the input, to start a process for starting a new live video communication session including a group of three or more participants, upon receiving that the input corresponds to the selection of a first participant affordance, and to start a process for selecting two or more participants, upon receiving that the input corresponds to the selection of a new session affordance, and starting a new live video communication session including participants associated with the device and the two or more selected participants.

[0031] The executable instructions that perform these functions are optionally included in other computer program products configured to be executed by a non-temporary computer-readable storage medium or by one or more processors.

[0032] Therefore, the effectiveness, efficiency, and user satisfaction of electronic devices can be enhanced by providing faster and more efficient methods and interfaces for live communication involving multiple participants. Such methods and interfaces can complement or replace other methods for live communication involving multiple participants.

[0033] To better understand the various embodiments described, the following “Modes for Carrying Out the Invention” should be referenced in conjunction with the following drawings. Here, similar reference numerals refer to the corresponding parts throughout all those drawings. [Brief explanation of the drawing]

[0034] [Figure 1A] This is a block diagram showing a portable multifunctional device with a touch-sensitive display, according to one embodiment. [Figure 1B] This is a block diagram showing exemplary components for event processing according to some embodiments. [Figure 2] This figure shows a portable multifunctional device having a touchscreen, according to one embodiment. [Figure 3] This is a block diagram of an exemplary multifunctional device comprising a display and a touch-sensitive surface, according to some embodiments. [Figure 4A] This figure shows an exemplary user interface for an application menu on a portable multifunction device, according to some embodiments. [Figure 4B] This figure shows an exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display, according to some embodiments. [Figure 5A] This figure shows a personal electronic device according to some embodiments. [Figure 5B] This is a diagram of a personal electronic device according to some embodiments. [Figure 5C] This figure shows exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor, according to some embodiments. [Figure 5D] This figure shows exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor, according to some embodiments. [Figure 5E] This figure shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 5F] This figure shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 5G] This figure shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 5H] This figure shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 6A] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6B] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6C] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6D] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6E] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6F] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6G] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6H] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6I] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6J] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6K] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6L] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6M]This figure shows an exemplary user interface according to one of the embodiments. [Figure 6N] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6O] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6P] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6Q] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6R] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6S] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6T] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6U] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6V] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6W] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6X] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6Y] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6Z] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6AA] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6AB] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6AC] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6AD] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6AE] This figure shows an exemplary user interface according to one of the embodiments. [Figure 6AF] This figure shows an exemplary user interface according to one of the embodiments. [Figure 7A] This figure shows an exemplary method according to some embodiments. [Figure 7B] This figure shows an exemplary method according to some embodiments. [Figure 7C] This figure shows an exemplary method according to some embodiments. [Figure 7D] This figure shows an exemplary method according to some embodiments. [Figure 7E] This figure shows an exemplary method according to some embodiments. [Figure 7F] This figure shows an exemplary method according to some embodiments. [Figure 8A] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8B] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8C] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8D] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8E] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8F] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8G] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8H] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8I]This figure shows an exemplary user interface according to one of the embodiments. [Figure 8J] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8K] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8L] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8M] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8N] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8O] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8P] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8Q] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8R] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8S] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8T] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8U] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8V] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8W] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8X] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8Y] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8Z] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AA] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AB] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AC] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AD] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AE] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AF] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AG] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AH] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AI] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AJ] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AK] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AL] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AM] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AN] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AO] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AP]This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AQ] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AR] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AS] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AT] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AU] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AV] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AW] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AX] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AY] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8AZ] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BA] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BB] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BC] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BD] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BE] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BF] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BG] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BH] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BI] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BJ] This figure shows an exemplary user interface according to one of the embodiments. [Figure 8BK] This figure shows an exemplary user interface according to one of the embodiments. [Figure 9A] This figure shows an exemplary method according to some embodiments. [Figure 9B] This figure shows an exemplary method according to some embodiments. [Figure 9C] This figure shows an exemplary method according to some embodiments. [Figure 9D] This figure shows an exemplary method according to some embodiments. [Figure 9E] This figure shows an exemplary method according to some embodiments. [Figure 9F] This figure shows an exemplary method according to some embodiments. [Figure 9G] This figure shows an exemplary method according to some embodiments. [Figure 9H] This figure shows an exemplary method according to some embodiments. [Figure 9I] This figure shows an exemplary method according to some embodiments. [Figure 9J] This figure shows an exemplary method according to some embodiments. [Figure 9K] This figure shows an exemplary method according to some embodiments. [Figure 10A] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10B] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10C] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10D] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10E] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10F] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10G] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10H] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10I] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10J] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10K] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10L] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10M] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10N] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10O] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10P] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10Q] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10R] This figure shows an exemplary user interface according to one of the embodiments. [Figure 10S]This figure shows an exemplary user interface according to one of the embodiments. [Figure 11A] This figure shows an exemplary method according to some embodiments. [Figure 11B] This figure shows an exemplary method according to some embodiments. [Figure 11C] This figure shows an exemplary method according to some embodiments. [Figure 11D] This figure shows an exemplary method according to some embodiments. [Figure 11E] This figure shows an exemplary method according to some embodiments. [Figure 11F] This figure shows an exemplary method according to some embodiments. [Figure 12A] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12B] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12C] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12D] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12E] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12F] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12G] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12H] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12I] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12J] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12K] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12L] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12M] This figure shows an exemplary user interface according to one of the embodiments. [Figure 12N] This figure shows an exemplary user interface according to one of the embodiments. [Figure 13A] This figure shows an exemplary method according to some embodiments. [Figure 13B] This figure shows an exemplary method according to some embodiments. [Figure 13C] This figure shows an exemplary method according to some embodiments. [Figure 13D] This figure shows an exemplary method according to some embodiments. [Figure 14A] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14B] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14C] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14D] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14E] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14F] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14G] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14H] This figure shows an exemplary user interface according to one of the embodiments. [Figure 14I] This figure shows an exemplary user interface according to one of the embodiments. [Figure 15A] This figure shows an exemplary method according to some embodiments. [Figure 15B]This figure shows an exemplary method according to some embodiments. [Figure 15C] This figure shows an exemplary method according to some embodiments. [Modes for carrying out the invention]

[0035] The following description includes exemplary methods, parameters, etc. However, it should be understood that the purpose of such description is not to limit the scope of this disclosure, but to provide an example of the embodiments.

[0036] There is a need for electronic devices that provide efficient methods and interfaces for live communications involving multiple participants. Such technologies can reduce the cognitive burden on users participating in live communications involving multiple participants, thereby increasing productivity. Furthermore, such technologies can reduce the power consumption of the processor and battery that would normally be wasted due to redundant user input.

[0037] Figures 1A and 1B, 2, 3, 4A and 4B, and 5A to 5H below provide a description of exemplary devices that perform the technique of live communication sessions between multiple participants. Figures 6A to 6AF show exemplary user interfaces for managing event notifications. Figures 7A to 7F are flowcharts illustrating a method for conducting live communication sessions between multiple participants according to some embodiments. The user interfaces in Figures 6A to 6AF are used to illustrate the processes described later, including the processes in Figures 7A to 7F.

[0038] Figures 8A to 8BK show exemplary user interfaces for live communication sessions between multiple participants. Figures 9A to 9K are flowcharts illustrating how to conduct a live communication session between multiple participants according to some embodiments. The user interfaces in Figures 8A to 8BK are used to illustrate the processes described later, including the processes shown in Figures 9A to 9K.

[0039] Figures 10A to 10S show exemplary user interfaces for live communication sessions between multiple participants. Figures 11A to 11F are flowcharts illustrating how to conduct a live communication session between multiple participants according to some embodiments. The user interfaces in Figures 10A to 10S are used to illustrate the processes described later, including the processes shown in Figures 11A to 11F.

[0040] Figures 12A to 12N show exemplary user interfaces for live communication sessions between multiple participants. Figures 13A to 13D are flowcharts illustrating how to conduct a live communication session between multiple participants according to some embodiments. The user interfaces in Figures 12A to 12N are used to illustrate the processes described later, including the processes shown in Figures 13A to 13D.

[0041] Figures 14A to 14I show exemplary user interfaces for live communication sessions between multiple participants. Figures 15A to 15C are flowcharts illustrating how to conduct a live communication session between multiple participants according to some embodiments. The user interfaces in Figures 14A to 14I are used to illustrate the processes described later, including the processes shown in Figures 15A to 15C.

[0042] In the following description, terms such as "first," "second," etc., are used to describe various elements, but these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first touch can be referred to as a second touch, and similarly, a second touch can be referred to as a first touch. Both the first touch and the second touch are touches, but they are not the same touch.

[0043] The terms used in the descriptions of the various embodiments described herein are intended solely to describe specific embodiments and not to limit them. As used in the descriptions of the various embodiments and in the appended claims, the singular forms "a," "an," and "the" are intended to also include the plural form unless the context explicitly indicates otherwise. As used herein, the terms "and / or" should be understood to mean and include any and all possible combinations of one or more of the enumerated items relating to the description. The terms "includes," "comprises," and / or "comprising," as used herein, specify the presence of the described features, integers, steps, actions, elements, and / or components, but not to exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0044] The term "if" can be interpreted, at will, depending on the context, as meaning "when," "upon," "in response to determining," or "in response to detecting." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" can be interpreted, at will, depending on the context, as meaning "upon determining," "in response to determining," "upon detecting [the stated condition or event]," or "in response to detecting [the stated condition or event]."

[0045] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, which also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices of Apple Inc., Cupertino, California. Other portable electronic devices, such as laptop or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads), are used optionally. It should also be understood that in some embodiments, the device is not a portable communication device but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0046] The following discussion describes electronic devices that include a display and a touch-sensitive surface. However, please understand that the term "electronic device" optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0047] This device generally supports a variety of applications, including drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0048] Various applications running on a device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, are optionally adjusted and / or modified on an application-by-application basis and / or within each application. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally supports a variety of applications with intuitive and transparent user interfaces for the user.

[0049] Here, we turn our attention to embodiments of portable devices having a touch-sensitive display. Figure 1A is a block diagram of a portable multifunction device 100 having a touch-sensitive display system 112, according to some embodiments. The touch-sensitive display 112 may be referred to as a “touchscreen” for convenience, and may be known or referred to as a “touch-sensitive display system.” Device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on Device 100 (e.g., a touch-sensitive surface such as the touch-sensitive display system 112 of Device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on Device 100 (for example, generating tactile output on a touch-sensitive surface such as the touch-sensitive display system 112 of Device 100 or the touchpad 355 of Device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0050] As used herein and in the claims, the term “strength” of contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of contact (e.g., finger contact) on the touch-sensitive surface, or an alternative (substitute) for the force or pressure of contact on the touch-sensitive surface. The strength of contact includes at least four distinct values ​​and, more typically, has a range of values ​​including hundreds (e.g., at least 256) distinct values. The strength of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors located beneath or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average) to determine an estimate of the force of contact. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Instead, the size and / or variation of the contact area detected on the touch-sensing surface, the capacitance and / or variation of the touch-sensing surface adjacent to the contact, and / or the resistance and / or variation of the touch-sensing surface adjacent to the contact are optionally used as substitutes for the force or pressure of the contact on the touch-sensing surface. In some implementations, the substitute measurement for the force or pressure of the contact is used directly to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some implementations, the substitute measurement for the force or pressure of the contact is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). By using the intensity of contact as an attribute of user input, users can access additional device functions that may not be accessible to the user otherwise on reduced-size devices where the area for displaying affordances (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical control unit such as a knob or button) is limited.

[0051] As used herein and in the claims, the term “tactile output” means a physical displacement of the device relative to its previous position, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by the user through the user’s sense of touch. For example, in a situation where the device or a component of the device is in contact with the touch-sensitive surface of the user (e.g., the user’s fingers, palm, or other part of the hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or a component of the device. For example, the movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) may optionally be interpreted by the user as a “down-click” or “up-click” of a physical actuator button. In some cases, the user may feel a tactile sensation such as a “down-click” or “up-click” even when there is no movement of a physical actuator button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user’s action. As another example, movement of a touch-sensitive surface can be selectively interpreted or perceived by a user as "roughness" of that surface, even if there is no change in the smoothness of the touch-sensitive surface. Such user interpretations of touch depend on the user's personal sensory perception, but there are many touch sensory perceptions common to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., "up-click," "down-click," "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the device or its components that produce the described sensory perception of a typical (or average) user.

[0052] Device 100 is merely an example of a portable multifunction device, and it should be understood that Device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of components. The various components shown in Figure 1A may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and / or application-specific integrated circuits.

[0053] Memory 102 optionally includes high-speed random-access memory and also optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0054] The peripheral interface 118 can be used to connect the input and output peripherals of this device to the CPU 120 and memory 102. One or more processors 120 operate or execute various software programs and / or instruction sets stored in memory 102 to perform various functions for device 100 and to process data. In some embodiments, the peripheral interface 118, CPU 120 and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0055] The RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals, or electromagnetic signals to electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a participant identification module (SIM) card, and memory. The RF circuit 108 optionally communicates wirelessly with networks such as the Internet, also known as the World Wide Web (WWW), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), and other devices. The RF circuit 108 optionally includes well-known circuits for detecting near-field communication (NFC) fields, such as those used for near-field communication wireless communication. Wireless communication utilizes any and all of the following communication standards, protocols, and technologies: Global System for Mobile Communications (GSM®), Enhanced Data GSM Environment (EDGE), High-speed Downlink Packet Access (HSDPA), High-speed Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long-Term Evolution (LTE), Near Field Communication (NFC), and Wideband Code Division Multiple Access.Multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth®, Bluetooth Low Energy (BTLE), Wireless Fidelity (Wi-Fi)® (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet message access protocol (IMAP) and / or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Examples of suitable communication protocols include, but are not limited to, Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or any other communication protocols not yet developed as of the filing date of this document.

[0056] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits the audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved by the peripheral interface 118 from memory 102 and / or RF circuit 108, and / or transmitted to memory 102 and / or RF circuit 108. In some embodiments, the audio circuit 110 further comprises a headset jack (e.g., 212 in Figure 2). The headset jack provides an interface between the audio circuit 110 and a detachable audio input / output peripheral, such as output-only headphones or a headset having both output (e.g., headphones for one or both ears) and input (e.g., a microphone).

[0057] The I / O subsystem 106 connects the input / output peripherals of device 100, such as the touchscreen 112 and other input control devices 116, to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, a depth camera controller 169, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive / transmit electrical signals to / from the other input control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally connected to (or not connected to) one of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. One or more buttons (e.g., 208 in Figure 2) optionally include up / down buttons for adjusting the volume of speaker 111 and / or microphone 113. One or more buttons optionally include push buttons (e.g., 206 in Figure 2).

[0058] As described in U.S. Patent Application No. 11 / 322,549, “Unlocking a Device by Performing Gestures on an Unlock Image,” filed December 23, 2005, U.S. Patent No. 7,657,849 (this patent is incorporated herein by reference in its entirety), a quick press of a push button optionally unlocks the touchscreen 112 or optionally initiates a process using gestures on the touchscreen to unlock the device. A longer press of a push button (e.g., 206) optionally turns the device 100 on or off. The function of one or more of the buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual or soft buttons and one or more soft keyboards.

[0059] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as “graphics”). In some embodiments, optionally, some or all of the visual output corresponds to user interface objects.

[0060] The touchscreen 112 has a touch-sensing surface, sensor, or set of sensors that accept user input based on tactile and / or tactile contact. The touchscreen 112 and the display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or cessation of contact) on the touchscreen 112 and translate the detected contact into interaction with user interface objects displayed on the touchscreen 112 (e.g., one or more soft keys, icons, web pages, or images). In an exemplary embodiment, the point of contact between the touchscreen 112 and the user corresponds to the user's finger.

[0061] The touchscreen 112 optionally uses LCD (liquid crystal display) technology, LPD (polymer light-emitting display) technology, or LED (light-emitting diode) technology, but other display technologies are used in other embodiments. The touchscreen 112 and display controller 156 optionally detect touch and any movement or cessation thereof using any of several currently known or future-developed touch sensing technologies, including but not limited to capacitive technology, resistive technology, infrared technology, and surface ultrasonic technology, as well as other proximity sensor arrays or other elements for determining one or more contact points with the touchscreen 112. In exemplary embodiments, projected mutual capacitive sensing technology is used, as seen in the iPhone® and iPod Touch® by Apple Inc. of Cupertino, California.

[0062] The touch-sensitive displays in some embodiments of the touchscreen 112 are optionally similar to the multi-touch-sensitive touchpads described in U.S. Patent No. 6,323,846 (Westerman et al.), No. 6,570,557 (Westerman et al.), and / or No. 6,677,932 (Westerman), and / or U.S. Patent Application Publication No. 2002 / 0015024(A1). These documents are incorporated herein by reference in their entirety. However, the touchscreen 112 displays visual output from device 100, whereas the touch-sensitive touchpad does not provide visual output.

[0063] Touch-sensitive displays in some embodiments of the touchscreen 112 are described in the following applications. (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, "Multipoint Touch Surface Controller", (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, "Multipoint Touchscreen", (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, "Gestures For Touch Sensitive Input Devices", (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, "Gestures For Touch Sensitive Input Devices", (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices", (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, "Virtual Input Device Placement On A Touch Screen User (7) "Interface", (8) "Operation Of A Computer With A Touch Screen Interface", (9) "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard", (11 / 228,737) filed on September 16, 2005, and (11 / 367,749) filed on March 3, 2006. All of these applications are incorporated herein by reference in their entirety.

[0064] The touchscreen 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. The user optionally touches the touchscreen 112 using any suitable object or accessory, such as a stylus or finger. In some embodiments, the user interface is designed to function primarily by finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​the finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into a precise pointer / cursor position or command to perform the user's desired action.

[0065] In some embodiments, in addition to the touchscreen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touchscreen 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0066] Device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power within the portable device.

[0067] Device 100 also optionally includes one or more optical sensors 164. Figure 1A shows optical sensors coupled to an optical sensor controller 158 in the I / O subsystem 106. Optical sensors 164 optionally include charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) phototransistors. Optical sensors 164 receive light from the environment projected through one or more lenses and convert that light into data that represents an image. In conjunction with an imaging module 143 (also referred to as a camera module), optical sensors 164 optionally capture still images or video. In some embodiments, optical sensors are located on the back of device 100, opposite the touchscreen display 112 which is on the front of the device, to enable the touchscreen display as a viewfinder for acquiring still and / or video images. In some embodiments, optical sensors are located on the front of the device to optionally obtain an image of the user for video conferencing while viewing other video conference participants on the touchscreen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (for example, by rotating the lens and sensor within the device housing) so that a single optical sensor 164 is used with the touchscreen display for both video conferencing and the acquisition of still and / or video images.

[0068] Device 100 also optionally includes one or more depth camera sensors 175. Figure 1A shows a depth camera sensor coupled to a depth camera controller 169 in an I / O subsystem 106. The depth camera sensor 175 receives data from the environment to create a three-dimensional model of an object in the scene (e.g., a face) from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with an imaging module 143 (also called a camera module), the depth camera sensor 175 is optionally used to determine depth maps of different parts of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is located on the front of Device 100 to optionally obtain an image of the user with depth information for video conferencing while viewing other video conference participants on a touchscreen display, and to capture a selfie image with depth map data. In some embodiments, the depth camera sensor 175 is located on the back of the device, or on both the back and front of Device 100. In some embodiments, the position of the depth camera sensor 175 can be changed by the user (for example, by rotating the lens and sensor within the device housing) so that a single depth camera sensor 175 is used with a touchscreen display for both video conferencing and the acquisition of still and / or video images.

[0069] In some embodiments, the depth map (e.g., depth map image) includes information (e.g., values) about the distance of objects in the scene from a viewpoint (e.g., camera, light sensor, depth camera sensor). In one embodiment of the depth map, each depth pixel defines the position where the corresponding 2D pixel is located on the Z-axis of the viewpoint. In some embodiments, the depth map consists of pixels, each defined by a value (e.g., 0 to 255). For example, a value of "0" represents the furthest pixel located in the "3D" scene, and a value of "255" represents the pixel located closest to the viewpoint (e.g., camera, light sensor, depth camera sensor) in the "3D" scene. In other embodiments, the depth map represents the distance between objects in the scene and the plane of the viewpoint. In some embodiments, the depth map includes information about the relative depth of various features of the object of interest in the view from the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of the user's face). In some embodiments, the depth map includes information that allows the device to determine the z-direction contour of the object of interest.

[0070] Device 100 also optionally includes one or more contact strength sensors 165. Figure 1A shows a contact strength sensor coupled to a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, pressure-power sensors, optical force sensors, capacitive touch-sensing surfaces, or other strength sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensing surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the surrounding environment. In some embodiments, at least one contact strength sensor is located on or near the touch-sensing surface (e.g., touch-sensing display system 112). In some embodiments, at least one contact strength sensor is located on the back of Device 100, opposite the touchscreen display 112 located on the front of Device 100.

[0071] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows a proximity sensor 166 coupled to a peripheral interface 118. Alternatively, the proximity sensor 166 is optionally coupled to an input controller 160 in the I / O subsystem 106. The proximity sensor 166 optionally functions as described in U.S. Patent Applications No. 11 / 241,839, “Proximity Detector In Handheld Device,” No. 11 / 240,788, “Proximity Detector In Handheld Device,” No. 11 / 620,702, “Using Ambient Light Sensor To Augment Proximity Sensor Output,” No. 11 / 586,862, “Automated Response To And Sensing Of User Activity In Portable Devices,” and No. 11 / 638,251, “Methods And Systems For Automatic Configuration Of Peripherals,” which are incorporated herein by reference. In some embodiments, if the multifunction device is positioned near the user's ear (for example, when the user is making a call), the proximity sensor turns off and disables the touchscreen 112.

[0072] Device 100 also optionally includes one or more tactile output generators 167. Figure 1A shows a tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices, such as a speaker or other audio component, and / or electromechanical devices that convert energy into linear motion, such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts an electrical signal into a tactile output on the device). The contact intensity sensor 165 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output on device 100 that can be sensed by the user of device 100. In some embodiments, at least one tactile output generator is located on or near a touch-sensitive surface (e.g., a touch-sensitive display system 112) and optionally generates a tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward from the surface of device 100) or laterally (e.g., forward / backward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0073] Device 100 also optionally includes one or more accelerometers 168. Figure 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. The accelerometer 168 optionally functions as described in U.S. Patent Application Publication No. 20050190059, “Acceleration-based Theft Detection System for Portable Electronic Devices,” and U.S. Patent Application Publication No. 20060017692, “Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer,” both of which are incorporated herein by whole by reference. In some embodiments, information is displayed on a touchscreen display in portrait or landscape orientation based on an analysis of data received from one or more accelerometers. Device 100 optionally includes, in addition to one or more accelerometers 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information regarding the position and orientation of Device 100 (e.g., vertical or horizontal orientation).

[0074] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, as shown in Figures 1A and 3, memory 102 (Figure 1A) or memory 370 (Figure 3) stores a device / global internal state 157. The device / global internal state 157 includes one or more of the following: active application state, indicating which application is active if there are currently active applications; display state, indicating which applications, views, or other information occupy different areas of the touchscreen display 112; sensor state, including information obtained from various sensors and input control devices 116 of the device; and position information relating to the location and / or orientation of the device.

[0075] An operating system 126 (for example, an embedded operating system such as Darwin®, RTXC®, LINUX®, UNIX®, OS X®, iOS®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware components and software components.

[0076] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) are adapted to connect to other devices directly or indirectly through a network (e.g., the Internet, Wi-Fi, etc.). In some embodiments, the external ports are the same as, or similar to, the 30-pin connector used on iPod® (a trademark of Apple Inc.) devices, and / or compatible multi-pin (e.g., 30-pin) connectors.

[0077] The contact / motion module 130 (in cooperation with the display controller 156) optionally detects contact with the touchscreen 112 and other touch-sensitive devices (e.g., a touchpad or a physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger being lowered), determining the intensity of contact (e.g., the force or pressure of contact, or an alternative to the force or pressure of contact), determining whether there is movement of contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more events of a finger being dragged), and determining whether contact has ceased (e.g., detecting a finger being lifted or cessation of contact). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by a series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions can be optionally applied to a single contact (e.g., a single finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0078] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (for example, to determine whether a user has "clicked" on an icon). In some embodiments, at least a subset of the intensity thresholds is determined according to software parameters (for example, the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse "click" threshold of a trackpad or touchscreen display can be set to one of a range of predetermined thresholds without changing the hardware of the trackpad or touchscreen display. In addition, in some implementations, the user of the device is provided with software settings to adjust one or more of the set of intensity thresholds (for example, by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once with a system-level click "intensity" parameter).

[0079] The contact / motion module 130 selectively detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different movement, timing, and / or intensity of the detected contact). Therefore, gestures are selectively detected by detecting a particular contact pattern. For example, detecting a finger tap gesture involves detecting a finger-down event (e.g., at the icon's position) followed by a finger-lift-off event at the same (or substantially the same) location as the finger-down event. As another example, detecting a finger swipe gesture on the touch-sensitive surface involves detecting a finger-down event followed by one or more finger-drag events, and then a finger-lift-off event.

[0080] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other display, including components for changing the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term “graphics” includes, but is not limited to, any objects that can be displayed to a user, including, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, and animations.

[0081] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like, along with coordinate data and other graphic characteristic data as needed, specifying the graphics to be displayed, and then generates screen image data to be output to the display controller 156.

[0082] The haptic feedback module 133 includes various software components for generating commands used by a haptic output generator(s) 167 to generate haptic outputs at one or more locations on the device 100 in response to user interaction with the device 100.

[0083] The text input module 134 is optionally a component of the graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other applications that require text input).

[0084] The GPS module 135 determines the device's location and provides this information for use in various applications (for example, to the phone 138 for use in location-based calling, to the camera 143 as photo / video metadata, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0085] Application 136 optionally includes the following modules (or instruction sets), or subsets or supersets thereof: ● Contact module 137 (sometimes also called the address book or contact list), ●Telephone module 138, ●Video conferencing module 139, ● Email client module 140, ● Instant messaging (IM) module 141, ●Training support module 142, ● Camera module 143 for still images and / or video images, ●Image management module 144, ●Video player module, ● Music player module, ● Browser module 147, ●Calendar module 148, ● A widget module 149 that optionally includes one or more of the following: weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5 and other widgets obtained by the user, and user-created widgets 149-6. ●Widget creation module 150 for creating user-created widget 149-6, ● Search module 151, ● A video and music player module 152 that integrates a video player module and a music player module. ●Memo Module 153, ● Map module 154 and / or ● Online video module 155.

[0086] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, Java®-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0087] Together with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (stored, for example, in the application internal state 192 of the contact module 137 in memory 102 or memory 370), which includes adding names(s) to the address book, removing names(s) from the address book, associating telephone numbers(s) to names, email addresses(s) to names, addresses(s) to names, or other information, associating images to names, categorizing and sorting names, and providing telephone numbers or email addresses to initiate and / or facilitate communication by telephone 138, video conferencing module 139, email 140, or IM 141.

[0088] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the telephone module 138 is used to optionally input a series of characters corresponding to a telephone number, access one or more telephone numbers in the contact module 137, modify the entered telephone number, dial the corresponding telephone number, conduct a conversation, and disconnect or hang up the phone when the conversation is complete. As previously stated, wireless communication may optionally use any of several communication standards, protocols, and technologies.

[0089] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touchscreen 112, display controller 156, light sensor 164, light sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, the video conferencing module 139 includes executable commands for initiating, conducting, and ending a video conference between the user and one or more other participants, according to user instructions.

[0090] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the email client module 140 includes executable commands for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with the image management module 144, the email client module 140 makes it very easy to create and send emails containing still or video images captured by the camera module 143.

[0091] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the instant messaging module 141 includes executable instructions for entering strings corresponding to instant messages, modifying entered characters, sending corresponding instant messages (for example, using the Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocol for phone-based instant messaging, or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to both telephone-based messaging (e.g., messages sent using SMS or MMS) and internet-based messaging (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0092] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, the training support module 142 includes executable commands for creating training (e.g., having time, distance, and / or calorie consumption targets), communicating with training sensors (sports devices), receiving training sensor data, calibrating sensors used to monitor training, selecting and playing music for training, and displaying, storing, and transmitting training data.

[0093] In conjunction with the touchscreen 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0094] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable commands for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still and / or video images.

[0095] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the browser module 147 includes executable commands for browsing the Internet according to user instructions, including searching for, linking, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0096] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, the calendar module 148 includes executable commands to create, display, modify, and store calendars and data associated with calendars (e.g., calendar items, to-do lists, etc.) according to user instructions.

[0097] In conjunction with the RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, the widget module 149 is a mini-application that can be optionally downloaded and used by the user (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or a mini-application created by the user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0098] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, the widget creation module 150 is optionally used by the user to create widgets (for example, to convert a user-specified section of a web page into a widget).

[0099] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching for text, music, sound, images, videos, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0100] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music player module 152 includes executable commands that allow the user to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable commands for displaying, presenting, or otherwise playing video (on the touchscreen 112 or on an external display connected via the external port 124). In some embodiments, the device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0101] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the memo module 153 includes executable commands for creating and managing memos, to-do lists, etc., according to user instructions.

[0102] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 is optionally used to receive, display, modify, and store maps and map-associated data (e.g., driving directions, data on shops and other points of interest at a specific location or nearby, and other location-based data) in accordance with user instructions.

[0103] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions that enable the user to access, view, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen or on an external display connected via external port 124), send emails containing links to specific online videos, and manage them in other ways. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Further descriptions of online video applications can be found in U.S. Patent Provisional Application No. 60 / 936,562, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed June 20, 2007, and U.S. Patent Application No. 11 / 968,067, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed December 31, 2007, the entire contents of which are incorporated herein by reference.

[0104] Each of the modules and applications identified above corresponds to an executable instruction set and a method of this application (e.g., a computer-implemented method and other information processing methods described herein) for performing one or more of the above functions. These modules (e.g., instruction sets) do not need to be implemented as separate software program procedures or modules; therefore, in various embodiments, various subsets of these modules are optionally combined or otherwise reorganized. For example, a video player module may optionally be combined with a music player module to form a single module (e.g., video and music player module 152 in Figure 1A). In some embodiments, memory 102 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0105] In some embodiments, device 100 is a device in which the operation of a predetermined set of functions on the device is performed exclusively through a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for the operation of device 100, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.

[0106] A predetermined set of functions performed exclusively through the touchscreen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, home menu, or root menu. In such embodiments, a “menu button” is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

[0107] Figure 1B is a block diagram showing exemplary components for event processing according to some embodiments. In some embodiments, memory 102 (Figure 1A) or memory 370 (Figure 3) includes an event sorter 170 (e.g., within the operating system 126) and a corresponding application 136-1 (e.g., any of the aforementioned applications 137-151, 155, 380-390).

[0108] The event sorter 170 receives event information and determines the application 136-1 and the application view 191 of application 136-1 to which the event information will be delivered. The event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view(s) displayed on the touch-sensitive display 112 when the application is active or running. In some embodiments, the device / global internal state 157 is used by the event sorter 170 to determine which application is currently active, and the application internal state 192 is used by the event sorter 170 to determine the application view 191 to which the event information will be delivered.

[0109] In some embodiments, the application internal state 192 includes additional information such as resume information used when the application 136-1 resumes execution, user interface state information indicating information that is displayed or ready to be displayed by the application 136-1, a state queue to allow the user to return to a previous state or view of the application 136-1, and a redo / undo queue for previous actions taken by the user.

[0110] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (for example, user touch on the touch-sensitive display 112 as part of a multi-touch gesture). The peripheral interface 118 transmits information received from the I / O subsystem 106 or sensors such as the proximity sensor 166, one or more accelerometers 168 and / or the microphone 113 (through the audio circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display 112 or touch-sensitive surface.

[0111] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when there is a significant event (e.g., receiving an input that exceeds a predetermined noise threshold and / or for a longer period than predetermined).

[0112] In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognition determination module 173.

[0113] The hit view determination module 172 provides software procedures for determining where a sub-event occurred within one or more views when the touch-sensitive display 112 displays one or more views. A view consists of a control unit and other elements that the user can see on the display.

[0114] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of each application) in which a touch is detected optionally corresponds to a program level within the application's program hierarchy or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally called a hit view, and the set of events recognized as appropriate input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0115] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchy that should process the sub-events. In most situations, the hit view is the lowest-level view where the first sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source that identified it as a hit view.

[0116] The active event recognition determination module 173 determines which view(s) in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognition determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognition determination module 173 determines that all views, including the physical location of the sub-event, are actively involved views, and therefore all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if the touch sub-event is entirely confined to a region associated with one particular view, higher-level views in the hierarchy still remain actively involved views.

[0117] The event dispatcher module 174 dispatches event information to an event recognition unit (e.g., an event recognition unit 180). In embodiments including an active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores the event information acquired by the corresponding event receiving unit 182 in an event queue.

[0118] In some embodiments, the operating system 126 includes an event sorter 170. Alternatively, application 136-1 includes an event sorter 170. In yet other embodiments, the event sorter 170 is either a standalone module or part of another module stored in memory 102, such as a contact / motion module 130.

[0119] In some embodiments, application 136-1 includes a plurality of event processing units 190 and one or more application views 191, each containing instructions for processing touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognition units 180. Generally, a corresponding application view 191 includes a plurality of event recognition units 180. In other embodiments, one or more of the event recognition units 180 are part of a separate module, such as a user interface kit or a higher-level object, from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event processing unit 190 includes one or more of the following: a data update unit 176, an object update unit 177, a GUI update unit 178, and / or event data 179 received from an event sorter 170. The event processing unit 190 optionally uses or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 to update the application's internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event processing units 190. In some embodiments, one or more of the data update unit 176, object update unit 177, and GUI update unit 178 are included in the corresponding application view 191.

[0120] The corresponding event recognition unit 180 receives event information (e.g., event data 179) from the event sorter 170 and identifies events from the event information. The event recognition unit 180 includes an event receiving unit 182 and an event comparison unit 184. In some embodiments, the event recognition unit 180 also includes at least a subset of metadata 183 and event distribution commands 188 (optionally including sub-event distribution commands).

[0121] The event receiving unit 182 receives event information from the event sorter 170. The event information includes information about sub-events, such as touches or touch motions. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch motion, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, an event includes a rotation of the device from one orientation to another (e.g., from portrait to landscape, or vice versa), and the event information includes corresponding information about the device's current orientation (also called the device's orientation).

[0122] The event comparison unit 184 compares event information with a predetermined event or sub-event definition, and based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. The event definition 186 includes, for example, the definition of an event (e.g., a predetermined series of sub-events), such as event 1 (187-1) and event 2 (187-2). In some embodiments, the sub-events in event (187) include, for example, the start of a touch, the end of a touch, the movement of a touch, the stopping of a touch, and multiple touches. In one example, the definition of event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch on the displayed object at a predetermined stage (start of touch), a first lift-off at a predetermined stage (end of touch), a second touch on the displayed object at a predetermined stage (start of touch), and a second lift-off at a predetermined stage (end of touch). In another example, the definition of event 2(187-2) is a drag operation on a displayed object. This drag operation includes, for example, a touch (or contact) on the displayed object at a given stage, the movement of the touch across the touch-sensitive display 112, and the lift-off of the touch (end of the touch). In some embodiments, the event also includes information about one or more associated event processing units 190.

[0123] In some embodiments, the event definition 187 includes an event definition for each user interface object. In some embodiments, the event comparison unit 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display 112, when a touch is detected on the touch-sensitive display 112, the event comparison unit 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event processing unit 190, the event comparison unit uses the results of the hit test to determine which event processing unit 190 should be activated. For example, the event comparison unit 184 selects the event processing unit associated with the object and the sub-event that triggers the hit test.

[0124] In some embodiments, the definition of a corresponding event (187) also includes a delay action that delays the delivery of event information until it is determined whether a series of sub-events correspond to an event type in the event recognition unit.

[0125] When the corresponding event recognition unit 180 determines that a series of sub-events does not match any of the events in the event definition 186, the corresponding event recognition unit 180 enters an event impossible, event failed, or event terminated state, and thereafter ignores subsequent touch-based gesture sub-events. In this situation, if there are any other event recognition units that remain active for the hit view, those event recognition units continue to track and process the ongoing touch-based gesture sub-events.

[0126] In some embodiments, the corresponding event recognition unit 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate to the actively involved event recognition unit how the event distribution system should distribute sub-events. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how, or are capable of, event recognition units interact with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are distributed to various levels in the view hierarchy or program hierarchy.

[0127] In some embodiments, each event recognition unit 180 activates an event processing unit 190 associated with an event when one or more specific sub-events of an event are recognized. In some embodiments, each event recognition unit 180 delivers event information associated with the event to the event processing unit 190. Activating the event processing unit 190 is separate from sending (and delaying the sending of) sub-events to the respective hit view. In some embodiments, the event recognition unit 180 sets a flag associated with the recognized event, and the event processing unit 190 associated with the flag captures the flag and performs predetermined processing.

[0128] In some embodiments, the event distribution command 188 includes a sub-event distribution command that distributes event information about a sub-event without activating an event processing unit. Instead, the sub-event distribution command distributes event information to an event processing unit associated with a set of sub-events, or to a view that is actively involved. The event processing unit associated with the set of sub-events, or to a view that is actively involved, receives the event information and performs predetermined processing.

[0129] In some embodiments, the data update unit 176 creates and updates data used in application 136-1. For example, the data update unit 176 updates telephone numbers used in contact module 137 or stores video files used in video player module. In some embodiments, the object update unit 177 creates and updates objects used in application 136-1. For example, the object update unit 177 creates new user interface objects or updates the position of user interface objects. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares display information and sends it to graphics module 132 for display on touch-sensitive display.

[0130] In some embodiments, the event processing unit(s) 190 includes or has access to a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are included in a single module of their respective applications 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0131] The preceding discussion regarding the event processing of user touches on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 with input devices, and it should be understood that not all of these begin on a touchscreen. For example, mouse movements and mouse button presses adapted to the pressing or holding of one or more keyboards, touch movements such as tapping, dragging, and scrolling on a touchpad, pen stylus input, device movement, verbal instructions, detected eye movements, biometric input, and / or any combination thereof may be optionally used as inputs corresponding to sub-events that define the events to be recognized.

[0132] Figure 2 shows a portable multifunctional device 100 having a touchscreen 112 according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment, and in other embodiments described later, the user can select one or more graphics by making gestures on the graphics using, for example, one or more fingers 202 (not shown in the figure to an exact scale) or one or more styluses 203 (not shown in the figure to an exact scale). In some embodiments, the selection of one or more graphics occurs when the user discontinues contact with one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, right to left, upward and / or downward), and / or rolling of a finger in contact with the device 100 (from right to left, left to right, upward and / or downward). In some implementations or situations, unintentional contact with a graphic does not result in the selection of a graphic. For example, when the gesture corresponding to selection is a tap, a swipe gesture (swiping over an application icon) will optionally not select the corresponding application.

[0133] Device 100 also optionally includes one or more physical buttons, such as a "Home" or menu button 204. As mentioned above, the menu button 204 is optionally used to navigate to any application 136 within the set of applications that can be optionally run on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in the GUI displayed on the touchscreen 112.

[0134] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push button 206 for turning the device on / off and locking the device, one or more volume buttons 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and an external docking / charging port 124. The push button 206 is optionally used to turn the device on / off by pressing and holding the button down for a predetermined time, to lock the device by pressing and releasing the button before a predetermined time has elapsed, and / or to unlock the device or initiate an unlocking process. In alternative embodiments, device 100 also accepts verbal input through a microphone 113 to activate or deactivate certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more tactile output generators 167 for generating tactile output to the user of device 100.

[0135] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface, according to some embodiments. Device 300 does not need to be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia module device, navigation device, educational device (such as a children's learning toy), game system, or control device (e.g., a home or commercial controller). Device 300 generally includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication buses 320 optionally include circuits (sometimes called chipsets) that interconnect and control communication between system components. Device 300 generally includes an input / output (I / O) interface 330, which includes a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 (similar to the tactile output generator(s) 167 described above with reference to Figure 1A) for generating tactile output on device 300, and a sensor 359 (e.g., an optical sensor, an accelerometer, a proximity sensor, a touch sensor and / or a contact intensity sensor similar to the contact intensity sensor(s) 165 described above with reference to Figure 1A). The memory 370 includes high-speed random access memory such as DRAM, SRAM, DDR RAM or other random access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or subsets thereof, programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (Figure 1A). Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, ​​word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, while memory 102 of portable multifunction device 100 (Figure 1A) does not optionally store these modules.

[0136] Each of the elements of Figure 3 identified above is optionally stored in one or more of the aforementioned memory devices. Each of the modules identified above corresponds to an instruction set that performs the function described above. The modules or programs (e.g., instruction sets) identified above do not need to be implemented as separate software programs, procedures, or modules; therefore, in various embodiments, various subsets of these modules are combined or otherwise optionally rearranged. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0137] Next, we consider an embodiment of a user interface that is optionally implemented on, for example, a portable multi-functional device 100.

[0138] Figure 4A shows an exemplary user interface for an application menu on a portable multifunction device 100, according to one embodiment. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or subsets or supersets thereof. ● Signal strength indicators (single or multiple) for wireless communication (single or multiple) such as cellular and Wi-Fi signals 402, ●Time 404, ●Bluetooth indicator 405, ●Battery status indicator 406, ●Tray 408 containing icons for frequently used applications such as the following: ○Optionally including an indicator 414 for the number of missed calls or audiomail messages, an icon 416 labeled "Phone" for the phone module 138, ○Optionally including an indicator 410 for the number of unread emails, an icon 418 labeled "Mail" for the email client module 140, ○ Icon 420 for browser module 147, labeled "Browser", and ○ Icon 422 for the video and music player module 152, also known as the iPod (trademark of Apple Inc.) module 152, labeled as "iPod", and ● Icons for other applications such as the following: ○ Icon 424 for IM module 141, labeled "Message" ○ Icon 426 for calendar module 148, labeled "Calendar" ○ Icon 428 for image management module 144, labeled "Photo" ○ Icon 430 for camera module 143, labeled "Camera" ○ Icon 432 for online video module 155, labeled "online video" ○ Icon 434 for stock price widget 149-2, labeled "Stock Price" ○ Icon 436 for map module 154, labeled "Map" ○ Icon 438 for weather widget 149-1, labeled "Weather" ○ Icon 440 for alarm clock widget 149-4, labeled "Clock" ○ Icon 442 for training support module 142, labeled "Training Support" ○ Icon 444 for memo module 153, labeled as "Memo", and ○ An icon 446 labeled "Settings" for a settings application or module, which provides access to settings for device 100 and its various applications 136.

[0139] Note that the icon labels shown in Figure 4A are for illustrative purposes only. For example, the icon 422 for the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label for each application icon includes the name of the application to which that application icon corresponds. In some embodiments, the label for a particular application icon is different from the name of the application to which that particular application icon corresponds.

[0140] Figure 4B shows an exemplary user interface on a device (e.g., device 300 in Figure 3) that has a touch-sensitive surface 451 (e.g., tablet or touchpad 355 in Figure 3) separate from the display 450 (e.g., touchscreen display 112). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of contact on the touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating tactile output for the user of device 300.

[0141] Some of the following examples will be described with reference to input on the touchscreen display 112 (when the touch-sensing surface and the display are combined), but in some embodiments, the device detects input on a touch-sensing surface separate from the display, as shown in Figure 4B. In some embodiments, this touch-sensing surface (e.g., 451 in Figure 4B) has a principal axis (e.g., 452 in Figure 4B) corresponding to a principal axis (e.g., 453 in Figure 4B) on the display (e.g., 450). According to these embodiments, the device detects contact with the touch-sensing surface 451 (e.g., 460 and 462 in Figure 4B) at positions corresponding to their respective locations on the display (e.g., in Figure 4B, 460 corresponds to 468 and 462 corresponds to 470). Thus, when the touch-sensitive surface is separate from the display, user input detected by the device on the touch-sensitive surface (e.g., 451 in Figure 4B) (e.g., touches 460 and 462, and their movement) is used by the device to operate the user interface on the display of the multifunction device (e.g., 450 in Figure 4B). It should be understood that a similar method may be optionally used for other user interfaces described herein.

[0142] Furthermore, while the following examples primarily refer to finger inputs (e.g., finger touch, finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced with a mouse click (e.g., instead of touch), followed by cursor movement along the swipe path (e.g., instead of touch movement). As another example, a tap gesture may optionally be replaced with a mouse click while the cursor is positioned over the tap gesture location (e.g., instead of touch detection and subsequent cessation of touch detection). Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice may optionally be used simultaneously, or a mouse and finger touch may optionally be used simultaneously.

[0143] Figure 5A shows an exemplary personal electronic device 500. Device 500 includes a body 502. In some embodiments, device 500 may include some or all of the features described with respect to devices 100 and 300 (e.g., Figures 1A to 4B). In some embodiments, device 500 has a touch-sensitive display screen 504 (hereinafter, touchscreen 504). Instead of, or in addition to, touchscreen 504, device 500 has a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally has one or more intensity sensors for detecting the intensity of the applied contact (e.g., touch). One or more intensity sensors on touchscreen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of device 500 may respond to touches based on the intensity of the touch, meaning that touches of different intensity may invoke different user interface behaviors on device 500.

[0144] Exemplary techniques for detecting and processing touch intensity are found, for example, in related applications, International Patent Application PCT / US2013 / 040061, filed 8 May 2013, entitled "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application" (published as International Publication No. 2013 / 169849), and PCT / US2013 / 069483, filed 11 November 2013, entitled "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships" (published as International Publication No. 2014 / 105276), each of which is incorporated herein by reference in its entirety.

[0145] In some embodiments, the device 500 has one or more input mechanisms 506 and 508. The input mechanisms 506 and 508, if included, can be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, the device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, allow the device 500 to be attached to, for example, a hat, eyewear, earrings, necklace, shirt, jacket, bracelet, watch band, chain, trousers, belt, shoes, wallet, backpack, etc. These attachment mechanisms allow the device 500 to be worn by a user.

[0146] Figure 5B shows an exemplary personal electronic device 500. In some embodiments, the device 500 may include some or all of the components described with respect to Figures 1A, 1B, and 3. The device 500 has a bus 512 that operably connects an I / O unit 514 to one or more computer processors 516 and memory 518. The I / O unit 514 may be connected to a display 504, which may have a touch-sensing component 522 and optionally an intensity sensor 524 (e.g., a contact intensity sensor). Furthermore, the I / O unit 514 may be connected to a communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near-field communication (NFC), cellular, and / or other wireless communication technologies. The device 500 may include input mechanisms 506 and / or 508. The input mechanism 506 may optionally be, for example, a rotatable input device or a pressable and rotatable input device. The input mechanism 508 may optionally be, in some examples, a button.

[0147] The input mechanism 508 is optionally a microphone in some examples. The personal electronic device 500 optionally includes various sensors such as a GPS sensor 532, an accelerometer 534, a direction sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or a combination thereof, all of which can be operably connected to the I / O unit 514.

[0148] The memory 518 of the personal electronic device 500 may include one or more non-temporary computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, can cause the computer processors to perform the techniques described below, including, for example, processes 700, 900, 1100, 1300, and 1500 (Figures 7A-7F, 9A-9K, 11A-11F, 13A-13D, and 15A-15C). The computer-readable storage media may be any medium capable of tangibly containing or storing computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a temporary computer-readable storage medium. In some examples, the storage medium is a non-temporary computer-readable storage medium. Examples of non-temporary computer-readable storage media include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state memory such as flash and solid-state drives. The personal electronic device 500 is not limited to the components and configurations shown in Figure 5B, and may include other or additional components in multiple configurations.

[0149] As used herein, the term “affordance” refers to user-interactive graphical user interface objects that are optionally displayed on the display screens of devices 100, 300, and / or 500 (Figures 1, 3, and 5A-5B). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each optionally constitute an affordance.

[0150] As used herein, the term “focus selector” refers to an input element that indicates the current portion of the user interface with which the user is interacting. In some implementations, including a cursor or other position marker, the cursor functions as a “focus selector,” and when input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in Figure 3, or touch-sensitive surface 451 in Figure 4B) while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), that particular user interface element is adjusted according to the detected input. In some implementations, including a touchscreen display that enables direct interaction with user interface elements on the touchscreen display (e.g., touch-sensitive display system 112 in Figure 1A, or touchscreen 112 in Figure 4A), the touch detected on the touchscreen acts as a “focus selector,” so that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of ​​the user interface to another without the movement of a corresponding cursor or touch on the touchscreen display (for example, by using the tab key or arrow keys to move focus from one button to another), and in those implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is, in general, a user interface element (or touch on the touchscreen display) controlled by the user to communicate the user's intended interaction with the user interface (for example, by indicating to the device the user interface element that the user intends to interact with).For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button indicates that the user is attempting to activate the corresponding button (rather than other user interface elements displayed on the device's display).

[0151] As used herein and in the claims, the term “characteristic intensity” of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predetermined number of intensity samples, or on a set of intensity samples collected over a predetermined period of time (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) for a predetermined event (e.g., after detection of contact, before detection of lift-off of contact, before or after detection of the start of movement of contact, before detection of the end of contact, before or after detection of an increase in contact intensity, and / or before or after detection of a decrease in contact intensity). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum intensity of the contact, the mean value of the contact intensity, the average value of the contact intensity, the top 10% of the contact intensity, half of the maximum intensity of the contact intensity, 90% of the maximum intensity of the contact intensity, etc. In some embodiments, the duration of contact is used to determine the characteristic intensity (for example, when the characteristic intensity is the average intensity of contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether the operation was performed by a user. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a first action is performed as a result of contact with a characteristic intensity not exceeding the first threshold, a second action is performed as a result of contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold, and a third action is performed as a result of contact with a characteristic intensity exceeding the second threshold. In some embodiments, the comparison between the characteristic intensity and one or more thresholds is not used to determine whether to perform the first or second action, but rather to determine whether to perform one or more actions (for example, whether to perform each action or refrain from performing each action).

[0152] Figure 5C shows the detection of multiple contacts 552A-552E on the touch-sensitive display screen 504 by multiple intensity sensors 524A-524D. Figure 5C also includes an intensity diagram showing the current intensity measurements of intensity sensors 524A-524D against intensity units. In this example, the intensity measurements of intensity sensors 524A and 524D are 9 intensity units each, and the intensity measurements of intensity sensors 524B and 524C are 7 intensity units each. In some implementations, the total intensity is the sum of the intensity measurements of the multiple intensity sensors 524A-524D, which in this example is 32 intensity units. In some embodiments, each contact is assigned a specific intensity, which is a portion of the total intensity. Figure 5D shows the assignment of the total intensity to contacts 552A-552E based on their distance from the center of force 554. In this example, contacts 552A, 552B, and 552E are each assigned a contact intensity of 8 intensity units of the total intensity, and contacts 552C and 552D are each assigned a contact intensity of 4 intensity units of the total intensity. More generally, in some implementations, each contact j is assigned an intensity Ij, which is a portion of the total intensity A, according to a predetermined mathematical function Ij = A·(Dj / ΣDi), where Dj is the distance from the center of force to each contact j, and ΣDi is the sum of the distances from the center of force to each of the contacts (e.g., from i=1 to the end). The operations described with reference to Figures 5C to 5D can be performed using electronic devices similar to or identical to devices 100, 300, or 500. In some embodiments, the characteristic intensity of a contact is based on one or more intensities of the contact. In some embodiments, an intensity sensor is used to determine a single characteristic intensity (e.g., a single characteristic intensity of a single contact). Please note that while the intensity diagram is not part of the displayed user interface, it is included in Figures 5C-5D for the reader's convenience.

[0153] In some embodiments, a portion of the gesture is identified for the purpose of determining characteristic intensity. For example, a touch-sensitive surface optionally receives a continuous swipe contact that transitions from a starting position to an ending position, where the intensity of the contact increases. In this example, the characteristic intensity of the contact at the ending position is optionally based on only a portion of the continuous swipe contact (e.g., only the portion of the swipe contact at the ending position), rather than the entire continuous swipe contact. In some embodiments, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some situations, these smoothing algorithms remove small increases or decreases in the intensity of the swipe contact for the purpose of determining characteristic intensity.

[0154] The intensity of contact on the touch-sensitive surface is optionally characterized to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to the intensity at which the device performs an operation generally associated with clicking a physical mouse button or trackpad. In some embodiments, the deep press intensity threshold corresponds to the intensity at which the device performs an operation different from an operation typically associated with clicking a physical mouse button or trackpad. In some embodiments, if contact is detected at a characteristic intensity below the light press intensity threshold (for example, below a slight contact detection intensity threshold below which contact is no longer detected), the device moves the focus selector in accordance with the movement of contact on the touch-sensitive surface without performing the actions associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise specified, these intensity thresholds are consistent across different sets of user interface diagrams.

[0155] An increase in the characteristic intensity of contact from an intensity below a light pressure intensity threshold to an intensity between the light and deep pressure intensity thresholds may be referred to as a "light pressure" input. An increase in the characteristic intensity of contact from an intensity below a deep pressure intensity threshold to an intensity above a deep pressure intensity threshold may be referred to as a "deep pressure" input. An increase in the characteristic intensity of contact from an intensity below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and a light pressure intensity threshold may be referred to as detection of contact on the touch surface. A decrease in the characteristic intensity of contact from an intensity above a contact detection intensity threshold to an intensity below a contact detection intensity threshold may be referred to detection of lift-off of contact from the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0156] In some embodiments described herein, one or more operations are performed in response to the detection of a gesture including each press input, or in response to the detection of each press input made at each (or more) contact, where each press input is detected at least in part on the detection of an increase in the intensity of the contact (or more) above a press input intensity threshold. In some embodiments, each operation is performed in response to the detection of an increase in the intensity of each contact (e.g., a "downstroke" of each press input) above a press input intensity threshold. In some embodiments, a press input includes an increase in the intensity of each contact above a press input intensity threshold and a decrease in the intensity of the contact below a subsequent press input intensity threshold, where each operation is performed in response to the detection of a decrease in the intensity of each contact below a subsequent press input threshold (e.g., an "upstroke" of each press input).

[0157] Figures 5E to 5H show the light pressure intensity threshold (for example, "IT") in Figure 5E. L From an intensity less than ) to the deep pressing intensity threshold (for example, "IT" in Figure 5H) DThe device indicates the detection of a gesture, including a pressing input corresponding to an increase in the intensity of contact 562 that reaches an intensity exceeding a deep pressing intensity threshold (e.g., "IT"). The gesture made by contact 562 is detected on the touch-sensitive surface 560 when the cursor 576 is displayed on application icon 572B corresponding to app 2 on the displayed user interface 570, which includes application icons 572A to 572D displayed in a predetermined area 574. In some embodiments, the gesture is detected on the touch-sensitive display 504. An intensity sensor detects the intensity of contact on the touch-sensitive surface 560. The device indicates that the intensity of contact 562 reaches a deep pressing intensity threshold (e.g., "IT"). D It is determined that the pressure has exceeded a peak (e.g., "IT") during the gesture. Contact 562 is maintained on the touch sensing surface 560. In response to the detection of a gesture, and during the gesture, a deep pressure intensity threshold (e.g., "IT") is determined to have been reached. D In response to a contact 562 with an intensity exceeding '', the app displays reduced representations 578A-578C (e.g., thumbnails) of recently opened documents for app 2, as shown in Figures 5F-5H. In some embodiments, the intensity compared to one or more intensity thresholds is the characteristic intensity of the contact. Note that the intensity diagram of contact 562 is not part of the displayed user interface, but is included in Figures 5E-5H for the reader's convenience.

[0158] In some embodiments, the display of expressions 578A to 578C includes animation. For example, expression 578A is initially displayed near the application icon 572B, as shown in Figure 5F. As the animation progresses, expression 578A moves upward, and expression 578B is displayed near the application icon 572B, as shown in Figure 5G. Then, as shown in Figure 5H, expression 578A moves upward, expression 578B moves upward toward expression 578A, and expression 578C is displayed near the application icon 572B. Expressions 578A to 578C form an array above the icon 572B. In some embodiments, the animation progresses according to the intensity of contact 562, as shown in Figures 5F to 5G, where the intensity of contact 562 reaches a deep pressing intensity threshold (e.g., "IT").D As it increases towards ") ", expressions 578A to 578C appear and move upward. In some embodiments, the intensity based on the progress of the animation is the characteristic intensity of the contact. The operations described with reference to FIGS. 5E to 5H can be performed using an electronic device similar to or the same as device 100, 300, or 500.

[0159] In some embodiments, the device employs intensity hysteresis to avoid unexpected inputs, sometimes called "jitter", and the device defines or selects a hysteresis intensity threshold having a predetermined relationship with the pressing input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the pressing input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable ratio of the pressing input intensity threshold). Thus, in some embodiments, the pressing input includes an increase in the corresponding contact intensity above the pressing input intensity threshold and a subsequent decrease in the contact intensity below the hysteresis intensity threshold corresponding to the pressing input intensity threshold, and the corresponding operation is performed in response to detecting a subsequent decrease in the corresponding contact intensity below that hysteresis intensity threshold (e.g., the "upstroke" of the corresponding pressing input). Similarly, in some embodiments, the pressing input is detected only when the device detects an increase in the contact intensity from an intensity below the hysteresis intensity threshold to an intensity above the pressing input intensity threshold and optionally a subsequent decrease in the contact intensity to an intensity below the hysteresis intensity, and the corresponding operation is performed in response to the detection of that pressing input (e.g., an increase in the contact intensity or a decrease in the contact intensity depending on the situation).

[0160] For ease of explanation, the description of an operation performed in response to a pressing input associated with a pressing input intensity threshold, or in response to a gesture including the pressing input, is based on an increase in the intensity of contact exceeding the pressing input intensity threshold, an increase in the intensity of contact from an intensity below the hysteresis intensity threshold to an intensity exceeding the pressing input intensity threshold, a decrease in the intensity of contact below the pressing input intensity threshold, and / or a decrease in the intensity of contact below the hysteresis intensity threshold corresponding to the pressing input intensity threshold, and is optionally triggered in response to detecting any of these. Additionally, in an example described as an operation being performed in response to detecting a decrease in the intensity of contact below the pressing input intensity threshold, the operation is optionally performed in response to detecting a decrease in the intensity of contact that corresponds to and is below a hysteresis intensity threshold lower than the pressing input intensity threshold.

[0161] As used herein, an "installed application" refers to a software application that has been downloaded to an electronic device (e.g., device 100, 300, and / or 500) and is ready to be launched (e.g., opened) on that device. In some embodiments, a downloaded application becomes an installed application by an installation program that extracts a program portion from the downloaded package and integrates the extracted portion with the operating system of the computer system.

[0162] As used herein, the terms "open application" or "running application" refer to a software application for which state information (e.g., as part of device / global internal state 157 and / or application internal state 192) is being maintained. An open or running application is optionally any one of the following types of applications. ● An active application that is currently displayed on the display screen of the device being used by the application ● Background applications (or background processes) that are not currently displayed but whose one or more processes related to that application are being handled by one or more processors, and ● An application that is not running but has state information stored in memory (volatile and / or non-volatile, respectively) that can be used to resume the execution of that application, either suspended or suspended.

[0163] As used herein, the term “closed application” refers to a software application that does not retain state information (for example, state information about a closed application is not stored in the device’s memory). Therefore, closing an application includes stopping and / or removing application processing related to that application, and removing state information related to that application from the device’s memory. Generally, opening a second application while a first application is running does not close the first application. When the second application is displayed and the display of the first application is stopped, the first application becomes a background application.

[0164] Here, we focus on embodiments of user interfaces ("UI") and related processing implemented on electronic devices such as portable multifunction device 100, device 300, or device 500.

[0165] Figures 6A to 6AF show exemplary user interfaces for initiating and joining a live communication session (e.g., from a messaging application) according to some embodiments. The user interfaces in these figures are used to illustrate the processes described later, including the processes shown in Figures 7A to 7F.

[0166] Figure 6A shows a device 600 equipped with a touch-sensitive display 602. In some embodiments, device 600 includes one or more features of devices 100, 300, and 500. In some embodiments, device 600 also includes a camera 603, which includes an image sensor capable of capturing data representing at least a portion of the light spectrum (e.g., visible light, infrared light, or ultraviolet light). In some embodiments, camera 603 includes multiple image sensors and / or other types of sensors. In addition to capturing data representing sensed light, in some embodiments, camera 603 may capture other types of data, such as depth data. For example, in some embodiments, camera 603 also captures depth data using techniques based on speckle, time-of-flight, parallax, or focus. The image data captured by device 600 using camera 603 includes data corresponding to a portion of the light spectrum for a scene within the camera's field of view. In addition, in some embodiments, the captured image data also includes depth data for the light data. In some other embodiments, the captured image data includes enough data to determine or generate depth data for a portion of the optical spectrum.

[0167] In some examples, the electronic device 600 includes a depth camera (e.g., as part of camera 603), such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light-emitting device (e.g., an optical projector), such as an infrared floodlight, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate the subject while images are being captured by the visible light camera and the depth camera (e.g., an IR camera), and information from the depth camera and the visible light camera is used to determine the depth map of different parts of the subject captured by the visible light camera. In some embodiments, the depth map (e.g., a depth map image) includes information (e.g., values) about the distance from a viewpoint (e.g., a camera) to objects in the scene. In one embodiment of the depth map, each depth pixel defines the position where the corresponding 2D pixel is located on the Z-axis of the viewpoint. In some examples, the depth map consists of pixels, each pixel defined by a value (e.g., 0 to 255). For example, a value of "0" represents the pixel located furthest away in a "3D" scene, and a value of "255" represents the pixel located closest to the viewpoint (e.g., camera) in a "3D" scene. In other examples, the depth map represents the distance between an object in the scene and the plane of the viewpoint. In some embodiments, the depth map includes information about the relative depth of various features of the object of interest in the view from the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of the user's face). In some embodiments, the depth map includes information that allows the device to determine the z-direction contour of the object of interest. In some embodiments, the lighting effects described herein are displayed using parallax information from two cameras for the rear image (e.g., two visible light cameras) and depth information from a depth camera combined with image data from a visible light camera for the front image (e.g., a selfie image).In some embodiments, the same user interface is used when determining depth information using two visible light cameras and when determining depth information using only a depth camera, providing a consistent experience to the user even when determining the information used to generate lighting effects using entirely different techniques. In some embodiments, while applying one of the lighting effects and displaying the camera user interface, the device detects a camera switching affordance selection and switches from the front camera (e.g., depth camera and visible light camera) to the rear camera (e.g., two spaced-apart visible light cameras) (or vice versa), while maintaining the display of the user interface controller for applying the lighting effect and replacing the display of the front camera's field of view with the rear camera's field of view (or vice versa).

[0168] As shown in Figure 6A, device 600 displays the messaging user interface 604 of the messaging application. The messaging user interface 604 includes a message area 606 containing messages 608A to 608D in a message conversation between a participant group ("The Dream Team") that includes three or more participants.

[0169] While displaying the messaging user interface 604, device 600 receives data indicating that a live communication session is available to participants of the group The Dream Team and that the live communication session is active (e.g., an invitation to join the live communication session). In some embodiments, the live communication session is an audio communication session or an audio / video communication session.

[0170] Upon receiving data indicating that a live communication session is active, device 600 displays a notification indicating that the live communication session is active. As shown in Figure 6B, the notification includes a notification 614 in the message area 606, indicating that Pablo has started the live communication session, as a message from participant Pablo. Notification 614 indicates the type of live communication session by the text "video" and a video camera icon. Notification 614 also indicates the number of currently active participants in the live communication session ("1 active"). In some embodiments, notification 614 indicates the names of one or more currently active participants in the live communication session. Notification 614 also includes a live communication participation affordance 614A, which, when selected, causes device 600 to initiate the process of joining the live communication session represented by notification 614. The technique for joining a live communication session is described in more detail below.

[0171] Optionally, in response to receiving data indicating that a live communication session is active, device 600 generates an audio output indicating that a live communication session is active. In some embodiments, the audio output is distinct from audio outputs related to messaging applications (e.g., incoming message notifications) and audio outputs related to telephone applications (e.g., ringtones). In some embodiments, the audio output is a hybrid of audio related to message notifications and audio related to incoming call notifications. In some embodiments, the audio output shares audio characteristics (e.g., common sound quality, timbre, rhythm) with both call and message notification audio outputs.

[0172] In response to receiving data indicating that a live communication session is active, device 600 also displays a live communication affordance 610. In some embodiments, the live communication affordance 610 is displayed (e.g., grayed out) before the live communication session begins, and is visually enhanced (e.g., highlighted) or modified (e.g., not grayed out) once the live communication session begins. In some embodiments, the selection of the live communication affordance 610 or the optional affordance 612 initiates the process of joining the live communication session. In some embodiments, the notification 614 and / or the live communication affordance 610 are displayed without audio and / or haptic output (e.g., incoming call output) (e.g., displayed in an active message conversation that the user may already be paying attention to).

[0173] As shown in Figure 6C, when an additional message is added to a message conversation within the message area 606, the notification 614 moves upward to a different location within the message area 606. In some embodiments, the device 600 moves the notification 614 to a location outside the message area (for example, the notification 614 moves from the message to a banner at the top of the display 602, outside the message area 606).

[0174] In some embodiments, device 600 moves notification 614 in response to the fulfillment of notification relocation criteria. In some embodiments, the notification relocation criteria include one or more criteria selected from the group consisting of: a predetermined amount of time elapsed (e.g., elapsed since notification 614 was displayed), discontinuation of the display of the messaging user interface 604, closing of the messaging application, and / or (e.g., by a new incoming message occupying the message area 606) the entirety or a portion of notification 614 being hidden within the message area 606. In some embodiments, notification 614 is moved to the bottom, right, or left of display 602. In some embodiments, notification 614 is a banner notification that appears near the top of display 612 in direct response to the receipt of data indicating that a live communication session is available.

[0175] In some embodiments, device 600 receives reminder data indicating a reminder to join a live communication session (e.g., a reminder sent by a participant during a conversation or after a predetermined time). As shown in Figure 6D, upon receiving data indicating a reminder to join a live communication session, device 600 displays a full-screen notification 616 and outputs an audio output 618. In some embodiments, audio output 618 is different from audio outputs related to messaging applications (e.g., incoming message notifications) and audio outputs related to telephone applications (e.g., ringtones). In some embodiments, audio output 618 is a hybrid of audio related to message notifications and audio related to incoming notifications. In some embodiments, audio output 618 is shorter than audio outputs related to telephone applications (e.g., ringtones) and longer than audio outputs related to messaging applications (e.g., incoming message notifications). In some embodiments, a ringtone includes multiple repetitions of the corresponding audio output, while audio output 618 is a single repetition of the corresponding audio output. In some embodiments, audio output 618 shares audio characteristics (e.g., common sound quality, timbre, rhythm) with both call and message notification audio outputs. In some embodiments, device 600 outputs haptic output. As shown in Figure 6D, the full-screen notification 616 includes the group name ("THE DREAM TEAM"), the names of the currently active participants in the live communication session ("PABLO, STEPHEN"), and the name of the participant who initially started the live communication session or issued a reminder ("PABLO"). In some embodiments, the full-screen notification 616 also includes the names of the participants invited to join the live communication session (for example, in the order in which they joined the group).The full-screen notification 616 also includes a response affordance 620 for joining a live communication session (e.g., joining the live communication session immediately or displaying a menu with options for joining the live communication session), a "later" affordance 622 for declining the full-screen notification 616, and a message affordance 624 for returning to the messaging user interface 604 or sending a message to the Dream Team group. The full-screen notification also includes an image (e.g., a video of the user from device 600 from camera 603) (e.g., as a preview of the image of the user that other participants in the live communication session will see if the user joins).

[0176] As shown in Figure 6D, device 600 receives user input 650A (e.g., tap) corresponding to the selection of the “later” affordance 622 (e.g., detect). As shown in Figure 6E, in response to user input 650A, device 600 stops displaying the full-screen notification 616 and returns to the messaging user interface 604.

[0177] As shown in Figure 6E, device 600 receives (e.g., detects) user input 650B (e.g., tap) corresponding to the selection of option affordance 612. In response to receiving user input 650B, device 600 extends the header of the messaging user interface 604 to include audio affordance 626A, video join affordance 626B, and group detail affordance 626C.

[0178] As shown in Figure 6F, device 600 receives (e.g., detects) a user input 650C (e.g., a tap) corresponding to the selection of the video join affordance 626B. In some embodiments, device 600 joins the live communication session directly in response to receiving the user input 650B (e.g., without requiring additional input). In some embodiments, device 600 joins the live communication session and transmits both video and audio. In some embodiments, device 600 joins the live communication session and transmits only audio (without transmitting video). In some embodiments, depending on the selection of the audio affordance 626A, device 600 either joins the live communication session with audio only or initiates a separate conference call with the group participants.

[0179] As shown in Figure 6G, in response to receiving user input 650C, device 600 displays the live communication interface 628 of the live communication application. The live communication interface 628 includes a group display ("THE DREAM TEAM"), an image (e.g., video of the user of device 600 from the front camera of device 600 (e.g., camera 603)), a display of currently active participants in the live communication session ("PABLO, STEPHEN"), a display of the participant who started the live communication session ("Video call from PABLO"), and a call control menu 630. The call control menu 630 includes handle affordances 631, effect affordances 632, call affordances 634, and menu affordances 636.

[0180] As shown in Figure 6G, device 600 receives (e.g., detects) user input 650D on menu affordance 636. As shown in Figure 6H, in response to user input 650D, device 600 extends the call control menu 630 to display additional information and controllers regarding the live communication session. In some embodiments, device 600 extends the call control menu 630 in response to an upward swipe starting near handle affordance 631. When extended, the call control menu 630 further includes audio on / off affordance 638, audio source menu affordance 640, video on / off affordance 642, group name 644 (with a display of the number of participants in the group), group message affordance 646, list 648 containing affordances 648A-648E corresponding to group participants, and participant add affordance 652. In some embodiments, upon receiving a selection of a group message affordance 646, the device 600 activates and / or displays a messaging user interface 604 (e.g., Figure 6F) for sending messages to group participants.

[0181] As shown in FIG. 6H, the affordances of the participants in list 648 include the display of the communication state of each participant regarding the live communication session. Representation 648A indicates that participant Pablo is connected to the live communication session and is providing video data (e.g., a live media stream including video and audio data). Representation 648B indicates that participant Stephen is connected and is providing audio data (e.g., without video). Representations 648C and 648D indicate that participants Marcel and Allison are not connected to the live communication session respectively (e.g., not providing video and audio data). A participant who is not connected has either never participated in the live communication session or had previously participated in the live communication session but has withdrawn from the session. Exemplary types of participant communication states include audio only (e.g., the participant is communicating using only audio), video (e.g., the participant is on a call using video and audio), video paused (e.g., the participant's video is paused), undecoded video (e.g., due to latency issues, issues with the format of the video stream, etc., the participant's video stream cannot be processed), exited (e.g., the participant has exited the live communication session), and waiting to participate (e.g., the participant has been invited to the live communication session but has not yet participated in the live communication session).

[0182] In some embodiments, the participants in a live video communication include group participants currently connected to the communication session and group participants not currently connected to the live video communication session. That is, regardless of the connection state, all group participants are considered participants in the live communication session. When the live communication session is active, a participant can participate (connect) in the live communication session or withdraw (disconnect) from the session.

[0183] As shown in Figure 6I, device 600 receives (e.g., detects) a user input 650E (e.g., a tap) corresponding to the selection of the video on / off affordance 642. As shown in Figure 6J, in response to receiving the user input 650E, device 600 modifies the video on / off affordance 642 to indicate the video off state and modifies the call affordance 634 to change the representation of the video camera in Figure 6I to the representation of the telephone in Figure 6J, thereby indicating the audio-only state. In the audio-only state, device 600 participates in the live communication session with audio only (e.g., device 600 does not provide video data).

[0184] As shown in Figure 6J, device 600 receives (e.g., detects) user input 650F (e.g., a tap) corresponding to the selection of call affordance 634. As shown in Figure 6K, in response to receiving user input 650F, device 600 initiates a connection to join a live communication session in audio-only mode. As shown in Figure 6K, device 600 discontinues displaying the extended call control menu 630 (e.g., device 600 reduces the call control menu 630) and displays the connection screen 654. In some embodiments, device 600 initiates a connection to join a live communication session and displays the connection screen 654 in response to user input 650C on video join affordance 626B. In some embodiments, notification 614 includes an affordance (e.g., a "Join" icon on notification 614 or the entire notification 614 becomes selectable). In some embodiments, device 600 initiates a connection to join a live communication session and displays the connection screen 654 in response to the selection on notification 614.

[0185] As shown in Figure 6L, after device 600 joins a live communication session, device 600 displays a live communication user interface 656 that includes a representation 658 of the user of device 600, a representation 670A of participant Pablo (e.g., the participant who started the live communication session), and a representation 670B of participant Stephen. Depending on the communication state shown in Listing 648, the representation 670A of participant Pablo includes a live video stream. Since participant Stephen provides only audio data, the representation 670B of participant Pablo includes an avatar. In some embodiments, the layout of the representations on the live communication user interface 656 is based on the number of participants currently connected to the live communication session (e.g., two participants and the user of device 600). The layout shown in Figure 6L is referred to as a 2-to-1 layout, having a first position corresponding to the position of representation 670A and a second position corresponding to the position of representation 670B.

[0186] In some embodiments, when device 600 joins a live communication session, device 600 determines whether the number of participants (e.g., active or complete (active and invited)) exceeds a predetermined threshold. Upon determining that the number of participants exceeds the threshold, device 600 displays an overflow area with an overflow representation (e.g., a thumbnail), as described in more detail below. Alternatively, upon determining that the number of participants does not exceed the threshold, device 600 does not display an overflow area with an overflow representation.

[0187] When device 600 joins a live communication session, device 600 transmits a live media stream to other participants (for example, participants who are also in the live communication session). If a participant associated with another device is in the live communication session and device 600 also joins, device 600 receives the media stream corresponding to that participant.

[0188] As shown in Figure 6L, while displaying the live communication user interface 656, device 600 receives (e.g., detects) user input 650G (e.g., a tap) on display 602. As shown in Figure 6M, in response to receiving user input 650G, device 600 displays the call control menu 630. Representations 670A and 670B are reduced in size, and representations 670B and 658 are moved upward on display 602 so that the call control menu 630 does not overlap with the representations. In contrast to the configuration shown in Figure 6L where representations 670A and 670B are aligned vertically and do not overlap, when the call control menu 630 is displayed in Figure 6M, representations 670A and 670B overlap with an offset from each other.

[0189] As shown in Figure 6N, device 600 receives (e.g., detects) user input 650H (e.g., a tap) on menu affordance 636 within call control affordance 630. As shown in Figure 6O, in response to receiving user input 650H, device 600 extends the call control menu 630 (e.g., as described with respect to Figure 6H). In some embodiments, device 600 displays a reminder affordance in the call control menu 630 for representations of participants who are not connected to the live communication session. As shown in Figure 6O, listing 648 indicates that participants Marcel and Allison are not connected to the live communication session. Since participants Marcel and Allison (e.g., devices associated with participants Marcel and Allison) are not connected to the live communication session, the extended call control menu 630 includes outgoing affordances 662, 664 in the representations 648C and 648D for participants Marcel and Allison, respectively, within listing 648. Depending on the selection of outgoing affordance 662 or 664, device 600 causes a notification (e.g., an audio output such as a call or outgoing call) to be provided on the device associated with the participant corresponding to the selected outgoing affordance. In some embodiments, the notification is a new notification different from a previous notification sent to the participant. In some embodiments, the new notification is more interfering than the previous notification (e.g., an audio ringtone, haptic output, or full-screen notification, compared to a banner notification without audio or haptic output). In this way, a user can remind participants who are not participating in a live communication session that the live communication session is active and that participants can join (for example, in the same way that a full-screen notification shown in Figure 6D reminds the user of device 600).

[0190] As shown in Figure 6O, device 600 receives (e.g., detects) user input 650I (e.g., a tap) corresponding to the selection of video on / off affordance 642. In response to receiving user input 650I, device 600 provides a live video stream (e.g., from the camera on device 600) to the live communication session and modifies video on / off affordance 642 to indicate that video is on, as shown in Figure 6P. In Figure 6Q, device 600 receives (e.g., detects) user input 650J (e.g., a downward swipe) on display 602 to minimize the call control menu 630. As shown in Figure 6R, when the call control menu 630 is minimized, the user-represented representation 658 of device 600 includes video from the camera 603 of device 600.

[0191] As shown in Figure 6S, device 600 receives (e.g., detects) user input 650K (e.g., tap) corresponding to the selection of call affordance 634. In response to user input 650K, device 600 disconnects from the live communication session. Figure 6T shows an embodiment of the messaging user interface 604 of the messaging application displayed after device 600 has disconnected from the live communication session. Although device 600 has disconnected from the live communication session, the live communication session remains active. In some embodiments, the live communication session remains active as long as at least one participant remains connected, even if the participant who initiated the live video communication session has left the session. As shown in Figure 6T, device 600 maintains the display of notification 614 and live communication affordance 610 to indicate that the live communication session is active. Notification 614 is updated to indicate the duration of the live communication session (e.g., 3 minutes 42 seconds, corresponding to the time since the live communication session was started).

[0192] While displaying notification 614 indicating that a live communication session is active, device 600 receives data indicating that the live communication session is no longer active (e.g., terminated). In some embodiments, the live communication session terminates if no participants are connected to the live communication session (e.g., no participants who started or joined the live communication session are connected). In response to receiving data indicating that the live communication session is no longer active, device 600 stops displaying notification 614 and / or updates notification 614 to indicate that the live communication session is no longer active. Figure 6U shows an embodiment of the messaging user interface 604 of a messaging application that is displayed after a live communication session has terminated (e.g., in response to receiving data indicating that the live communication session is no longer active). As shown in Figure 6U, notification 614 is updated to include a display indicating that the live communication session is no longer active (e.g., the text “Call ended”), and the live communication affordance 610 is removed. In some embodiments, the live communication affordance 610 and / or notification 614 are grayed out to indicate that the live communication session is no longer active. In some embodiments, notification 614 becomes unavailable after the live communication session has ended. In some embodiments, notification 614 remains available after the live communication session has ended and can be selected to start a new live communication session with the group. In some embodiments, if device 600 did not participate in the live communication session, device 600 updates notification 614 to indicate that the user missed the live communication session (e.g., "missed call").

[0193] As shown in Figure 6U, device 600 receives (e.g., tap) user input 650L (e.g., detect) corresponding to the selection of the return affordance 611. As shown in Figure 6V, in response to receiving user input 650L, device 600 displays the messaging application user interface 671, which includes a list 673 of affordances 673A-673G for accessing each message conversation. Affordance 673D corresponds to a message conversation between a group of three participants (Matthew, Danielle, and the user of device 600). Affordance 673D includes a live communication indicator 675, which indicates that there is an active live communication session for this group. In contrast, since there is no active live communication session for The Dream Team, affordance 673A corresponding to The Dream Team does not include the live communication indicator 675.

[0194] Looking at Figure 6W, device 600 is shown as displaying an interface other than the messaging user interface 604. In Figure 6W, device 600 displays a lock screen 668 indicating that device 600 is locked. The lock screen 668 includes notifications 677A and 677B. Notification 677A indicates that a message from user John Appleseed was received 5 minutes ago. Notification 677B represents a live communication session that the user was unable to join (e.g., an absent video call). The absent video call notification includes the invited participants (John, Amy, Rodrigo), the status of the live communication session ("Absent"), and the time the live communication session started or ended ("1 hour ago"). In the locked state, device 600 does not display any active applications.

[0195] While locked, device 600 receives data indicating that there is a live communication session it can join. In some embodiments, the device 600's response to receiving data indicating that there is a live communication session it can join depends on the context of device 600, the available live communication sessions, or other factors (one or more). In some embodiments, upon receiving that a live communication session is available between two participants (e.g., a one-to-one communication session), device 600 displays a first type of notification. As shown in Figure 6X, the first type of notification includes a full-screen notification 672 and an audio output 674. In some embodiments, the audio output 674 is different from the audio output associated with call notifications and the audio output associated with message notifications. In some embodiments, the audio output 674 shares audio characteristics (e.g., common sound quality, timbre, rhythm) with both the audio outputs for call and message notifications. Alternatively, upon receiving that a live communication session is available between three or more participants (e.g., a group live communication session), device 600 displays a second type of notification. As shown in Figure 6Y, the second type of notification includes a banner notification 676 (e.g., not full screen) and a haptic output 678 (e.g., no audio output). In some embodiments, upon receiving notification that a live communication session is available among three or more participants (e.g., a group live communication session), the device 600 first displays a full screen notification (e.g., similar to the full screen notification 616 shown in Figure 6E) and then displays a banner notification 676 (e.g., transitions to a banner notification). In some embodiments, the notification 676 includes an affordance that, when selected, initiates the process of joining the corresponding live communication session (e.g., as described above with reference to Figures 6F-6L).

[0196] In contrast to notification 677B, which corresponds to an absent video call, notification 676 corresponds to an active live communication session. Notification 676 includes an indication of the group (The Dream Team), the participant (Pablo) who performed the action that triggered the notification, and the time the live communication session started ("now"). In some embodiments, notification 676 is updated to accurately reflect the time the live communication session started and the state of the live communication session. In some embodiments, when the live communication session represented by notification 676 ends, the text of notification 676 is updated from "Join video call from PABLO" to "Absent video call from PABLO".

[0197] In some embodiments, if it is determined that no active application is displayed (as in Figure 6W), device 600 displays a full-screen notification upon receiving data indicating that there is a live communication session that can be joined. In some embodiments, if it is determined that no active application (e.g., a messaging application) is displayed, device 600 displays a non-full-screen notification (e.g., a banner notification).

[0198] In some embodiments, upon determining that an alert should be issued to the user of device 600, device 600 outputs an alert (e.g., a notification or audio or haptic output), and, in accordance with the user's decision not to issue an alert to device 600, device 600 refrains from outputting an alert. In some embodiments, the determination to issue an alert to the user is based on a disturbance criterion (e.g., whether the device is locked or unlocked). In some embodiments, upon determining that device 600 is locked (e.g., as shown in Figure 6Y), the device outputs an alert (e.g., a haptic output 678), and upon determining that device 600 is unlocked (e.g., as shown in Figure 6Z), device 600 displays a notification (e.g., a banner notification 676) and refrains from providing an alert (e.g., no haptic output).

[0199] In some embodiments, the disturbance criterion indicates whether the user is looking at device 600 (for example, based on data from a sensor on the front of device 600). Figure 6AA shows a user 684 holding device 600. The user's line of sight 680 is not directed at device 600. Device 600 determines whether user 684 is looking at device 600, and upon determination that user 684 is not looking at device 600, device 600 provides a non-visual notification 682 (e.g., haptic or audio output). As shown in Figure 6AB, the user's line of sight 680 is directed at device 600. Upon determination that user 684 is looking at device 600, device 600 refrains from providing a non-visual notification (e.g., device 600 displays a banner notification without haptic or audio output). In some embodiments, device 600 determines whether the user is looking at device 600 using an image sensor configured to identify the user's face.

[0200] Looking at Figures 6AC to 6AF, device 600A includes a display 602A (e.g., a touch-sensitive display), a rotatable input mechanism 601A, and a mechanical button 601B. In some embodiments, device 600A includes features of devices 100, 300, 500, and 600. In some embodiments, device 600A communicates with device 600 (e.g., by bidirectional wireless communication). In some embodiments, device 600A is in a pairing relationship with device 600.

[0201] In some embodiments, device 600A receives data (e.g., from device 600) indicating that a live communication session is active and available to three or more participants. As shown in Figure 6AC, in response to receiving data indicating that a live communication session is active, device 600A displays a user interface 604A, which includes a notification 605, a live communication participation affordance 607, and a rejection affordance 609. In some embodiments, notification 605 includes the features of the aforementioned notification 614 (e.g., the names of the participants, the number of participants, and the names of the participants who triggered the notification). In some embodiments, device 600A generates audio and / or haptic output in response to receiving data indicating that a live communication session is active. In some embodiments, device 600A generates audio and / or haptic output different from the audio and / or haptic output generated by device 600 in response to receiving data indicating that a live communication session is active. In some embodiments, upon receiving initial data indicating that a live communication session is active, a notification (e.g., display, audio, and / or haptic output) indicating that a live communication session is active is provided on device 600 (e.g., not on device 600A), and upon receiving reminder data indicating that a live communication session is active, a notification is provided on both device 600 and device 600A.

[0202] In some embodiments, device 600A receives user input (e.g., a tap on live communication affordance 617) corresponding to the selection of live communication participation affordance 607. In response to the selection of live communication participation affordance 607, device 600A causes device 600 and / or device 600A to participate in the live communication session or to start the process of participating in the live communication session. In some embodiments, in response to the selection of live communication affordance 607, device 600A causes device 600 to display a live communication user interface 628 (Figure 6G), 654 (Figure 6K), or 656 (Figure 6L).

[0203] Figure 6AD shows the user interface 604A displayed on device 600A after joining a live communication session. In some embodiments, device 600A includes a microphone for the user to provide audio input to the live communication session via device 600A. In Figure 6AD, the user interface 604A includes a volume controller 611, a display of participants in the live communication session 613 ("PABLO and 2 others") and duration ("9 minutes 45 seconds"), a call end affordance 615, an audio on / off affordance 617, and a menu affordance 619.

[0204] In Figure 6AE, device 600A receives (e.g., detects) user input 650M (e.g., an upward swipe gesture beginning at or near menu affordance 619). As shown in Figure 6AF, in response to receiving user input 650M, device 600A modifies display 613 ("Party of 4") and displays keypad affordance 621, audio source affordance 623 (e.g., for joining a live communication session using device 600), and list 625 of participants in the live communication session (e.g., participants who have joined and / or been invited). In some embodiments, device 604A displays list 625 in response to receiving user input 650M on user interface 604 in Figure 6AC (e.g., prior to joining a live communication session). In some embodiments, list 625 includes a display of the participants' communication status. In some embodiments, list 625 includes a reminder affordance that can be selected to send a reminder notification to participants who are not participating in the live communication session.

[0205] Figures 7A to 7F are flowcharts illustrating a method according to some embodiments. Method 700 is performed on a device equipped with a display (e.g., 100, 300, 500, or 600). Some operations of Method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0206] As will be discussed later, Method 700 provides, among other things, an intuitive way to provide interactive notifications (e.g., 614, 672, 676) for live communication sessions between multiple participants. This method creates a more efficient human-machine interface by reducing the cognitive burden on users using the innovative technology that provides notifications for live communication sessions between multiple participants. For battery-powered computing devices, the ability for users to quickly interact with notifications for live communication sessions between multiple participants is faster and more efficient, thus saving power and extending the battery charging interval.

[0207] In block 702, the device displays the messaging user interface (e.g., 604) of a messaging application. The messaging user interface (e.g., 604) includes a message area (e.g., 606). The message area contains multiple messages (e.g., 608A-608D) in a messaging conversation between three or more participants.

[0208] In block 704, the device receives first data indicating that a live communication session (e.g., an audio communication session or an audio / video communication session) is active and available to three or more participants. In some embodiments, participants in a live communication session may be in a state of "currently joined" (e.g., providing a live media stream to other participants, also called "connected" or "active") or "currently not joined" (e.g., a participant who has never joined the session, or a participant who previously joined the session but has left, also called "not connected" or "invited"). In some embodiments, participants in a live communication session include participants who are currently joined to the live communication session and participants who are not currently joined to the live communication session (e.g., 648A-648D). When a live communication session is active, participants can join (connect to) or leave (disconnect from) the live communication session. When a participant associated with another device joins the live communication session, the device receives a media stream corresponding to that participant. When a user of a device joins a live communication session, the device transmits a live media stream to other participants (e.g., participants joining the live communication session). In some embodiments, the live communication session remains active as long as at least one participant is connected (for example, even if the participant who initiated the live communication session leaves the session).

[0209] In block 706, upon receiving first data indicating that a live communication session between three or more participants is active, the device displays a notification (e.g., 614) indicating that the live communication session is active. In some embodiments, the notification indicating that the live communication session is active includes a live communication participation affordance (e.g., 614A). In some embodiments, the notification includes a display of the number of participants in the live communication session (e.g., the number of active participants in the session, the number of invited participants). In some embodiments, the notification includes a display (e.g., name, initials, photo, or avatar) of the participant who performed an action on the display that triggered the presentation of the notification (e.g., the participant who started the live communication session, or the participant who selected a “recall” button, etc., to remind the device user to join the live communication session). In some embodiments, upon receiving first data indicating that a live communication session between three or more participants is active, the device generates an audio output (e.g., 618) indicating that the live communication session is active by one or more of the device’s audio output devices, and the audio output indicating that the live communication session is active is different from the audio outputs associated with incoming notifications and the audio outputs associated with message notifications. In some embodiments, an audio output indicating that a live communication session is active shares audio characteristics (e.g., common sound quality, timbre, rhythm) with both incoming and message notification audio outputs. By displaying a notification (e.g., 614) indicating that a live communication session is active and / or generating an audio output (e.g., 618), the user is provided with feedback on the current state of the live communication session, and visual feedback is provided to the user indicating that if the user activates the notification (e.g., when the notification includes a join affordance), specific actions related to the notification and / or the live communication session will be taken.By providing users with improved visual feedback, device usability is enhanced, the user-device interface becomes more efficient (for example, by helping users provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling users to use the device more quickly and efficiently.

[0210] Optionally, in block 704, the device displays a notification indicating that a live communication session is active at a first location in the message area (e.g., 606). In some embodiments, the notification indicating that a live communication session is active (e.g., 614) includes a live communication participation affordance (e.g., 614A).

[0211] In block 710, while displaying a notification (e.g., 614) indicating that a live communication session is active, the device receives second data indicating that a live communication session between three or more participants has become inactive.

[0212] Optionally, in block 712, the device moves a notification indicating that a live communication session is active from a first location (e.g., the bottom of message area 606) to a second location (e.g., the center or top of message area 606, or the top, bottom, right, and / or left of display (602)). In some embodiments, the movement of the notification (e.g., 614) occurs in response to the fulfillment of notification relocation criteria. In some embodiments, the notification relocation criteria include one or more criteria such as the elapsed time (e.g., elapsed since the notification was displayed), the display of the messaging user interface (e.g., 604) being discontinued, the messaging application being closed, or the notification (e.g., 614) being obscured in whole or in part within the message area (e.g., 606) by a new incoming message occupying the message area (e.g., 606). By moving the notification indicating that a live communication session is active, the user is provided with feedback on the current state of the live communication session, and, if selected, is provided with visual feedback indicating that a specific action related to the notification and / or the live communication session will be performed, reducing the number of inputs required to perform the operation. By providing the user with improved visual feedback, the usability of the device is enhanced, the user-device interface is made more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power consumption is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently. In addition, the usability of the device is further enhanced by reducing the number of inputs required to perform the operation, the user-device interface is made more efficient (e.g., by providing the user with more accessible inputs and reducing the number of inputs), and in addition, the device's power consumption is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0213] Optionally, in blocks 714 and 716, the device receives a first user input (e.g., 650C) corresponding to a selection of a live communication participation affordance (e.g., 614A or 626B). In response to receiving the first user input corresponding to a selection of a live communication participation affordance, the device participates in a live communication session (e.g., participates in a live communication session and transmits both video and audio from the electronic device, and / or participates in a live communication session and transmits only audio (without transmitting video) from the electronic device). By participating in live communication in response to receiving a selection of a live communication participation affordance contained in the notification, the number of inputs required to perform the operation of participating in a live communication session is reduced. This reduction in the number of inputs required to perform the operation further enhances the usability of the device, makes the user-device interface more efficient (e.g., by providing the user with more accessible inputs and reducing the number of inputs required to participate in a live communication session), and, in addition, reduces the device's power consumption and improves battery life by allowing the user to use the device more quickly and efficiently.

[0214] Optionally, in blocks 718 and 720, the device receives reminder data (e.g., Figure 6D) indicating a reminder to join a live communication session (e.g., a reminder sent by a participant during a conversation or after a predetermined time). In response to receiving the reminder data indicating a reminder to join a live communication session (e.g., the reminder is sent by a participant during a conversation), the device generates an audio output (e.g., 618) (e.g., a ringtone output) indicating that the live communication session is active, using one or more of the device's audio output devices. In some embodiments, the audio output (e.g., 618) is generated in conjunction with the redisplay of a notification (e.g., 616). In some embodiments, the display of the notification is in progress when the reminder data is received. In some embodiments, the audio output indicating that the live communication session is active is different from the audio output associated with incoming call notifications and the audio output associated with message notifications. In some embodiments, the audio output indicating that the live communication session is active shares audio characteristics (e.g., common sound quality, timbre, rhythm) with both the incoming call and message notification audio outputs. In some embodiments, the initial audio output indicating that a live communication session is active differs from (e.g., is shorter, quieter, or weaker) the reminder audio output indicating that a live communication session is active (e.g., an audio output generated based on the participant selecting the “call” affordance). In some embodiments, the initial audio output indicating that a live communication session is active is a single repetition of a repeating audio output used for incoming calls, and the reminder audio output indicating that a live communication session is active is a multiple repetition of a repeating audio output used for incoming calls, or is the same as the audio output used for incoming calls. In some embodiments, upon receiving reminder data indicating a reminder to join a live communication, the device displays a representation of the participant in the live communication session that initiated the reminder (e.g., name, initials, photo, or avatar).By generating a reminder that includes audio output when certain conditions are met, users can quickly recognize that a live communication session is available if they miss a visual notification (e.g., 614). Optimized operation without requiring further user input when a set of conditions is met enhances device usability and makes the user-device interface more efficient (e.g., by assisting users in providing appropriate input when operating / interacting with the device, thereby reducing user errors). Additionally, power consumption is reduced and battery life is improved by enabling users to use the device more quickly and efficiently.

[0215] Optionally, in embodiments where the notification includes a menu access affordance (e.g., 626B), the device receives a second user input (e.g., 650C) corresponding to the selection of the menu access affordance. In response to receiving the second user input corresponding to the selection of the menu access affordance, the device displays a menu (e.g., 630) containing video participation options (e.g., 634) (e.g., affordance, dropdown menu, checkbox) and audio-only options (e.g., 642) (e.g., affordance, dropdown, checkbox). By displaying a menu containing different options for joining a live communication session in response to the selection of the menu access affordance, additional control options are provided without cluttering the UI with additional controllers displayed. By providing additional control options without cluttering the UI with additional controllers displayed, the usability of the device is enhanced, the user-device interface becomes more efficient (for example, by helping the user to provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power usage is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0216] Optionally, in blocks 726 and 728, the device receives a third user input while displaying a menu (e.g., 630). Upon receiving a third user input (e.g., 650F) corresponding to the selection of a video participation option (e.g., 634), the device participates in a live communication session using audio and video transmitted from the electronic device. In some embodiments, the device prompts the user to confirm whether they wish to participate audio-only and / or video-only (e.g., "Do you want to participate with video?"). By participating in a live communication session using audio and video transmitted from the electronic device when certain conditions are met (e.g., in response to the selection of the video participation option), the user can participate in a live communication session using audio and video without the user having to select or modify additional inputs and outputs of the device (e.g., turn on the camera and / or microphone). By performing optimized actions without requiring further user input when a set of conditions is met, the device's usability is enhanced, the user-device interface becomes more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0217] Optionally, in block 730, upon receiving a selection of an audio-only participation option (e.g., 642) as a third user input, the device participates in the live communication session using audio transmitted from the electronic device (e.g., audio recorded by one or more microphones of the device) without transmitting video data from the electronic device. In some embodiments, the device displays an indicator (e.g., 658 in Figure 6L) indicating that the user has participated in the live communication using audio only. This provides the user with the assurance that video is not being shared. By participating in the live communication session using audio transmitted from the electronic device when certain conditions are met, the user can participate in the live communication session using audio only without the user having to select or modify additional inputs and outputs of the device (e.g., turning off the camera and / or turning on the microphones). By performing optimized actions without requiring further user input when a set of conditions is met, the device's usability is enhanced, the user-device interface becomes more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0218] In block 732, upon receiving second data indicating that a live communication session between three or more participants has become inactive, the device either stops displaying a notification (e.g., 614) or updates the notification to indicate that the live communication session has become inactive. In some embodiments, while the live communication session is active, the notification is updated to indicate the current duration of the live communication session. In some embodiments, when the live communication session becomes inactive, the notification stops displaying the session duration and displays a notification that the session has ended (e.g., "Call Ended"). By stopping the display of the notification upon receiving data indicating that the live communication session has become inactive, the user is provided with visual feedback indicating that the notification and / or a particular action related to the live communication session is no longer available. By providing the user with improved visual feedback, the usability of the device is enhanced, the user-device interface is made more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power usage is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently. In addition, by ceasing to display notifications upon receiving data indicating that a live communication session has become inactive, the system automatically performs actions without further user input when a set of conditions is met. By performing optimized actions without requiring further user input when a set of conditions is met, the device's usability is enhanced, the user-device interface is made more efficient (for example, by reducing unnecessary elements on the user interface, assisting the user in providing appropriate input when operating / interacting with the device, and reducing user errors), and the device's power usage is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0219] Optionally, in blocks 736, 738, and 740, the device receives third data indicating that a second live communication session is available among two or more participants. Upon receiving that a second live communication session is available among two participants, the device displays a first type of notification (e.g., 672) (e.g., a full-screen notification) indicating that the second live communication is active. In some embodiments, the first type of notification (e.g., 672) includes a menu (e.g., 630) with options for video (e.g., 634), audio only (e.g., 643), and / or decline (e.g., 622). Upon receiving that a second live communication session is available among three or more participants, the device displays a second type of notification (e.g., 676) that is different from the first type of notification, indicating that the second live communication is active. In some embodiments, the second type of notification is a banner notification (e.g., 676) that is not full-screen or is not persistent full-screen. Optionally, in block 742, the device initially displays a second type of notification in the first state (e.g., full-screen state 672). Optionally, in step 744, the device transitions the second type of notification (e.g., 672) to a second state (e.g., 676) (e.g., a persistent non-full-screen state (e.g., a persistent banner positioned at the top edge of the display)). By displaying a specific type of notification indicating that a live communication session is active based on predetermined conditions, the device enables the user to quickly recognize that a live communication session is available, while providing more and / or less distraction reduction based on predetermined conditions (e.g., displaying a full-screen notification when there is a live communication session available between two participants, and displaying a banner when there is a live communication session available between two participants).By performing optimized actions without requiring further user input when a set of conditions is met, the device's usability is enhanced, the user-device interface becomes more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0220] Optionally, in blocks 746, 748, 750, and 752, the device receives fourth data indicating that a third live communication session is available. In response to receiving the fourth data indicating that a third live communication session is available, and determining that an active application (e.g., a messaging application, or a third-party application provided by an application developer other than the operating system developer, such as an application that replaces the device's lock screen (e.g., 668) or home screen (e.g., 1400)) is displayed, the device displays a notification of a third type (e.g., a non-fullscreen state (e.g., a banner placed on the top edge of the display)) (e.g., 614 or 676) indicating that a third live communication session is available. In response to receiving the fourth data indicating that a third live communication session is available, and determining that no active application is displayed (e.g., Figure 6W), the device provides a notification of a fourth type (e.g., 678 or a fullscreen notification (e.g., 616)) indicating that a third live communication session is available. By displaying specific types of notifications for live communication sessions based on when certain conditions are met, it is possible to reduce the amount of scrutiny required from the user based on those conditions (e.g., displaying a full-screen notification when the device is viewing an application and a banner when the device is not viewing an application), while allowing the user to quickly recognize that a live communication session is available. By performing optimized actions without requiring further user input when a set of conditions are met, the usability of the device is improved, the user-device interface is made more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power usage is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0221] Optionally, in blocks 754, 756, 758, and 760, the device receives fifth data indicating that a fourth live communication session is available. In response to receiving the fifth data indicating that a fourth live communication session is available, and determining that the current status of the electronic device (e.g., the current operating mode of the device (e.g., whether the device is in do-not-disturb (DND) mode, whether a particular application or hardware is operationally active, whether the device is locked or unlocked, or whether one or more sensors of the device detect that a user is looking at the display)) meets the alert notification criteria, the device displays a notification (e.g., 676) indicating that a fourth live communication session is available while providing a non-visual notification (e.g., 678) (e.g., audio or haptic output). In some embodiments, the output or alert corresponds to the notification (e.g., 614). In response to receiving the fifth data indicating that a fourth live communication session is available, and determining that the current status of the electronic device does not meet the alert notification criteria, Upon determination, the device displays a notification (e.g., 614) indicating that a fourth live communication session is available, without providing a non-visual notification (e.g., 678) indicating that a fourth live communication session is available. In some embodiments, a second type of alert (e.g., a visual alert such as a banner notification (e.g., 676)) is announced, while a first type of alert is not announced. In some embodiments, the relevant status of the device is whether the device is locked or unlocked. In some embodiments, if the device is locked, a first type of alert (e.g., an audio or haptic output) is announced, and if the device is unlocked (e.g., the user is actively operating / interacting with the device), no alert is announced (e.g., suppressed). In some embodiments, the relevant status of the device is whether the device detects (e.g., by one or more sensors (e.g., a camera, an infrared sensor)) that the user is looking at the display.In some embodiments, a first type of alert (e.g., audio or haptic output) is broadcast when the device is locked. In some embodiments, a first type of alert (e.g., 682) is generated when the device detects that the user is not looking at the display (e.g., Figure 6AA). In some embodiments, if the user is actively operating / interacting with the device (e.g., Figure 6AB), the first type of alert is not generated (e.g., suppressed).

[0222] In some embodiments, the alert notification criteria include the requirement that the device is locked for the alert notification criteria to be met. In some embodiments, the alert notification criteria include the requirement that the device does not detect user attention to the device for the alert notification criteria to be met (e.g., based on user input detected by an input device of an electronic device, or based on user gaze (e.g., 680) detected by eye tracking, face tracking, etc.). By determining whether to display a notification (e.g., 614) and provide non-visual notifications based on when a set of conditions are met, it is possible to quickly let the user know that live communication is available while resulting in more and / or fewer interruptions based on predetermined conditions. By performing optimized operation without requiring further user input when a set of conditions is met, the usability of the device is improved, the user-device interface is made more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the power usage of the device is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0223] It should be noted that the details of the processing described above regarding Method 700 (for example, Figures 7A to 7F) can also be applied in a similar manner to the methods described later. For example, Method 700 may optionally include one or more characteristics of the various methods described later regarding Methods 900, 1100, 1300, and 1500. For example, Method 700 may use Methods 1300 and 1500 to initiate a live communication session, and the initiation and execution of the live communication session may include Methods 900 and 1100. For brevity, these details will not be repeated below.

[0224] Figures 8A to 8BK show exemplary user interfaces for live communication sessions according to some embodiments. These user interfaces are used to illustrate the processes described later, including the processes shown in Figures 9A to 9K.

[0225] Figure 8A shows device 600 displaying a messaging user interface 604 for message conversations between the group The Dream Team. A live communication session between group participants is not active (as indicated by the absence of, for example, the live communication affordance 610). As shown in Figure 8A, the header of the messaging user interface 604 is extended to indicate the video join affordance 626B (as previously described). As shown in Figure 8A, device 600 receives user input 850A (e.g., a tap on the video join affordance 626B) in response to a request to display the user interface for a live communication session between participants of the participant group The Dream Team. As shown in Figure 8A, the request includes a request to start a new live video communication session. In some embodiments, the request includes a request to join an existing live communication session (e.g., as previously described with respect to Figure 6F).

[0226] In response to user input 650A, device 600 displays a message 800 in message area 606 and an indicator 802 indicating that a live communication session has been started. In some embodiments, message 800 includes some of the aforementioned characteristics of notification 614. Once a live communication session is started, device 600 displays a live communication user interface 804, which simultaneously includes representations of multiple participants in the live communication session.

[0227] If it is determined that the number of participants in a live communication session exceeds the participant threshold (for example, more than 5 participants, including the user of device 600), device 600 will simultaneously display representations of a first group of participants other than the user of device 600 in one or more sizes larger than their respective threshold sizes (for example, three different sizes), while displaying representations of a second group of participants, including one or more participants not included in the first group of participants, in an overflow area. The representations of one or more other participants displayed in the overflow area will be displayed in sizes smaller than their respective threshold sizes.

[0228] In the embodiments shown in Figures 8A to 8BK, the threshold number of participants is 5 (including the user of device 600). Since the group The Dream Team includes 15 participants, device 600 determines that the number of participants in the live communication session is greater than the threshold number of participants. As shown in Figure 8C, in response to this determination, the live communication user interface 804 includes a 4:1 layout of four primary representations 810A to 810D (collectively 810) of each of the four participants other than the user of device 600, displayed in the canvas area 811, as well as an overflow area 820 with a user representation 806 of the camera view of device 600 (corresponding to the user of device 600) and overflow representations 822A to 822J (collectively 822) corresponding to participants in the live communication session other than the user of device 600 and the participants represented in the canvas area 811, with subsets of these (e.g., 822A to 822D) always displayed. In some embodiments, the layout is one of several predetermined layouts in which participant representations are fixed at different locations on the user interface 804. Figure 8AW shows another embodiment of the user interface 804. In the embodiment shown in Figure 8AW, the overflow area 820 includes overflow representations of all participants other than the user of device 600 in the live communication session, and participants represented in the canvas area 811 are also represented in the overflow area 820.

[0229] If the number of group participants is five or less, the device 600 determines that the number of participants in the live communication session is five or less, and displays the main representation of the participants and the representation of the user of device 600 without displaying an overflow representation in the overflow area.

[0230] Each primary expression 810 is displayed in one of three predetermined display sizes (small, medium, and large). Primary expression 810A is displayed in small size, primary expression 810B is displayed in large size, and primary expressions 810C and 810D are displayed in medium size. In Figure 8C, each primary expression 810 overlaps with at least one other primary expression. All overflow expressions 822 are displayed in the same size as the primary expression 810 but smaller than the small display size associated with the primary expression 810. In the illustrated embodiment, the respective threshold sizes described above are larger than the size of the overflow expression 822 and smaller than the small display size of the primary expression 810. The user expression 806 is the same size as the overflow expression 822 and is displayed alongside or within the overflow area 820.

[0231] As shown in Figure 8C, the live communication session user interface 804 displays three full overflow representations and one partial overflow representation at a time. The remaining overflow representations are hidden and can be displayed by scrolling the overflow area 820 (in response to any detection event, such as user input or active participation by a participant speaking or moving). In Figure 8C, the overflow representation 820D is partially shaded to indicate that more overflow representations exist and / or that the overflow area 820 can be scrolled. In some embodiments, the order of the overflow representations 822 within the overflow area 820 depends on the order in which participants join the live communication session. In some embodiments, the order of the overflow representations 822 within the overflow area 820 changes over time (for example, as participants join or leave the live communication session, or as participants enter and exit the canvas area 811, for example, by user selection of an overflow representation or active participation by a participant represented in the overflow area, as will be described in more detail later). In some embodiments, the order of overflow representations within the overflow region 820 is fixed (for example, the order in which overflow representations are displayed does not change over time). In some embodiments, the order of overflow representations is based on the order of a list associated with a group (for example, the order in which participants were added to a group when the group was created). In some embodiments, the display of the primary representation 810 (for example, a live video feed) is updated at a higher rate than the display of the overflow representation 822.

[0232] Figure 8C shows the live communication session interface 804 immediately after the live communication session is started and before any participant (other than the participant associated with device 600) joins the live communication session. As shown in Figure 8C, the primary representation 810 and the overflow representation 822 include placeholders for participants not connected to the live communication session. As shown in Figure 8C, the placeholder includes a representation with a display of the participant's name (e.g., first name, or initials of first and last name) and a colored object. In some embodiments, the color and / or pattern of each colored object is different or selected from a predetermined set of colors and / or patterns. Optionally, the placeholder includes a status indicator indicating that the participant is waiting to connect to the live communication session and a reminder affordance ("outgoing") for sending a reminder to the corresponding participant that the live communication session is available. In some embodiments, the reminder affordance is not initially included in the placeholder and is displayed after a predetermined time has passed since it has been determined that the corresponding participant has not joined the live communication session.

[0233] As shown in Figure 8C, device 600 receives (e.g., tap) user input 850A (e.g., detect) corresponding to the selection of the primary expression 810C. As shown in Figure 8D, in response to receiving user input 850A, device 600 modifies the primary expression 810C to include additional identification information (e.g., participant's name, Marcel) and a reminder affordance 808A for sending a reminder to the corresponding participant that a live communication session is available.

[0234] Figure 8D also shows that a participant corresponding to primary expression 810B also joins the live communication session. In some embodiments, device 600 receives data indicating that a participant corresponding to primary expression 822B has joined the live communication session. As shown in Figure 8D, in response to a participant corresponding to primary expression 822B joining the live communication session, device 600 replaces the placeholder in primary expression 822B with live video data from the corresponding participant. In some embodiments (e.g., Figure 8AX), device 600 replaces the overflow expression 822B corresponding to the participant with live video data from the participant. As used herein, the phrase “live video data from participant” (or its equivalent) includes live video data transmitted by a device associated with the participant (e.g., a smartphone, laptop computer, desktop computer, or tablet computer) or a device associated with an account associated with the participant (e.g., an email account or cloud storage account).

[0235] As shown in Figure 8D, device 600 receives (e.g., detects) a user input 850B (e.g., a tap) corresponding to the selection of a reminder affordance 808. In response to input 850B, device 600 sends a command to cause a new notification (e.g., a call, a ringtone, etc.) to occur on the device associated with the corresponding participant, indicating that a live communication session is available, where the new notification is different from a previous notification sent to the participant. In some embodiments, the new notification is more interfering than the previous notification (e.g., an audio outgoing, haptic output, or full-screen notification, compared to a banner notification without audio or haptic output). In Figure 8D, device 600 also displays a reminder affordance 808B on the main expression 810D to indicate that the live communication session is available, such as a reminder sent by another participant currently connected to the live communication session (e.g., a participant represented by the main expression 810B), or a reminder previously sent by a user of device 600. Device 600 also modifies the reminder affordance 808A to indicate that the corresponding participant is notified, such as by replacing "ongoing" with "ongoing" as shown in Figure 8E.

[0236] As shown in Figure 8E, when it is determined that a predetermined amount of time has elapsed since the start of the live communication session, device 600 stops displaying the overflow representation 822 and increases the size of the user representation 806. In some embodiments, the main representation 810 is resized and / or moved when the overflow representation 822 is deleted. In the illustrated embodiment, the main representation 810D is moved downward. In some embodiments, when it is determined that the number of participants currently participating in the live communication session is less than or equal to the participant threshold (for example, 5 participants including the user of device 600), device 600 stops displaying the overflow representation 822.

[0237] At the point corresponding to Figure 8E, two participants are in the live communication session. In some embodiments, after a further period of time, device 600 discontinues displaying one of the main representations 810 and changes the layout of the live communication user interface 804 to a 3:1 layout. In some embodiments, the 4:1 and 3:1 layouts are part of a predetermined set of layouts in which the relative arrangement of the main representations 810 is similar. In some embodiments, the arrangements include such similar arrangements of the main representations that when there is a transition between layouts within the set, the main representations remain in similar positions on the user display 602 (for example, the position of a representation in one layout overlaps with the position of a representation in another layout in the same set).

[0238] In some embodiments, device 600 determines which primary representation to delete based on whether the corresponding participant is participating in the live communication session (for example, if the corresponding participant is participating, the representation is not deleted), whether the participant has been reminded (for example, the representation of a reminded participant is not deleted for a predetermined time after the corresponding reminder affordance is selected), the placement of the participant in the participant list (for example, the representation of an earlier participant in the list is deleted before the representation of a later participant in the list), and the placement of representations on the live communication user interface 804 (for example, representations closer to the top of display 602 are deleted before representations lower on display 602). As shown in Figure 8F, device 600 discontinues displaying the primary expression 810A and moves the primary expression 810B upward (for example, to reduce the overlap between expression 810B and expression 810C) while maintaining the relative vertical order of the remaining expressions (for example, expression 810B is larger than expression 810C and is above expression 810C, and expression 810C is above expression 810D and is the same size as expression 810D). Figure 8F also shows that reminder affordance 808A is modified to “Recall,” indicating that the corresponding participant has been reminded at least once and is not currently being reminded. Reminder affordance 808B is also modified (for example, from “Calling” to “Calling,” and in a different color) to indicate that the corresponding participant has been reminded previously, but not by the user of device 600.

[0239] At the point corresponding to Figure 8F, the live communication session still has only two participants. In some embodiments, after some time, device 600 discontinues displaying another primary representation 810 and changes the layout of the live communication user interface 804 to a 2:1 layout (for example, in the same predetermined set of layouts as the 4:1 and 3:1 layouts shown in Figures 8E-8F). As shown in Figure 8G, device 600 discontinues displaying primary representation 810D and expands primary representations 810B and 810C. Figure 8G shows a layout for three participants (e.g., 2:1) where the primary representations of the two participants other than the user of device 600 are aligned vertically and of equal size. The vertical order of primary representations 810B and 810C is maintained with primary representation 810B above primary representation 810C. In the layout of Figure 8F, there is no overlap between the primary representations, and the user representation 806 overlaps primary representation 810C.

[0240] At the point corresponding to Figure 8G, the live communication session still has only two participants. In some embodiments, after a further period of time, device 600 discontinues displaying another primary representation and changes the layout of the live communication user interface 804 to a one-to-one layout. As shown in Figure 8H, device 600 discontinues displaying primary representation 810C and expands primary representation 810B to full screen size. In some embodiments, device 600 changes from the layout of Figure 8G to the layout of Figure 8H after a predetermined time following the selection of reminder affordance 808A. Figure 8H is a two-participant (e.g., one-to-one) layout with the remaining primary representation 810B displayed in full screen and the user representation 806 superimposed. As shown in Figure 8H, the remaining main representations 810B and user representations 806 are vertically elongated rectangles (for example, in the orientation of the profile) compared to the square-shaped main representations 810 and user representations 806 in the 4:1, 3:1, and 2:1 layouts shown in Figures 8E to 8G, respectively.

[0241] Looking at Figure 8I, a third participant, Stephen, has joined the live communication session. As shown in Figure 8I, in response to participant Stephen joining the live communication session, device 600 transitions from the one-to-one layout display in Figure 8H to a two-to-one layout where the main representation 810E of the most recently joined participant is above the representation 810B of an existing participant (other than the user of device 600). In some embodiments, device 600 displays a notification 812 indicating that a participant has joined the live communication session, and / or generates audio or haptic output, as shown in Figure 8I. In some embodiments, a new main representation is displayed when participant Stephen joins the live communication session, so device 600 refrains from displaying notification 812 or generating audio or haptic output, allowing the main representation 810E to serve as the indicator that a participant has joined the live communication session.

[0242] Participant Stephen joins the live communication session in audio-only mode and provides live audio data without video. Participant Stephen's audio-only communication status is indicated by a main expression 810E and a notification 812. The main expression 810E indicates the audio-only communication status by displaying the participant's name and an avatar associated with the participant. The notification 812 indicates the communication status in text ("You have joined this call in audio"). As shown in Figure 8J, device 600 stops displaying the notification 812 (for example, after a predetermined time has passed since participant Stephen joined the live communication session).

[0243] In some embodiments, in response to receiving data indicating that an audio-only participant is actively participating (e.g., speaking), device 600 displays a visual representation of the speaking participant. In Figure 8J, a representative image (e.g., an avatar) within the main representation 810E is animated (e.g., in a "voice circle") to indicate that participant Stephen is speaking.

[0244] Looking at Figure 8K, we see that the fourth participant, Marcel, has joined the live communication session. As shown in Figure 8K, in response to the fourth participant joining the live communication session, device 600 changes from a 2:1 layout display to a 3:1 layout (as shown in Figure 8F, for example), where the main representation 810F of the most recently joined participant is above the main representations 810B and 810E of existing participants (other than the user of device 600).

[0245] Participant Marcel joins the live communication session in video mode and provides live video and audio data. Participant Marcel's video communication status is indicated by the main display 810F. The main display 810F indicates the video communication status by displaying the live video stream. As shown in Figure 8K, device 600 does not immediately display a notification that participant Marcel has joined the live communication session.

[0246] Looking at Figure 8L, we see that the fifth participant, Hoan, has joined the live communication session. As shown in Figure 8L, in response to participant Hoan joining the live communication session, device 600 changes from a 3:1 layout to a 4:1 layout, where the main representation 810G of the most recently joined participant is positioned above and to the right of the smallest main representation, main representation 810F. In some embodiments, upon determination that both participants Marcel and Hoan have joined the live communication session within a predetermined grace period, device 600 displays a combined notification 816 indicating that participants Marcel and Hoan have joined the live communication session, as shown in Figure 8L. In some embodiments, the combined notification 816 is displayed upon determination that the participants have joined in the same communication state (e.g., video).

[0247] As indicated by the live video stream displayed in the main representation 810G and the representation of the video camera in the notification 816A, participant Hoan joins a live communication session in video mode so that device 600 can receive live video and audio data related to participant Hoan.

[0248] As shown in Figure 8L, device 600 receives (e.g., detects) user input 850C (e.g., a double tap) at the position of the primary representation 810F on display 602. In some embodiments, as shown in Figure 8M, in response to receiving user input 850C, device 600 enlarges the primary representation 810F to its maximum representation size and reduces the primary representation 810E to a medium size (while maintaining a 4:1 layout). As shown in Figure 8M, device 600 stops displaying notification 816 (e.g., after a predetermined time has passed since participant Hoan joined the live communication session).

[0249] In some embodiments, the layout configuration is dynamically changed in response to the reception of data indicating that a participant is actively participating in a live communication session. In Figure 8M, device 600 receives data indicating that participant Hoan, corresponding to primary representation 810G, is actively participating. As shown in Figure 8M, in response to the reception of data indicating that participant Hoan, corresponding to primary representation 810G, is actively participating, device 600 enlarges primary representation 810G from a small representation size to a medium representation size and reduces primary representation 810B from a medium representation size to a small representation size. In this way, the 4:1 layout maintains one small representation, two medium representations, and one large representation.

[0250] As shown in Figure 8M, device 600 receives (e.g., detects) user input 850D (e.g., a double tap) at the position of the primary representation 810G on display 602. In some embodiments, as shown in Figure 8N, in response to receiving user input 850D, device 600 enlarges the primary representation 810G, pushing it to the forefront in front of the other primary representations being displayed. The background behind the enlarged primary representation 810G is grayed out, except for user representation 806.

[0251] As shown in Figure 8O, device 600 receives (e.g., detects) a user input 850E (e.g., a tap) on the cancel affordance 818. As shown in Figure 8P, in response to receiving the user input 850E, device 600 reduces the size of the main representation 810G and returns the display to the 4:1 layout that was displayed prior to the user input 850D, except that the main representation 810G occupies the larger main representation instead of the reduced main representation 801F.

[0252] Looking at Figure 8Q, a sixth participant joins the live communication session. As shown in Figure 8Q, in response to the sixth participant joining the live communication session and the number of connected participants exceeding a predetermined threshold of five participants, device 600 displays the sixth participant's overflow representation 822A in the overflow area 820 and moves the user representation 806 into the overflow area 820 so that the overflow representation 822A is to the left of the user representation 806. As shown in Figure 8Q, the overflow representation 822A and the user representation 806 are aligned in the center of the overflow area 820. In some embodiments, device 600 replaces the smallest primary representation (e.g., 810B) with the representation of the newly joined participant (e.g., the sixth participant) and displays the replaced participant's overflow representation in the overflow area 820 (e.g., representation 810B is reduced in size and moved to the position of overflow representation 822A in the overflow area 820). In some embodiments, when a sixth participant joins a live communication session and the number of connected participants exceeds a predetermined threshold of five participants, the device 600 displays an overflow representation of the currently participating participants or all participants (currently participating participants and invited participants) in the overflow area, as shown in Figure 8AY, for example.

[0253] As shown in Figure 8Q, the overflow region 820 and the main representations 810 do not overlap. In some embodiments, to make space for the overflow region, device 600 reduces the size of one or more main representations and / or shifts them vertically. In some embodiments, device 600 reduces the size of the medium-sized main representation while maintaining the sizes of the small and large-sized main representations. In some embodiments, device 600 reduces the vertical gap between main representations and / or increases the vertical overlap.

[0254] Looking at Figure 8R, a seventh participant has joined the live communication session. In response to the seventh participant joining the live communication session, device 600 adds the seventh participant's overflow representation 822B to the left of overflow representation 822A in the overflow region 820. As shown in Figure 8R, overflow representation 822A and user representation 806 are moved to the right (compared to Figure 8Q) so that overflow representations 822A and 822B, as well as user representation 806, are aligned in the center of the overflow region 820. In some embodiments, the seventh participant's overflow representation 822B is added to the right of overflow representation 822A, between overflow representation 822A and user representation 806. In the embodiment shown in Figure 8AZ, in response to the seventh participant joining the live communication session, device 600 replaces the smallest primary representation (e.g., 810H in Figure 8AY) with primary representation 810I and changes the overflow representation 822F corresponding to participant Pablo from a placeholder to a live video stream.

[0255] As shown in Figure 8S, an eighth participant joins the live communication session. In response to the eighth participant joining the live communication session, device 600 adds the eighth participant's overflow representation 822C to the left of overflow representation 822B in the overflow region 820. As shown in Figure 8S, overflow representations 822A and 822B, as well as user representation 806, are moved to the right (compared to Figure 8Q) so that overflow representations 822A, 822B and 822C, as well as user representation 806, are aligned in the center of the overflow region 820.

[0256] Looking at Figure 8T, the ninth participant has joined the live communication session. In response to the ninth participant joining the live communication session, device 600 adds the ninth participant's overflow representation 822D to the left of overflow representation 822C in the overflow region 820. As shown in Figure 8T, overflow representations 822D, 822C, and 822B, as well as user representation 806, are fully displayed. Overflow representation 822A is partially displayed, with user representation 806 overlapping the right portion. Overflow representation 822A is shaded (e.g., by a shading effect) where it meets user representation 806. In the illustrated embodiment, the sizes of overflow representation 822 and user representation 806 remain the same as when the seventh, eighth, and ninth participants joined the live communication session. Figure 8T shows that device 600 displays a combined notification 824 indicating that participants Mary, Mick, John, and Tom (represented by overflow representations 822A, 822B, 822C, and 822D, respectively) have joined the live communication session. In some embodiments, when additional participants (e.g., beyond a ninth participant) join the live communication session, the existing overflow representations shift to the right within the overflow area 820 to make space for the new overflow representations, and the overflow representations of the newly joined participants are added to the left end of the existing overflow representations, while the user representation 806 remains fully visible on the right side of the overflow area 820 (e.g., the existing overflow representations are scrolled to the right, "below" the user representation 806). In some embodiments, the overflow representations of the newly joined participants are added to the right end of the existing overflow representations, while the user representation 806 remains fully visible on the right side of the overflow area 820. In some embodiments, if the overflow area is completely occupied, the representation of a newly joined participant added to the right end of the existing overflow representation is not immediately displayed (for example, the new overflow representation is displayed as the overflow representation is scrolled to the left).

[0257] Looking at Figure 8U, device 600 receives data indicating that the participant corresponding to the overflow representation 822B is actively participating in the live communication session. In some embodiments, device 600 replaces the primary representation with the representation of the actively participating participant when it determines that the activity level of the actively participating participant has risen above the activity level of the participant represented by the primary representation. In the illustrated embodiment, upon receiving data indicating that the participant corresponding to the overflow representation 822B is actively participating in the live communication session, device 600 replaces the smallest primary representation with the representation of the actively participating participant and displays the participant's overflow representation corresponding to the replaced primary representation. As shown in Figure 8U, the primary representation 810B (the smallest primary representation) is shaded or begins to fade, the overflow representation 822B of actively participating participants is shaded or begins to fade out and shrinks in size, and the overflow representations 822C and 822D are shifted to the right as the new overflow representation 822E corresponding to the participants of the primary representation 810B (the primary representation being replaced) enters the overflow region 820 from the left edge of the display 602 to the left of the overflow representation 822D. In some embodiments, the primary representation 810B is slightly enlarged (for example, to emphasize that it is being replaced). As shown in Figures 8V to 8W, the main representation 810B is replaced by the main representation 810H of the actively participating participant, the overflow representation 822B continues to contract until it is deleted, the overflow representation 822E is fully displayed, and the overflow representations 822C and 822D continue to shift to the right until the overflow representation 822C occupies the position of the overflow region 820 previously occupied by the deleted overflow representation 822B. In Figures 8U to 8W, the overflow representation 822A and the user representation 806 remain in the same position.

[0258] In some embodiments, the device 600 visually distinguishes user representations 806 in response to active participation by the user of the device 600. In the embodiment shown in Figure AZ, the device 600 displays a thick border around the user representation 806 and highlights an initial bar below the user representation 806.

[0259] Looking at Figure 8X, the device 600 is rotated from a portrait (vertical) orientation to a landscape (horizontal) orientation. As the device 600 rotates, as shown in Figure 8X, the device 600 maintains the representations within each area of ​​the display 602 that were occupied prior to the rotation of the device 600 (for example, the user representation and each primary and overflow representation occupy the same areas on the display 602 before and after the rotation of the device 600). The images within the representations are rotated so that the participant is upright relative to the landscape orientation. In some embodiments, the device 600 displays an animated rotation of the primary representation or an image within the primary representation (for example, 90 degrees opposite to the direction of rotation of the device 600). In some embodiments where the overflow representation is not square, when the device 600 rotates, the participant's video data is cropped and / or translated so that the user's face is included in each overflow representation (for example, centered). In some embodiments where the primary representation is not square, when device 600 is rotated, the participant's video data is cropped and / or translated to center the user's face on each primary representation. In some embodiments, the participant's video data is cropped to include the user's face (whether or not device 600 is rotated, for example, when the received video data has a different aspect ratio or format than the representation being displayed).

[0260] As shown in Figure 8X, participants corresponding to the primary representation 810H begin to actively participate. In response, device 600 enlarges the primary representation 810H and reduces the size of the primary representation 810E, as shown in Figure 8Y. In the embodiments shown in Figures 8BA to 8BB, device 600 visually distinguishes the overflow representation 822F corresponding to active participants (e.g., a highlighted initial bar).

[0261] In some embodiments, when a participant joins a live communication session, device 600 replaces an existing primary representation with the primary representation of the newly joined participant and moves the replaced participant to the overflow area 820. Looking at Figure 8Z, participant Pablo has joined the live communication session. Accordingly, device 600 replaces the smallest primary representation 810E (Stephen) with the primary representation 810I of the newly joined participant Pablo and displays the overflow representation 822F (Stephen) in the overflow area 820. To make room for the overflow representation 822F, the overflow representations 822A, 822C, 822D, and 822E are scrolled (for example, upward depending on the orientation of device 600 as shown in Figure 8Z). As shown in Figure 8Z, the overflow representation 822A is no longer visible, and the overflow representation 822C is partially hidden by the user representation 806.

[0262] Looking at Figure 8AA, device 600 is rotated from a landscape (horizontal) orientation to a portrait (vertical) orientation. Accordingly, device 600 rotates the images of the participants in the representation while maintaining the area on the display occupied by each representation (for example, device 600 reverses the rotation that occurred when the device was rotated from a portrait orientation to a landscape orientation).

[0263] In Figure 8AB, the participant corresponding to the primary representation 810G has left (disconnected from) the live communication session. Optionally, as shown in the embodiment in Figure 8BC, the device 600 displays a notification 828 indicating that the participant has left the live communication session. Since the number of participants remains greater than five, the device 600 maintains a 4:1 layout with the display of the overflow area 820. As shown in Figure 8AC, the primary representation 810G is replaced by the primary representation 810J of participant Stephen, who has the next highest activity level, and the overflow representation 822F corresponding to participant Stephen is removed from the overflow area 820. Once the overflow representation 822F is removed, the remaining overflow representations 822A, 822C, 822D, and 822E are scrolled to the left. In the embodiment shown in Figure 8BD, the overflow representation 822D of participant Hoan is updated to display a placeholder indicating that participant Hoan is not connected, and a reminder affordance to request participant Hoan to rejoin.

[0264] In some embodiments, when a participant leaves a live communication session, if the number of participants in the live communication decreases from six to five, the device 600 stops displaying the overflow area 820 and optionally increases the size and / or vertical spacing of one or more of the main representations.

[0265] In Figure 8AC, device 600 receives (e.g., detects) user input 850F (e.g., double tap) corresponding to the selection of participant Allison's overflow representation 822E. As shown in Figure 8AD, in response to receiving user input 850F, device 600 displays participant Allison's enlarged representation 830 in front of the currently displayed 4:1 layout of the primary representation 810. The display of the enlarged representation 830 is similar to the display of the enlarged representation 810G in Figure 8N, except that the enlarged representation 830 is not pushed in front of the primary representation in the 4:1 layout.

[0266] As shown in Figure 8AD, device 600 receives (e.g., a tap) user input 850G (e.g., a detection) corresponding to the selection of the close affordance 818. As shown in Figures 8AE to 8AF, in response to receiving user input 850G, device 600 displays participant Allison's primary representation 810K instead of participant Stephen's primary representation 810J (e.g., the participant with the lowest activity level among the participants with the current primary representation). Primary representation 810K (at least initially) includes a ribbon (or band) 842A along the bottom edge, which includes the participant's name and affordances for enlarging the representation (e.g., as shown in Figure 8AD). ​​Figures 8AE to 8AF show an exemplary animation of replacing primary representation 810J with primary representation 810K. Primary representation 810K is initially displayed at a small representation size (similar to primary representation 810J in Figure 8AC) and then enlarged to a medium representation size, while primary representation 810H is reduced from medium to small. The overflow representation 822E corresponding to participant Allison is reduced in size and / or fades out, while the overflow representation 822F of the replaced participant is translated into the overflow region 820. In some embodiments, discontinuing the display of the primary representation 810J and displaying the primary representation 810K involves a crossfade from primary representation 810J to primary representation 810K and / or a color scheme that produces a "flash" effect. Figures 8BE to 8BH show embodiments in which the primary representation 810J is removed with a flash effect.

[0267] In some embodiments, the overflow region 820 is scrollable. Referring to embodiments shown in Figures 8BH to BI, device 600 scrolls the overflow representation 822 within the overflow region 820 in response to active participation by a participant whose corresponding overflow representation is not currently displayed or is not fully displayed. As shown in Figure 8BH, the participant's overflow representation 822F (corresponding to the main representation 810I) is partially hidden within the overflow region 820. In Figure 8BH, the participant corresponding to the main representation 810I actively participates in the live communication session. As shown in Figure 8BI, in response to receiving data indicating that the participant corresponding to the main representation 810I is actively participating, device 600 enlarges the main representation 810I and scrolls the overflow representation 822 so that the overflow representation 822F is fully displayed (for example, near the center of the display portion of the overflow region 820), and visually indicates the overflow representation 822F by highlighting the initial bar at the bottom of the overflow representation 822F.

[0268] Looking at Figure 8AG, device 600 detects contact 850H, which moves horizontally from right to left on the overflow region 820 while remaining in contact with the display 602. As shown in Figure 8AH, in response to receiving contact 850H, device 600 scrolls the overflow representation 822 to the left on the display 602 so that the overflow representation 822F is partially removed from the left edge of the display 602 and the user representation 806 no longer overlaps with the overflow representation 822A. ISE, contact 850H and the overflow representation 822 move horizontally by the same amount.

[0269] While scrolling through the overflow area 820, the primary representation 810 remains unchanged. The user representation 806 also remains within the overflow area 820.

[0270] In the embodiments shown in Figures 8BJ to 8BK, contact 850H begins on the left side of the overflow region 820 and moves horizontally to the right. Accordingly, device 600 scrolls the overflow representation 822 to the right on display 602. In Figures 8BJ to 8BK, when the representation reaches user representation 806, device 600 removes that representation from display 602 (e.g., stops displaying it).

[0271] Turning to Figure 8AH, device 600 receives (e.g., detects) user input 850I (e.g., a tap) at a location on display 602 where the primary representation 810 is not displayed. As shown in Figure 8AI, in response to receiving user input 850I, device 600 displays a call control menu 832 which includes a call affordance 834 (for example, to exit a live communication session), an effect affordance 836, a menu affordance 838, and a handle affordance 840.

[0272] As shown in Figure 8AI, the call control menu 832 does not overlap with the overflow area 820 and the main representation 810. To make space for the call control menu 832, device 600 reduces the size of one or more of the main representations 810 and / or shifts them vertically (for example, reducing the size of the canvas area 811) and moves the overflow area 820 upward on the display 602. In some embodiments, device 600 reduces the size of the medium-sized main representation while maintaining the sizes of the small and large main representations. In some embodiments, device 600 reduces the vertical gap between the main representations and / or increases the vertical overlap.

[0273] In response to user input 850I, device 600 also displays ribbons 842A-842D for each of the main representations. Ribbons 842A-842D include the names of the corresponding participants and affordances for expanding the representations (for example, as shown in Figure 8O). In some embodiments, the vertical height of ribbons 842A-842D is the same as the vertical overlap between the main representations.

[0274] As shown in Figure 8AJ, device 600 receives (e.g., detects) user input 850J (e.g., a tap) corresponding to the selection of menu affordance 838. In some embodiments, user input 850J includes an upward swipe starting from call control menu 832 (e.g., on handle affordance 840). As shown in Figure 8AK, in response to user input 850J, device 600 extends call control menu 832 to display additional information and controllers regarding the live communication session, similar to the additional information and controllers described with respect to call control menu 630 in Figure 6H.

[0275] As shown in Figure 8AL, device 600 receives user input 850K (e.g., tap) corresponding to the selection of effect affordance 836 (e.g., detect). As shown in Figure 8AM, in response to receiving user input 850K, device 600 displays an enlarged image 843 from camera 603 of the user of device 600 and modifies the call control menu 832. In the modified call control menu 832, effect affordance 836 is highlighted and effect option affordances 844A to 844E are displayed. In Figure 8AN, device 600 receives user input 850L (e.g., tap) corresponding to the selection of effect option affordance 844A (e.g., detect). As shown in Figure 8AO, in response to receiving user input 850L, device 600 modifies the call control menu 832 to include avatar option affordances 846A to 846C. In Figure 8AO, device 600 receives (e.g., detects) user input 850M (e.g., tap) corresponding to the selection of avatar affordance 846C. As shown in Figure 8AP, in response to receiving user input 850M, device 600 displays effect 843A on the user's face in image 843 and scrolls the call control menu 832 to center the affordance 846C of the currently selected avatar. In Figure 8AP, device 600 receives (e.g., detects) user input 850N (e.g., tap) on cancel affordance 848. As shown in Figure 8AQ, in response to receiving user input 850N, device 600 returns the call control menu 832 to the configuration shown in Figure 8AN and maintains the display of effect 843A on the enlarged image 843. In Figure 8AQ, device 600 receives (e.g., detects) user input 850O (e.g., a tap) outside of the magnified image 843 and call control menu 832 on display 602. As shown in Figure 8AR, in response to receiving user input 850O, device 600 stops displaying the magnified image 843 and displays effect 843A on the user's image in the main representation 806.When effect 843A is activated, the user's live video stream on device 600 includes effect 843A, as shown in Figure 8AR.

[0276] As shown in Figure 8AR, the call control menu 832 includes media affordance 844E. Device 600 receives (e.g., detects) user input 850P (e.g., tap) corresponding to the selection of media affordance 844E. As shown in Figure 8AS, in response to receiving user input 850P, device 600 modifies the call control menu 832 to include media option affordances 852A to 852D. In Figure 8AS, device 600 receives (e.g., detects) user input 850Q (e.g., tap) corresponding to the selection of media option affordance 852A (sketch). In response to receiving user input 850Q, device 600 displays a representation of media content associated with media option affordance 852A in the canvas area 811.

[0277] As shown in Figure 8AT, in response to receiving user input 850Q, device 600 replaces the primary representation 810K (e.g., the smallest primary representation) with the representation 810L of the selected media item. In Figure 8AT, device 600 receives (e.g., detects) user input 850R (e.g., a downward swipe on the call control menu 832) in response to a request to hide the call control menu 832. As shown in Figure 8AU, in response to receiving user input 850R, device 600 stops displaying the call control menu 832.

[0278] In some embodiments, the selection of media option affordance 852A allows the relevant media content to be shared with other participants in the live communication session (e.g., displayed in a canvas area on the display of another participant's device). In some embodiments, participants in the live communication session can interact with the shared media content (e.g., by a sequence of one or more inputs in the user interface of the live communication application). Figure 8AV shows an embodiment in which additional feature 810L-1 is added to the media content of representation 810L. In some embodiments, in response to a participant's interaction with the media content associated with representation 801L, device 600 expands representation 801L (for example, by being selected after the main representation has been expanded, or by a participant corresponding to the representation actively participating). As shown in Figure 8AV, representation 810L is expanded in response to a participant's interaction with the relevant media content (compared to Figure 8AT). Figure 8AV also shows representation 810M of other content shared in the live communication session (e.g., a game between two participants).

[0279] Figures 9A to 9K are flowcharts illustrating a method according to some embodiments. Method 900 is performed on a device equipped with a display (e.g., 100, 300, 500, or 600). Some operations of Method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0280] As will be described later, Method 900 provides, among other things, an intuitive way to provide an adaptive and intelligent live communication user interface. In addition, Method 900 provides, among other things, an intuitive way for a user to interact with the live communication user interface. This method creates a more efficient human-machine interface by reducing the cognitive burden on the user by providing an adaptive and intelligent live communication interface based on the number of participants in the live communication. In the case of battery-operated computing devices, the ability for the user to view the live communication user interface based on a predetermined number of participants in the live communication results in faster processing, more efficient power saving, and longer battery charging intervals.

[0281] In block 901, the device receives a request (e.g., 850A) to display a user interface for a live communication session between two or more participants.

[0282] In block 902, in response to a request to display a user interface for a live communication session between two or more participants, the device displays a live communication user interface (e.g., 804), including simultaneously displaying representations of multiple participants (e.g., 810A-810D) in the live communication session, and displaying representations of multiple participants in the live communication session includes blocks 902 and 903. In some embodiments, the live communication session user interface includes placeholders (e.g., 810A in Figure 8C) (e.g., graphic placeholders (e.g., images or virtual avatars), text placeholders (e.g., names or initials)) related to a first participant who has been invited to the live communication session but has not yet joined the live communication session (e.g., a participant other than a user of the electronic device who has not yet joined the communication session and has been sent an invitation to join the communication session). In some embodiments, if participants are sharing a live media stream that includes a live video stream, the representation includes an image of the live video stream (e.g., 810B in Figure 8D). In some embodiments, if participants are sharing an audio-only live video stream, the representation includes the avatar of each participant (e.g., 810B in Figure 8I). In some embodiments, the representation of multiple participants in a live communication session includes user communication states selected from a group consisting of audio-only state (e.g., participant is communicating using audio only), video state (e.g., participant is speaking using both video and audio), video paused state (e.g., participant's video is paused), video undecoded state (e.g., participant's video stream cannot be processed due to latency issues, video stream format issues, etc.), exited state (e.g., participant has left the live communication session), and waiting to join state (e.g., participant has been invited to the live communication session but has not yet joined the live communication).

[0283] In block 903, upon determining that the number of participants in a live communication session exceeds the participant threshold, the device simultaneously displays representations of a first group of participants other than the user of the electronic device (e.g., 810A-810D) in one or more sizes larger than their respective threshold sizes, while displaying representations of a second group of participants (e.g., 822) in an overflow area (e.g., 820), where the second group of participants includes one or more participants not included in the first group of participants, and the representations of one or more other participants displayed in the overflow area are displayed in sizes smaller than their respective threshold sizes. In some embodiments, the currently active participant (e.g., 810E in Figure 8J) (e.g., the most active participant, a participant speaking or moving to satisfy a set of criteria for active participants) is displayed as part of the first group of participants. In some embodiments, the currently active participant is displayed in the main area of ​​the user interface (e.g., 811) (e.g., the area where the first group of participants is displayed). In some embodiments, if a participant who is not currently displayed in the main area (e.g., the participant corresponding to 822B in Figure 8U) becomes an active participant, a representation of that participant (e.g., 810H in Figure 8V) is displayed in the main area. In some embodiments, representations of a first group of participants (e.g., 810) are displayed in two or more different sizes (e.g., more active participants or more recently active participants are displayed in a larger size than less active participants or less recently active participants), while representations of a second group of participants in the overflow area (e.g., 822) are displayed in the same size.

[0284] In some embodiments, the overflow area (e.g., 820) further includes representations of a first group of participants (e.g., 810A-810D) (e.g., representations of participants displayed in the main area of ​​the user interface are also displayed in the overflow area) (e.g., Figure 8AX). In some embodiments, the overflow area (e.g., 820) further includes placeholder representations (e.g., 822A in Figure 8AW) (e.g., avatar, name, initials) for invitees to the live communication session who are not currently connected to the live communication session. By displaying participant representations differently based on whether certain conditions are met (e.g., whether the live communication session has more than a certain number of participants), it is made possible for the user to view participant representations in the live communication session more easily or efficiently. By performing optimized actions without requiring further user input when a set of conditions is met, the device's usability is enhanced, the user-device interface becomes more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0285] Optionally, in blocks 904 and 905, the device updates the display of the representations of a first group of participants (e.g., 810A-810D) (e.g., arrangement of representations, multiple positions, size, location, visual representation, visual content, and / or video feed) at a first rate (e.g., the representations of the first group of participants are live video feeds updated at a typical video refresh rate such as 24, 30, or 60 frames per second or a higher frame rate), and updates the display of the representations of a second group of participants (e.g., 822) in the overflow area (e.g., arrangement of representations, multiple positions, size, location, visual representation of representations, visual content of representations, and / or video feed) at a second rate, the second rate being lower than the first rate (e.g., thumbnails in the overflow area are updated over time, but at a lower rate than the video refresh rate of the representations of the group of participants in canvas area 811, such as 1, 5, or 10 frames per second). By updating the display of sets of representations at different rates, the device is enabled to provide the user with feedback on the current state of representations in live communication while conserving processing power by providing sets of representations at a faster rate than other sets of representations, thereby demonstrating to the user the ability to view higher-priority sets of representations at a higher rate. By providing the user with improved visual feedback, the device's usability is enhanced, making the user-device interface more efficient (for example, by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power usage is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0286] Optionally, in block 906, the device displays a representation of the electronic device's camera view (e.g., the view of a self-view camera) (e.g., 806) in the overflow region (e.g., 820) while displaying representations of a second group of participants (e.g., 822) in the overflow region (e.g., 820). By displaying a representation of the electronic device's camera view, the user is provided with visual feedback on the current state of the user camera view (e.g., what the user is showing to other participants in a live communication session). By providing the user with improved visual feedback, the usability of the device is enhanced, the user-device interface is made more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power usage is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0287] Optionally, in blocks 907, 908, 909, and 910, the device detects an input corresponding to the overflow region (e.g., 850H) (e.g., a horizontal finger drag). In response to the detection of an input corresponding to the overflow region, the device scrolls the overflow region to discontinue displaying (e.g., to hide) a subset of representations of the second multiple participants (e.g., 822G and 822H in Figures 8BJ-8BK) (e.g., scrolls the representations of the second multiple participants in a first direction in response to the input). While scrolling the overflow region, the device maintains a representation of the camera view of the electronic device (e.g., 806) within the overflow region (e.g., Figures 8BJ-8BK). In some embodiments, scrolling the overflow region includes displaying an animation of one or more representations of the second multiple participants that appear to scroll below the representation of the camera view of the device. By maintaining the display of the electronic device's camera view representation while scrolling through the overflow area, continuous feedback on the current state of the electronic device's camera view (e.g., what the user is showing to other participants in a live communication session) is provided to the user, while allowing the user the ability to scroll through the participant representations within the overflow area. By providing the user with improved visual feedback, device usability is enhanced, the user-device interface is made more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, power usage of the device is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0288] Optionally, blocks 911, 912, and 913 detect an input (e.g., 850F) corresponding to the selection of a representation of a second group of participants in the overflow region (e.g., 822E in Figure 8AC) (e.g., a tap gesture corresponding to a representation in the overflow region 820, or a gesture with a characteristic intensity exceeding an intensity threshold corresponding to a representation in the overflow region 820). In response to the detection of an input corresponding to the selection of a representation of a second group of participants in the overflow region, the device replaces the display of a representation of a first group of participants (e.g., 810J in Figure 8AC) with a representation of a participant corresponding to the selected representation in the overflow region (e.g., 810K in Figure 8AE) (e.g., replacing the least active participant in the main display). In some embodiments, replacing the display of a representation of a first group of participants with a representation of a participant corresponding to the selected representation in the overflow region includes discontinuing the display of a selected representation of a second group of participants in the overflow region. In some embodiments, discontinuing the display of a selected representation in the overflow area includes an animation in which the selected representation shrinks. In some embodiments, other representations in the overflow area are translated to fill the gap left by the selected representation. In some embodiments, replacing the display of one representation of a first group of participant representations with a participant representation corresponding to the selected representation in the overflow area includes displaying the participant representation corresponding to the selected representation at a first size, and then scaling the participant representation corresponding to the selected representation to a second size that is larger than the first size. In some embodiments, the selected participant representation automatically increases in size (e.g., to a medium representation size) after replacing the smallest representation in the canvas area (corresponding to the participant with the lowest activity level in the canvas area), while other representations (corresponding to the next lowest activity level in the canvas area) decrease in size (e.g., to a small representation size).

[0289] By replacing the display of the representation corresponding to the selected representation in the overflow area, and / or discontinuing the display of the selected representation in the overflow area, additional control options are provided without cluttering the UI, and by helping the user quickly replace representations with user-selected representations, the user gains greater control over the device and representation display. By providing additional control without cluttering the UI with additional displayed controllers, the usability of the device is improved, the user-device interface is made more efficient (for example, by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power consumption is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0290] Optionally, in block 914, the device displays a first animation effect that affects one of the representations of a first group of participants (e.g., Figures 8AA-8AC) (e.g., the representation becomes larger and then smaller with a crossfade and color change that optionally produces a “flash” effect). Optionally, in block 915, the device displays a second animation effect that affects a selected representation of a second group of participants, sharing animation properties with the first animation effect (e.g., Figures 8BF-8BH) (e.g., the fading / shrinking rate from the main display can correspond to the flashing rate of the overflow display). By displaying a second animation corresponding to the first animation, visual feedback is provided to quickly identify which representation is being switched and / or replaced. By providing users with improved visual feedback, device usability is enhanced, the user-device interface becomes more efficient (for example, by helping users provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling users to use the device more quickly and efficiently.

[0291] Optionally, in block 916, in response to the detection of an input corresponding to the overflow region (e.g., 850H), the device displays a new (e.g., previously hidden) representation (e.g., 822A in Figures 8AG-8AH) of a second group of participants in the overflow region. By displaying new representations of participants in the overflow region in response to input, the accessibility of the displayed representations is increased, allowing the user to view the representations more easily and efficiently. By performing optimized operation without requiring further user input when a set of conditions is met, the usability of the device is enhanced, the user-device interface is made more efficient (e.g., by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, the device's power consumption is reduced and battery life is improved by allowing the user to use the device more quickly and efficiently.

[0292] Optionally, in block 917, upon receiving data indicating that a new participant has joined or been added to a live communication session, the device displays a representation of the new participant (e.g., 822B in Figure 8R) in the overflow region at the end of a sequence of representations of a second group of participants in the overflow region.

[0293] Optionally, in blocks 918, 919, 920, and 921, the device receives second data relating to the activity level (e.g., speech, movement) of the first participant in the second group of participants. In response to receiving the second data, the device emphasizes (e.g., increases or starts) the visual characteristics (e.g., size, highlighting (e.g., display borders), animation (e.g., pulsating avatar), and bringing the image to the foreground) of the representation (e.g., 822B in Figure 8U) corresponding to the representation of the first participant in the second group of participants by increasing the activity level of the first participant in the second group of participants. In response to receiving the second data, the device suppresses (e.g., increases or starts) the emphasis (e.g., increases or starts) of the visual characteristics (e.g., size, highlighting (e.g., display borders), and animation (e.g., pulsating avatar) that bring the image to the foreground) of the representation (e.g., size, highlighting (e.g., display borders), and animation (e.g., pulsating avatar)) of the representation corresponding to the first participant in the second group of participants by decreasing the activity level of the first participant in the second group of participants. By highlighting and / or suppressing visual indicators when certain conditions are met, users can quickly recognize the activity levels of participants, including the most active and / or least active participants. Optimized operation without requiring further user input when a set of conditions is met enhances device usability and makes the user-device interface more efficient (for example, by assisting users in providing appropriate input when operating / interacting with the device and reducing user errors). Additionally, power consumption is reduced and battery life is improved by enabling users to use the device more quickly and efficiently. Furthermore, visual feedback indicating each participant's activity level is provided to the user by highlighting and / or suppressing when certain conditions are met.By providing users with improved visual feedback, device usability is enhanced, the user-device interface becomes more efficient (for example, by helping users provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling users to use the device more quickly and efficiently.

[0294] Optionally, in blocks 922 and 923, while displaying the live communication user interface, the device detects a second participant who will be the currently active participant among a second group of participants (e.g., the most active participant, a participant speaking or moving to satisfy a set of criteria for an active participant). Upon detecting a second participant who will be the currently active participant among a second group of participants, and determining that a representation of the second participant is not currently displayed in the overflow area, the device scrolls the overflow area to display a representation of the second participant (e.g., 822F in Figures 8BH to 8BI). By dynamically scrolling the overflow area to display the active participant, it becomes possible to see the representation of the active participant in the overflow area without selecting any additional input. Reducing the number of inputs required to perform an operation improves the usability of the device, making the user-device interface more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, reducing the device's power consumption and improving battery life by allowing users to use the device more quickly and efficiently.

[0295] In block 924, upon determining that a live communication session contains a number of participants less than or equal to a threshold number, the device displays representations of participants other than the user of the electronic device in one or more sizes greater than their respective threshold size, rather than displaying them in one or more sizes less than or equal to their respective threshold size (e.g., Figure 8P). In some embodiments, the electronic device receives multiple live media streams associated with each participant of multiple participants in a live video communication session (e.g., prior to displaying a user interface for the live communication session). In some embodiments, the number of received live media streams corresponds to the number of participants currently participating in the live video communication session, and does not include the participant using the device. By displaying participant representations differently based on whether certain conditions are met (e.g., displaying representations without an overflow area based on whether the live communication session is less than or equal to the number of participants), the user can view participant representations in the live communication session more easily or efficiently. By performing optimized actions without requiring further user input when a set of conditions is met, the device's usability is enhanced, the user-device interface becomes more efficient (for example, by assisting the user in providing appropriate input when operating / interacting with the device and reducing user errors), and in addition, power consumption of the device is reduced and battery life is improved by enabling the user to use the device more quickly and efficiently.

[0296] In some embodiments, if it is determined that the number of participants in a live communication session is less than a second threshold number of participants, the multiple representations of remote participants (e.g., non-user participants) do not overlap with each other (e.g., Figure 8J). In some embodiments, if it is determined that the number of participants in a live communication session is equal to or greater than the second threshold number of participants, at least some of the multiple representations of remote participants (e.g., non-user participants) overlap with each other (e.g., Figure 8K).

[0297] Optionally, in blocks 925 and 926, if it is determined that the number of participants in the live communication session is less than a second threshold number of participants (e.g., two participants including the user of the electronic device (e.g., one other participant)), the device displays at least one representation of a participant other than the device user in a first aspect ratio (e.g., 810B in Figure 8H) (e.g., a rectangular aspect ratio). If it is determined that the number of participants in the live communication session is greater than a second threshold number of participants (e.g., three or more participants), the device displays at least one representation of a participant other than the device use...

Claims

1. In electronic devices equipped with a display, Displaying a messaging user interface for a messaging application, wherein the messaging user interface includes a message area containing multiple messages in a message conversation between three or more participants. Receiving first data indicating that a live communication session available to the three or more participants is active, In response to receiving the first data indicating that the live communication session among the three or more participants is active, a notification indicating that the live communication session is active is displayed. While the notification indicating that the live communication session is active is displayed, a second data is received indicating that the live communication session among the three or more participants has become inactive. In response to receiving the second data indicating that the live communication session among the three or more participants has become inactive, the display of the notification is stopped, or the notification is updated to indicate that the live communication session has become inactive. Methods that include...

2. Displaying the notification indicating that the live communication session is active includes displaying the notification indicating that the live communication session is active at a first position in the message area, and the method is The method according to claim 1, further comprising moving the notification indicating that the live communication session is active from the first position to the second position in the message area.

3. The notification indicating that the live communication session is active includes a live communication participation affordance, and the method is The system receives a first user input corresponding to the selection of the aforementioned live communication participation affordance, In response to receiving the first user input corresponding to the selection of the live communication participation affordance, the user participates in the live communication session. The method according to claim 1 or 2, further comprising:

4. The notification indicating that the live communication session is active is displayed without generating a corresponding audio output, and the method Receiving reminder data indicating a reminder to participate in the aforementioned live communication, In response to receiving the reminder data indicating the reminder to participate in the live communication, one or more audio output devices of the device generate an audio output indicating that the live communication session is active. The method according to any one of claims 1 to 3, further comprising:

5. The notification indicating that the live communication session is active includes a menu access affordance, and the method is The system receives a second user input corresponding to the selection of the aforementioned menu access affordance, In response to receiving the second user input corresponding to the selection of the menu access affordance, a menu including video participation options and audio-only options is displayed. While the aforementioned menu is displayed, a third user input is received, In response to the third user input corresponding to the selection of the video participation option, the user participates in the live communication session using audio and video transmitted from the electronic device, In response to the third user input corresponding to the selection of the audio-only participation option, the user participates in the live communication session using audio transmitted from the electronic device without transmitting video data from the electronic device, The method according to any one of claims 1 to 4, further comprising:

6. The method according to any one of claims 1 to 5, wherein the notification indicating that the live communication session is active is displayed in the message area.

7. Receiving third data indicating that a second live communication session is available for two or more participants, Upon learning that the second live communication session is available between the two participants, a first type of notification indicating that the second live communication is active is displayed. In response to the fact that the second live communication session is available to three or more participants, a second type of notification, different from the first type of notification, is displayed to indicate that the second live communication is active. The method according to any one of claims 1 to 6, further comprising:

8. Displaying the second type of the aforementioned notice means The first display of the second type of notification in the first state, Transitioning the second type of notification to the second state, The method according to claim 7, including the method described in claim 7.

9. Receiving a fourth piece of data indicating that a third live communication session is available, In response to the receipt of the fourth data indicating that the third live communication session is available, Upon determining that an active application is displayed, a third type of notification is displayed indicating that the third live communication session is available, Upon determining that no active application is displayed, a fourth type of notification is displayed indicating that the third live communication session is available. The method according to any one of claims 1 to 8, further comprising:

10. Receiving a fifth piece of data indicating that a fourth live communication session is available, In response to the receipt of fifth data indicating that the fourth live communication session is available, Upon determining that the current status of the electronic device meets the alert notification criteria, a non-visual notification is provided indicating that the fourth live communication session is available, along with a notification that the fourth live communication session is available. Upon determining that the current status of the electronic device does not meet the alert notification criteria, the system displays the notification that the fourth live communication session is available without providing the non-visual notification that the fourth live communication session is available. The method according to any one of claims 1 to 9, further comprising:

11. The method according to claim 10, wherein the alert notification criterion includes the requirement that the device is locked in order for the alert notification criterion to be satisfied.

12. The method according to claim 10, wherein the alert notification criterion includes the requirement that the device does not detect user attention to the device in order for the alert notification criterion to be satisfied.

13. The method according to claim 4, further comprising displaying the participants in the live communication session that initiated the reminder, in response to the receipt of the reminder data indicating participation in the live communication.

14. The method according to claim 4, wherein the audio output indicating that the live communication session is active is different from the audio output associated with telephone call notifications and the audio output associated with message notifications.

15. The method according to any one of claims 1 to 14, wherein the notification includes displaying the number of participants in the live communication session.

16. The method according to any one of claims 1 to 15, wherein the notification includes a display on the display of the participant who performed an action that triggers the presentation of the notification.

17. The method according to any one of claims 1 to 16, further comprising, in response to receiving the first data indicating that the live communication session among the three or more participants is active, one or more audio output devices of the device generate a second audio output indicating that the live communication session is active, wherein the second audio output indicating that the live communication session is active is different from an audio output related to a telephone call notification and an audio output related to a message notification.

18. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 17.

19. The display and One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 17.

20. The display and Means for carrying out the method described in any one of claims 1 to 17, An electronic device equipped with the following features.

21. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs are Display the messaging user interface of a messaging application, including a message area containing multiple messages in a message conversation between three or more participants. Upon receiving first data indicating that a live communication session is active and available to the three or more participants, In response to receiving the first data indicating that the live communication session among the three or more participants is active, a notification indicating that the live communication session is active is displayed. While displaying the notification indicating that the live communication session is active, the system receives second data indicating that the live communication session among the three or more participants has become inactive. A non-temporary computer-readable storage medium including instructions for stopping the display of the notification or updating the notification to indicate that the live communication session among the three or more participants has become inactive, in response to the receipt of the second data indicating that the live communication session among the three or more participants has become inactive.

22. The display and One or more processors, An electronic device comprising: a memory that stores one or more programs configured to be executed by one or more processors, wherein the one or more programs are Display the messaging user interface of a messaging application, including a message area containing multiple messages in a message conversation between three or more participants. Upon receiving first data indicating that a live communication session is active and available to the three or more participants, In response to receiving the first data indicating that the live communication session among the three or more participants is active, a notification indicating that the live communication session is active is displayed. While displaying the notification indicating that the live communication session is active, the system receives second data indicating that the live communication session among the three or more participants has become inactive. An electronic device including a command to stop displaying the notification or update the notification to indicate that the live communication session among the three or more participants has become inactive, in response to the receipt of the second data indicating that the live communication session among the three or more participants has become inactive.

23. The display and A means for displaying a messaging user interface for a messaging application, which includes a message area containing multiple messages in a message conversation between three or more participants, means for receiving first data indicating that the live communication session available to the three or more participants is active, In response to receiving the first data indicating that the live communication session among the three or more participants is active, means for displaying a notification indicating that the live communication session is active, In response to the display of the notification indicating that the live communication session is active, means for receiving second data indicating that the live communication session among the three or more participants has become inactive, In response to receiving the second data indicating that the live communication session among the three or more participants has become inactive, the display of the notification is stopped, or the notification is updated to indicate that the live communication session has become inactive. An electronic device equipped with the following features.

24. In electronic devices equipped with a display, To receive requests to display a user interface for live communication sessions between two or more participants, In response to receiving the request to display the user interface for the live communication session between two or more participants, the live communication user interface is displayed, which includes simultaneously displaying representations of multiple participants in the live communication session. Including the display of the representations of the multiple participants in the live communication session, Upon determining that the number of participants in the live communication session exceeds the threshold number of participants, the representations of the first group of participants other than the user of the electronic device are simultaneously displayed in one or more sizes larger than their respective threshold sizes, while the second group of participants, including one or more participants not included in the first group of participants, are displayed in an overflow area, and the representations of the one or more other participants displayed in the overflow area are displayed in a size smaller than their respective threshold sizes. Upon determining that the number of participants in the live communication session is less than or equal to the threshold number of participants, the electronic device will not display the representation of participants other than the user in one or more sizes smaller than or equal to their respective threshold sizes, but will instead display the representation of participants other than the user in one or more sizes larger than their respective threshold sizes. Methods that include...

25. Updating the representation of the first group of participants at the first rate, Updating the display of the representations of the second group of participants in the overflow region at a second rate lower than the first rate, The method according to claim 24, further comprising:

26. The aforementioned representations of the first group of participants are displayed in two or more different sizes. The method according to claim 24 or 25, wherein the representations of the second plurality of participants within the overflow region are displayed at the same size.

27. The method according to any one of claims 24 to 26, further comprising displaying a representation of the camera view of the electronic device in the overflow region while the representations of the second plurality of participants are displayed in the overflow region.

28. To detect the input corresponding to the overflow region, In response to the detection of the input corresponding to the overflow region, Scrolling the overflow area to stop displaying a subset of the representations of the second group of participants, Scrolling the overflow region while maintaining the display of the representation of the camera view of the electronic device within the overflow region, The method according to claim 27, further comprising:

29. Displaying the representations of the multiple participants in the aforementioned live communication session is, Upon determining that the number of participants in the live communication session is less than a second threshold number of participants, at least one of the representations of the participants other than the user of the device is displayed in the first aspect ratio. Upon determining that the number of participants in the live communication session exceeds the second threshold number of participants, at least one of the representations of the participants other than the user of the device is displayed in a second aspect ratio different from the first aspect ratio. The method according to any one of claims 24 to 28, further comprising:

30. Displaying the representations of the multiple participants in the aforementioned live communication session is, Upon determining that the number of participants in the live communication session is less than the second threshold number of participants, the camera view of the device is displayed in a third aspect ratio. Upon determining that the number of participants in the live communication session is equal to or greater than the second threshold number of participants, the representation of the view of the camera of the device is displayed in a fourth aspect ratio different from the third aspect ratio. The method according to any one of claims 24 to 29, further comprising:

31. While the device is displaying representations of participants other than the user, it receives data indicating that the number of participants in the live communication session has increased, In response to the receipt of the data indicating an increase in the number of participants in the live communication session, To reduce the size of one or more of the representations of participants other than the user of the device, Displaying one or more representations of new participants in the aforementioned live communication session, The method according to any one of claims 24 to 30, further comprising:

32. Displaying the representations of the multiple participants in the aforementioned live communication session is, Given that the number of participants in the live communication session is the first number of participants, the representation of the participants is displayed in the first layout, In response to the fact that the number of participants in the live communication session is a second number of participants, the representation of the participants is displayed in a second layout different from the first layout. The method according to any one of claims 24 to 31, further comprising:

33. The method according to claim 32, wherein the first layout is one of a plurality of predetermined layouts in which the user's representation is fixed at different positions in the user interface.

34. The method according to claim 32 or 33, wherein the first layout and the second layout are included in a predetermined family of layouts having similar relative arrangements of user representations.

35. The method according to any one of claims 24 to 34, wherein the live communication session user interface further includes placeholders relating to a first participant who has been invited to the live communication session but has not yet joined the live communication session.

36. The placeholder includes a first reminder affordance for the first participant, and the method is: Receiving an input corresponding to the selection of the first reminder affordance, In response to receiving the input corresponding to the selection of the first reminder affordance, a new notification different from the previous notification sent to the first participant is sent to the first participant, The method according to claim 35, further comprising:

37. The placeholders and the representations of the first group of participants are arranged in a first layout, and the method is as follows: If it is determined that the first participant did not participate in the live communication session within the specified time, To discontinue the display of the aforementioned placeholder, The arrangement of the representations of the first group of participants is changed to a second layout different from the first layout, which includes enlarging and / or moving one or more of the representations of the participants. The method according to claim 35 or 36, further comprising:

38. The method according to any one of claims 35 to 37, further comprising determining that the first participant has joined the live communication session, and modifying the display of the placeholder to represent the first participant.

39. Displaying the representations of the multiple participants in the aforementioned live communication session is, Upon determining that the number of participants in the live communication session is less than the second threshold number of participants, the multiple representations of the remote participants do not overlap with each other. Upon determining that the number of participants in the live communication session is equal to or greater than the second threshold number of participants, at least a portion of the multiple representations of the remote participant overlap each other, The method according to any one of claims 24 to 38, including the method described above.

40. While displaying the representations of the multiple participants in the live communication session, the system detects changes in the number of participants in the live communication session. In response to the detection of the change in the number of participants in the live communication session, a notification indicating that the number of participants in the live communication session has changed will be displayed. The method according to any one of claims 24 to 39, further comprising:

41. To detect an input corresponding to the selection of one of the representations of the second plurality of participants in the overflow region, In response to the detection of the input corresponding to the selection of one of the representations of the second plurality of participants in the overflow region, Replacing the representation of one of the representations of the first plurality of participants with the representation of the participant corresponding to the selected representation in the overflow area, The method according to any one of claims 24 to 40, further comprising:

42. The method according to claim 41, wherein replacing the display of one of the representations of the first plurality of participants with the representation of the participant corresponding to the selected representation in the overflow region includes displaying a first animation effect that affects the representation of the first plurality of participants.

43. The method of claim 42, further comprising replacing the display of the representation of the first plurality of participants with the representation of the participant corresponding to the selected representation in the overflow region, which further includes displaying a second animation effect that shares animation properties with the first animation effect, affecting the selected representation of the second plurality of participants.

44. The method according to any one of claims 24 to 43, further comprising displaying a new representation of the second plurality of participants in the overflow region in response to the detection of the input corresponding to the overflow region.

45. The method according to any one of claims 24 to 44, further comprising displaying a representation of the new participant in the overflow region at the end of a sequence of representations of the second plurality of participants in the overflow region, in response to the receipt of data indicating that a new participant has joined or been added to the live communication session.

46. Receiving second data regarding the activity level of the first participant among the second group of participants, Upon receiving the second data, In response to an increase in the activity level of the first participant among the second plurality of participants, the visual characteristics of the representation corresponding to the representation of the first participant among the representations of the second plurality of participants in the overflow region are emphasized, In response to a decrease in the activity level of the first participant among the second plurality of participants, the emphasis on the visual characteristics of the representation corresponding to the first participant among the representations of the second plurality of participants in the overflow region is suppressed, The method according to any one of claims 24 to 45, further comprising:

47. The method according to any one of claims 24 to 46, wherein being currently active is displayed as part of the first group of participants.

48. While the live communication user interface is being displayed, it is detected that the second participant among the second group of participants becomes the currently active participant. In response to the detection that the second participant among the second group of participants becomes the currently active participant, and in response to the determination that the representation of the second participant among the second group of participants is not currently displayed in the overflow area, the overflow area is scrolled to display the representation of the second participant among the second group of participants in the overflow area. The method according to any one of claims 24 to 47, further comprising:

49. The method according to any one of claims 24 to 48, wherein the overflow region further includes representations of the first plurality of participants.

50. The method according to any one of claims 24 to 49, wherein the overflow region further includes a placeholder representation for an invitee to the live communication session who is not currently connected to the live communication session.

51. As part of the representation of the multiple participants in the live communication session, the representation of the camera view of the device is displayed at a first size, While displaying the representations of the multiple participants in the live communication session, a first input corresponding to a selection for displaying a first menu is received. In response to receiving the input corresponding to the selection for displaying the first menu, the first menu, which includes options for displaying visual effects on the representation of a participant in the live communication session, is displayed on the live communication user interface. In response to receiving a second input corresponding to the selection of the option for displaying a visual effect, the representation of the view of the camera of the device is displayed in a second size that is larger than the first size, The method according to any one of claims 24 to 50, further comprising:

52. Receiving a third input corresponding to a selection for expanding the first expression among the expressions of the multiple participants in the live communication session, In response to receiving the third input corresponding to the selection for expanding the first expression among the expressions of the multiple participants in the live communication session, the first expression is expanded, To reduce the size of at least one representation of the multiple participants in the live communication user interface that is different from the first representation, The method according to any one of claims 24 to 51, further comprising:

53. The method according to any one of claims 24 to 52, wherein the representation of the plurality of participants in the live communication session includes a user communication state selected from the group consisting of audio-only state, video state, video paused state, video undecoded state, exit state, and waiting state.

54. Receiving data indicating a change in the orientation of the aforementioned electronic device, In response to receiving the data indicating a change in the orientation of the electronic device, Rotating the display of the representations of the multiple participants in the live communication session, While rotating the aforementioned display, the relative characteristics of the previous display of the aforementioned representations of the multiple participants in the live communication session are maintained. The method according to any one of claims 24 to 53, further comprising:

55. The method according to any one of claims 24 to 54, further comprising changing the display of the first representation among the representations of the plurality of participants in the live communication session based on the position of a face in the first representation in response to the reception of the first data indicating a change in the orientation of the electronic device.

56. The representation of the multiple participants in the live communication includes a second representation which includes a first portion of a live media stream that is received by the electronic device and displayed in a sixth aspect ratio, and the method is While displaying the second representation among the representations of the multiple participants, a request is received to change the aspect ratio of at least one representation among the representations of the multiple participants in the live communication session. In response to receiving the request to change the aspect ratio of at least one representation of the multiple participants in the live communication session, the aspect ratio of the second representation is changed from the sixth aspect ratio to the seventh aspect ratio, wherein the second representation includes a second portion of the live media stream received by the electronic device while it has the seventh aspect ratio, and the second portion is selected to include a face detected in the live media stream. The method according to any one of claims 24 to 55, further comprising:

57. While displaying the representations of the multiple participants in the live communication session, a fourth input corresponding to a request to display a menu is received, In response to receiving the fourth input, Displaying a menu on the user interface for the live communication session, To move at least the third representation among the representations of the multiple participants in the live communication, To change the size of at least the fourth representation among the representations of the multiple participants in the live communication session, The method according to any one of claims 24 to 56, further comprising:

58. The method according to claim 41, wherein replacing the display of the expression among the first plurality of participant expressions with the expression of the participant corresponding to the selected expression in the overflow region further includes discontinuing the display of the selected expression among the second plurality of participant expressions in the overflow region.

59. The method according to claim 41, wherein replacing the display of the representation of one of the first plurality of participants with the representation of the participant corresponding to the selected representation in the overflow region further includes displaying the representation of the participant corresponding to the selected representation at a first size, and then enlarging the representation of the participant corresponding to the selected representation to a second size that is larger than the first size.

60. Receiving data indicating that the activity level of the first participant among the second plurality of participants represented in the overflow region rises above the activity level of the participant corresponding to the first representation among the representations of the first plurality of participants, In response to the receipt of data indicating that the activity level of the first participant among the second plurality of participants represented in the overflow region rises above the activity level of the participant corresponding to the first representation among the representations of the first plurality of participants, Replacing the representation of the first expression among the representations of the first group of participants with the representation of the first participant among the second group of participants, To discontinue the display of the representation of the first participant within the overflow region, The method according to any one of claims 24 to 59, further comprising:

61. The method according to claim 60, wherein replacing the display of the first representation among the representations of the first plurality of participants includes displaying the representation of the participant corresponding to the replaced first representation in the overflow area.

62. While displaying the representations of the multiple participants in the live communication session, the system receives input corresponding to a request to display a first menu. In response to receiving the input corresponding to the request to display the first menu, the first menu, which includes options for selecting content items, is displayed on the live communication user interface. It receives input corresponding to the selection of content items, In response to receiving the input corresponding to the selection of the content item, the representation of the participant displayed in a size larger than the respective threshold size is replaced with the representation of the content item in a size larger than the respective threshold size, The method according to any one of claims 24 to 61, further comprising:

63. Displaying the representations of the second group of participants in the overflow area means Upon determining that a first number of participants are participating in the live communication session, the first set of representations is displayed in the overflow area. Upon determining that a second number of participants, different from the first number of participants, are participating in the live communication session, a second set of representations, different from the first set of representations, which includes a different number of representations from the first set of representations, is displayed in the overflow area. The method according to any one of claims 24 to 62, including the method described above.

64. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs include instructions for performing the method according to any one of claims 24 to 63.

65. The display and One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 24 to 63.

66. The display and Means for carrying out the method described in any one of claims 24 to 63, An electronic device equipped with the following features.

67. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs are We received a request to display a user interface for live communication sessions between two or more participants. In response to receiving the request to display the user interface for the live communication session between the two or more participants, the command includes displaying a live communication user interface, which includes simultaneously displaying representations of multiple participants in the live communication session. Displaying the representations of the multiple participants in the aforementioned live communication session is, Upon determining that the number of participants in the live communication session exceeds the participant threshold, the representations of a first group of participants other than the user of the electronic device are simultaneously displayed in one or more sizes larger than their respective threshold sizes, while the representations of a second group of participants are displayed in the overflow area, and the second group of participants includes one or more participants not included in the first group of participants, and the representations of the one or more other participants displayed in the overflow area are displayed in a size smaller than their respective threshold sizes. Upon determining that the number of participants in the live communication session is less than or equal to the threshold number of participants, the electronic device will not display the representation of participants other than the user in one or more sizes smaller than or equal to their respective threshold sizes, but will instead display the representation of participants other than the user in one or more sizes larger than their respective threshold sizes. Methods that include...

68. The display and One or more processors, An electronic device comprising: a memory that stores one or more programs configured to be executed by one or more processors, wherein the one or more programs are We received a request to display a user interface for live communication sessions between two or more participants. In response to receiving the request to display the user interface for the live communication session between two or more participants, the command includes displaying the representations of multiple participants simultaneously in the live communication session, and the displaying of the representations of multiple participants in the live communication session includes, Upon determining that the number of participants in the live communication session exceeds the threshold number of participants, the representations of a first group of participants other than the user of the electronic device are simultaneously displayed in one or more sizes larger than their respective threshold sizes, while the representations of a second group of participants are displayed in an overflow area, and the second group of participants includes one or more participants not included in the first group of participants, and the representations of the one or more other participants displayed in the overflow area are displayed in a size smaller than their respective threshold sizes. Upon determining that the number of participants in the live communication session is less than or equal to the threshold number of participants, the electronic device will not display the representation of participants other than the user in one or more sizes smaller than or equal to their respective threshold sizes, but will instead display the representation of participants other than the user in one or more sizes larger than their respective threshold sizes. Methods that include...

69. The display and Means for requesting the display of a user interface for a live communication session between two or more participants, The system includes means for displaying a live communication user interface, which includes simultaneously displaying representations of multiple participants in the live communication session in response to receiving the request to display the user interface for the live communication session between two or more participants, and the means for displaying representations of multiple participants in the live communication session is A means for determining that the number of participants in the live communication session exceeds the threshold number of participants, to simultaneously display representations of a first plurality of participants other than the user of the electronic device in one or more sizes larger than their respective threshold sizes, while displaying representations of a second plurality of participants in the overflow area, wherein the second plurality of participants includes one or more participants not included in the first plurality of participants, and the representations of one or more other participants displayed in the overflow area are displayed in sizes smaller than their respective threshold sizes, A means for determining that the number of participants in the live communication session is less than or equal to the threshold number of participants, to display the representations of participants other than the user on the electronic device in one or more sizes larger than the respective threshold size, rather than displaying them in one or more sizes smaller than or equal to the respective threshold size. An electronic device equipped with the following features.

70. In electronic devices equipped with a display, To display a live video communication user interface that simultaneously displays the representation of a first participant in a live video communication session, the representation of a second participant in the live video communication session, and the representation of a third participant in the live video communication session, While the live video communication user interface is displayed, data is received indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session. In response to the receipt of data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, Changing the size and / or position of the representation of the first participant in the live video communication user interface, Changing the size and / or position of the representation of the second participant in the live video communication user interface, Changing the size and / or position of the representation of the third participant in the live video communication user interface, Methods that include...

71. After changing the size and / or position of the representation of the first participant in the live communication user interface, the changed representation of the first participant occupies at least a portion of the area that the representation of the first participant occupied in the live communication user interface before changing at least one of the size or position of the representation of the first participant. After changing the size and / or position of the representation of the second participant in the live communication user interface, the changed representation of the second participant occupies at least a portion of the area that the representation of the second participant occupied in the live communication user interface before changing at least one of the size or position of the representation of the second participant. The method according to claim 70, wherein, after changing the size and / or position of the representation of the third participant in the live communication user interface, the changed representation of the third participant occupies at least a portion of the area that the representation of the third participant occupied in the live communication user interface before changing at least one of the size or position of the representation of the third participant.

72. In response to the receipt of data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, The size of the representation of the first participant in the live communication user interface is changed. The size of the representation of the second participant in the live communication user interface is changed. The size of the representation of the third participant in the live communication user interface is changed. The method according to claim 70 or 71, including the act of doing so.

73. In response to the receipt of data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, The position of the representation of the first participant in the live communication user interface is changed. The position of the representation of the second participant in the live communication user interface is changed. The position of the representation of the third participant in the live communication user interface is changed. The method according to any one of claims 70 to 72, including the following:

74. The live communication user interface displays the representation of the fourth participant, In response to receiving data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, the size and / or position of the representation of the fourth participant in the live communication user interface is changed. The method according to any one of claims 70 to 73, further comprising:

75. After changing the size and / or position of the representation of the first participant in the live video communication user interface, receiving second data indicating that the activity level of the first participant in the live video communication session has risen above the activity level of the second participant in the live video communication session, In response to the receipt of the second data indicating that the activity level of the first participant in the live video communication session has risen above the activity level of the second participant in the live video communication session, Changing the size and / or position of the representation of the first participant in the live video communication user interface, Changing the size and / or position of the representation of the second participant in the live video communication user interface, Maintaining the size and position of the representation of the third participant in the live communication session, The method according to any one of claims 70 to 74, further comprising:

76. The method according to any one of claims 70 to 75, wherein at least the activity level of the first participant is based on an audio and / or video feed received from the first participant's device.

77. The method according to claim 76, wherein at least the activity level of the first participant is based on audio filtered from an audio feed received from the first participant's device.

78. The method according to claim 76 or 77, wherein at least the activity level of the first participant is based on movement that satisfies a set of movement criteria, identified in a video feed received from the first participant's device.

79. In response to the receipt of the first data indicating that the activity level of the second participant in the live communication session has risen above the activity level of the first participant in the live communication session, In the live communication user interface, the visual display of the second participant's representation is emphasized, In the live communication user interface, the emphasis on the visual display of the first participant's representation is suppressed. The method according to any one of claims 70 to 78, further comprising:

80. Prior to receiving the data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, The aforementioned representation of the first participant is displayed in a first size, The representation of the second participant is displayed in a second size that is smaller than the first size. The method according to any one of claims 70 to 79, wherein changing the size and / or position of the representation of the second participant includes increasing the size of the representation of the second participant from the second size to the first size.

81. Changing the size and / or position of the representation of the first participant is, This includes changing the size and / or position of the representation of the first participant based on facial recognition data obtained from the video feed of the second participant, provided that changing the size and / or position of the representation of the first participant does not result in obscuring the face of the second participant. The method according to any one of claims 70 to 80.

82. The method according to any one of claims 70 to 81, wherein, prior to receiving the data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, one or more representations of the first participant, the second participant, and the third participant overlap with at least one other representation of the participant.

83. Prior to receiving the data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, the representations of the first, second, and third participants in the live communication user interface are displayed in z-order. The method according to claim 82, wherein changing the size and / or position of the representations of the first, second, and third participants in the live communication user interface includes maintaining the z-order of the representations of the first, second, and third participants.

84. The method according to any one of claims 70 to 83, wherein a portion of the representation of the participant with the highest activity level among the first, second, and third participants overlaps with a portion of the representation of another participant among the first, second, and third participants in the live communication user interface.

85. The method according to any one of claims 70 to 84, wherein a portion of the representation of the participant with the lowest activity level among the first, second, and third participants overlaps with a portion of the representation of another participant among the first, second, and third participants in the live communication user interface.

86. After receiving the data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, the input corresponding to the representation of the first participant is detected. The method according to any one of claims 70 to 85, in response to the detection of the input corresponding to the representation of the first participant of the first participant.

87. The fourth participant in the live communication session receives data indicating that they are more active than the participant with the lowest activity level among the first, second, and third participants, In response to receiving data indicating that the fourth participant in the live communication session is more active than the participant with the lowest activity level among the first, second, and third participants, the display of the representation of the participant with the lowest activity level among the first, second, and third participants is replaced with the display of the representation of the fourth participant. The method according to any one of claims 70 to 86, further comprising:

88. Replacing the representation of the participant with the lowest activity level among the first, second, and third participants with the representation of the fourth participant is: The method according to claim 87, further comprising displaying the representation of the fourth participant in a size larger than the size of the representation of the participant with the lowest activity level among the first, second, and third participants.

89. The method according to any one of claims 70 to 88, wherein changing the size and / or position of the representation of the first participant includes gradually increasing the size of the representation from a first size to a second size.

90. The method according to any one of claims 70 to 89, wherein the live communication interface includes a canvas area, and the representation of the first participant in the live communication session, the representation of the second participant in the live communication session, and the representation of the third participant in the live communication session are displayed in the canvas area.

91. Detecting canvas resizing events, In response to the detection of the canvas resize event, Resize the aforementioned canvas area, Modify the layout of the aforementioned representations of the first, second, and third participants, The method according to claim 90, further comprising:

92. The method according to claim 91, further comprising resizing one or more of the representations of the first, second, and third participants in response to the detection of the canvas resizing event.

93. The live communication session user interface displays a representation of the camera view of the electronic device, In response to the receipt of the first data indicating that the activity level of the second participant in the live communication session has risen above the activity level of the first participant in the live communication session, Maintaining the size of the representation of the camera view of the electronic device in the live communication session user interface, The method according to any one of claims 70 to 92, further comprising:

94. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs include instructions for performing the method according to any one of claims 70 to 93.

95. The display and One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 70 to 93.

96. The display and Means for carrying out the method described in any one of claims 70 to 93, An electronic device equipped with the following features.

97. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs are A live video communication user interface is displayed that simultaneously shows the representation of a first participant in a live video communication session, the representation of a second participant in the live video communication session, and the representation of a third participant in the live video communication session. While the live video communication user interface is being displayed, data is received indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session. In response to the receipt of data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, The size and / or position of the representation of the first participant in the live video communication user interface is changed. The size and / or position of the representation of the second participant in the live video communication user interface is changed. A non-temporary computer-readable storage medium containing instructions for changing the size and / or position of the representation of the third participant in the live video communication user interface.

98. The display and One or more processors, An electronic device comprising: a memory that stores one or more programs configured to be executed by one or more processors, wherein the one or more programs are A live video communication user interface is displayed that simultaneously shows the representation of a first participant in a live video communication session, the representation of a second participant in the live video communication session, and the representation of a third participant in the live video communication session. While the live video communication user interface is being displayed, data is received indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session. In response to the receipt of data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, The size and / or position of the representation of the first participant in the live video communication user interface is changed. The size and / or position of the representation of the second participant in the live video communication user interface is changed. An electronic device including commands for changing the size and / or position of the representation of the third participant in the live video communication user interface.

99. The display and Means for displaying a live video communication user interface that simultaneously displays the representation of a first participant in a live video communication session, the representation of a second participant in the live video communication session, and the representation of a third participant in the live video communication session; Means for receiving data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session while the live video communication user interface is being displayed, In response to the receipt of data indicating that the activity level of the second participant in the live video communication session has risen above the activity level of the first participant in the live video communication session, The size and / or position of the representation of the first participant in the live video communication user interface is changed. The size and / or position of the representation of the second participant in the live video communication user interface is changed. Means for changing the size and / or position of the representation of the third participant in the live video communication user interface, An electronic device equipped with the following features.

100. In electronic devices equipped with a display, Displaying the messaging user interface of a messaging application, wherein the messaging user interface is A message area containing multiple messages between participants in a message conversation involving three or more participants, The live video communication application includes an affordance separate from the message area for initiating a live video communication session, To detect the input corresponding to the selection of the aforementioned affordance, In response to the detection of the input corresponding to the selection of the affordance, the live video communication application initiates a process to start the live video communication session, wherein the live video communication session includes the participants of the message conversation. Methods that include...

101. In the live communication application, starting the process of starting the live communication session means that Transitioning from the messaging user interface to the messaging user interface of the live communication application, To initiate the aforementioned live communication session, The method according to claim 100, including the method described in claim 100.

102. Initiating the process of initiating the live communication session in the live communication application includes displaying a group contact user interface that includes group contact affordances for initiating the live communication session, and the method is To detect an input corresponding to the selection of the group contact affordance for initiating the live communication session, In response to the detection of the input corresponding to the selection of the group contact affordance for initiating the live communication session, the live communication application initiates the live communication session. The method according to claim 100, further comprising:

103. After the aforementioned live communication session has started, In response to the determination that the first participant did not participate in the live communication session within a predetermined time, a reminder option related to the first participant is displayed on the group contact user interface. While displaying the reminder options related to the first participant, the system receives input corresponding to the selection of the reminder options, Upon receiving the input corresponding to the selection of the reminder option, a notification is sent to the first participant indicating that the live communication session is available. The method according to claim 102, further comprising:

104. The group contact user interface further includes a plurality of participant affordances, including a first participant affordance relating to a first participant among the participants of the message conversation, and the method To detect an input corresponding to the selection of the first participant affordance related to the first participant in the message conversation, In response to the detection of the input corresponding to the selection of the first participant affordance related to the first participant in the message conversation, an individual participant user interface including the characteristics of the first participant is displayed. The method according to claim 102, further comprising:

105. The first participant affordance includes an option for selecting a communication mode for contacting the first participant in the message conversation, and the method is: To detect an input corresponding to the selection of the option for selecting the communication mode, In response to the detection of the input corresponding to the selection of the communication mode option, the process of selecting the commuting mode for the first participant is initiated. The method according to claim 104, further comprising:

106. The group contact user interface further includes new contact affordances, and the method is To detect input corresponding to the selection of the new contact affordance, In response to the detection of the input corresponding to the selection of the new contact affordance, the process of adding the new contact to the live communication session is initiated. The method according to any one of claims 103 to 105, further comprising:

107. Initiating the process of starting the live video communication session means, In the aforementioned live communication application, the live communication session is initiated, Displaying the live communication session user interface, wherein the live communication session user interface is This includes a placeholder relating to the first participant among the participants of the live communication session, The method according to any one of claims 100 to 106, including the following:

108. The first placeholder includes a first reminder affordance for the first participant among the participants of the live communication session, and the method is Receiving an input corresponding to the selection of the first reminder affordance, In response to receiving the input corresponding to the selection of the first reminder affordance, a new notification different from the previous notification sent to the first participant is sent to the first participant, The method according to claim 107, further comprising:

109. The live communication session user interface further includes a second placeholder relating to a second participant, the first placeholder and the second placeholder are arranged according to a first layout, and the method is as follows: If it is determined that the first participant did not participate in the live communication session within the specified time, To discontinue the display of the first placeholder, The arrangement of the second placeholder is changed to a second layout that is different from the first layout, The method according to claim 107 or 108, further comprising:

110. The method according to any one of claims 107 to 109, further comprising determining that the first participant has joined the live communication session, and modifying the display of the first placeholder to represent the first participant.

111. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs include instructions for performing the method according to any one of claims 100 to 110.

112. The display and One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 100 to 110.

113. The display and Means for performing the method described in any one of claims 100 to 110, An electronic device equipped with the following features.

114. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display and a touch-sensing surface, wherein the one or more programs are A messaging user interface for a messaging application, A message area containing multiple messages between participants in a message conversation involving three or more participants, A messaging user interface is displayed in a live video communication application, which includes an affordance separate from the message area for initiating a live video communication session. The input corresponding to the selection of the aforementioned affordance is detected, A non-temporary computer-readable storage medium containing an instruction for initiating the process of starting the live video communication session, including the participants of the message conversation, in response to the detection of the input corresponding to the selection of the affordance.

115. The display and One or more processors, An electronic device comprising: a memory that stores one or more programs configured to be executed by one or more processors, wherein the one or more programs are A messaging user interface for a messaging application, A message area containing multiple messages between participants in a message conversation involving three or more participants, A messaging user interface is displayed in a live video communication application that includes an affordance separate from the message area for initiating a live video communication session. The input corresponding to the selection of the aforementioned affordance is detected, An electronic device comprising an instruction to initiate a process in the live video communication application to start the live video communication session, including the participants of the message conversation, in response to the detection of the input corresponding to the selection of the affordance.

116. It is an electronic device, The display and A means for displaying the messaging user interface of a messaging application, wherein the messaging user interface is A message area containing multiple messages between participants in a message conversation involving three or more participants, A means for initiating a live video communication session in a live video communication application, including an affordance separate from the message area, means for detecting an input corresponding to the selection of the aforementioned affordance, A means for initiating a process to start the live video communication session in the live video communication application in response to the detection of the input corresponding to the selection of the affordance, wherein the live video communication session includes the participants of the message conversation, An electronic device equipped with the following features.

117. In electronic devices equipped with a display, A live video communication user interface for a live video communication application, Multiple participant affordances, each corresponding to one or more participants, including a first participant affordance corresponding to a group of three or more participants, Displaying a live video communication user interface, including a new session affordance for starting a new live video communication session, To detect input corresponding to a request to initiate the process of starting a new live video communication session, In response to the detection of the aforementioned input, Upon receiving confirmation that the input corresponds to the selection of the first participant affordance, the process of starting a new live video communication session including the group of three or more participants is initiated. Upon receiving that the input corresponds to the selection of the new session affordance, the process of selecting two or more participants is initiated, and a new live video communication session is started including the participant associated with the device and the two or more selected participants. Methods that include...

118. The method according to claim 117, wherein the live communication user interface further includes a list of recent live communication sessions.

119. The method according to claim 117 or 118, wherein the plurality of participant affordances include a second participant affordance corresponding to a proposed group of participants, which is determined based on data from a second application different from the live video communication application.

120. The method according to claim 119, wherein the proposed participant group corresponds to a participant in a group conversation or conference call conducted in the second application.

121. The method according to claim 119 or 120, wherein the second participant affordance includes the representation of the second application.

122. The method according to any one of claims 119 to 121, wherein the plurality of participant affordances include a third participant affordance corresponding to a proposed second group of participants, which is determined based on data from a third application that is different from the live video communication application and also different from the second application.

123. The live communication user interface further includes participant selection affordances, and the method is Receiving a second input corresponding to the selection of the aforementioned participant selection affordance, Upon receiving the second user input corresponding to the selection of the participant selection affordance, A text input field to identify one or more participants, Keyboard and Audio-specific affordances, Displaying a group selection user interface, including video affordances, While the aforementioned group selection user interface is displayed, a third user input selection is received. In response to receiving the third user input, In response to the third user input corresponding to the selection of the audio-only affordance, the new live communication session is initiated using the audio transmitted from the electronic device without transmitting video data from the electronic device. In response to the third user input corresponding to the selection of the video affordance, the new live communication session is initiated using the audio and video transmitted from the electronic device, The method according to any one of claims 117 to 122, further comprising:

124. The aforementioned new live communication session will be initiated, After the aforementioned new live communication session is started, the fourth application displays a representation of the new live communication session that is different from the aforementioned live communication session. The method according to any one of claims 117 to 123, further comprising:

125. The aforementioned group of participant affordances includes a fourth participant affordance corresponding to a group of three or more participants. Upon determination that the fourth participant affordance corresponds to an active live communication session, the fourth participant affordance is displayed having a first visual appearance. The method according to any one of claims 117 to 124, wherein, upon determination that the fourth participant affordance corresponds to an inactive live communication session, the fourth participant affordance is displayed having a second visual appearance different from the first visual appearance.

126. Initiating the process of selecting two or more participants includes displaying representations of potential participants. Upon determination that the potential participant is associated with a device capable of participating in a live communication session, the representation of the potential participant is displayed having a first visual appearance. The method according to any one of claims 117 to 125, wherein, upon determination that the participant is not associated with a device capable of participating in a live communication session, the representation of the potential participant is displayed having a second visual appearance different from the first visual appearance.

127. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display, wherein the one or more programs include instructions for performing the method according to any one of claims 117 to 126.

128. The display and One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 117 to 126.

129. The display and Means for performing the method described in any one of claims 117 to 126, An electronic device equipped with the following features.

130. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device, wherein the one or more programs are A live video communication user interface for a live video communication application, Multiple participant affordances, each corresponding to one or more participants, including a first participant affordance corresponding to a group of three or more participants, Display a live video communication user interface, including a new session affordance for starting a new live video communication session. Detect the input corresponding to the request to initiate the process of starting a new live video communication session, In response to the detection of the aforementioned input, Upon receiving confirmation that the input corresponds to the selection of the first participant affordance, the process of starting a new live video communication session including the group of three or more participants is initiated. A non-temporary computer-readable storage medium containing instructions for initiating a process to select two or more participants, and initiating a new live video communication session including the participants associated with the device and the two or more selected participants, upon receiving that the input corresponds to the selection of the new session affordance.

131. The display and One or more processors, An electronic device comprising: a memory that stores one or more programs configured to be executed by one or more processors, wherein the one or more programs are In electronic devices equipped with a display, A live video communication user interface for a live video communication application, Multiple participant affordances, each corresponding to one or more participants, including a first participant affordance corresponding to a group of three or more participants, Display a live video communication user interface, including a new session affordance for starting a new live video communication session. Detect the input corresponding to the request to initiate the process of starting a new live video communication session, In response to the detection of the aforementioned input, Upon receiving confirmation that the input corresponds to the selection of the first participant affordance, the process of starting a new live video communication session including the group of three or more participants is initiated. An electronic device that, upon receiving that the input corresponds to the selection of the new session affordance, initiates a process to select two or more participants, and initiates a new live video communication session including the participants associated with the device and the two or more selected participants.

132. The display and A live video communication user interface for a live video communication application, Multiple participant affordances, each corresponding to one or more participants, including a first participant affordance corresponding to a group of three or more participants, A means for displaying a live video communication user interface, including a new session affordance for starting a new live video communication session, Means for detecting input corresponding to a request to initiate the process of starting a new live video communication session, In response to the detection of the aforementioned input, Upon receiving confirmation that the input corresponds to the selection of the first participant affordance, the process of starting a new live video communication session including the group of three or more participants is initiated. In response to the input corresponding to the selection of the new session affordance, means for starting a process to select two or more participants and starting a new live video communication session including the participant associated with the device and the two or more selected participants, An electronic device equipped with the following features.

Images