Real-time communication user interface
By simplifying the real-time communication session management method and interface, the problems of complex interface and low efficiency in the existing technology are solved, achieving faster and more efficient communication management, improving user experience and device energy efficiency.
Patent Information
- Application Number
- CN202510977559.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-05-09
- Filing Date
- 2022-05-13
- Publication Date
- 2025-10-31
AI Technical Summary
Existing technologies for managing real-time communication sessions have complex and inefficient user interfaces, wasting user time and device energy, especially in battery-powered devices.
It provides a faster and more efficient method and interface for real-time communication session management, allowing users to select communication protocols and invite participants by displaying the interaction between the generating components and input devices, including multiple sharing options and security controls.
It reduces the cognitive burden on users, improves the efficiency and safety of devices, saves power for battery-powered devices, and extends battery life.
Smart Images

Figure CN120881039A_ABST
Abstract
Description
[0001] Case Separation Statement
[0002] This application is a divisional application of the invention patent application with PCT international application number PCT / US2022 / 029273, international application date May 13, 2022, Chinese national phase application number 202280034909.1, and invention title "Real-time Communication User Interface".
[0003] Cross-references to related applications
[0004] This application claims priority to U.S. Patent Application Serial No. 17 / 740,104 entitled “REAL-TIME COMMUNICATION USER INTERFACE”, filed May 9, 2022, and U.S. Provisional Patent Application Serial No. 63 / 189,081 entitled “REAL-TIME COMMUNICATION USER INTERFACE”, filed May 15, 2021, the contents of each of which are incorporated herein by reference in their entirety. Technical Field
[0005] This disclosure relates generally to computer user interfaces, and more specifically to techniques for managing real-time communication sessions. Background Technology
[0006] The electronic device includes a camera and microphone that enable users to communicate with each other in real time via audio and / or video. Furthermore, the electronic device displays a user interface that allows users to select one or more participants to engage in the real-time communication. Summary of the Invention
[0007] However, some technologies used to manage real-time communication sessions with electronic devices are often cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may involve multiple keystrokes or button presses. These technologies require more time than necessary, resulting in wasted user time and device power. This latter consideration is particularly important in battery-powered devices.
[0008] Therefore, the present invention provides electronic devices with a faster and more efficient method and interface for managing real-time communication sessions. Such methods and interfaces optionally complement or replace other methods for managing real-time communication sessions. These methods and interfaces reduce the cognitive burden on users and result in a more effective human-machine interface. Such methods and interfaces enable users to invite participants to real-time communication sessions via multiple communication protocols. Furthermore, such methods and interfaces facilitate the creation and joining of real-time communication sessions. In addition, such methods and interfaces enhance the security of electronic devices by restricting access and / or functionality for certain users. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charging.
[0009] According to some embodiments, a method is described. The method is performed at a computer system communicating with a display generation component and one or more input devices. The method includes: displaying a user interface of a real-time communication application via the display generation component, the user interface including user-interactive graphical user interface elements corresponding to a link to a real-time communication session; receiving user input via one or more input devices corresponding to selection of the user-interactive graphical user interface elements; and in response to receiving the user input: initiating a process for sharing a link to the real-time communication session, wherein the process for sharing a link to the real-time communication session includes displaying a sharing user interface via the display generation component, the sharing user interface including a plurality of optional options for sharing the link to the real-time communication session, including simultaneously displaying: a first optional option for sharing a link to the real-time communication session via a first communication protocol; and a second optional option for sharing a link to the real-time communication session via a second communication protocol different from the first communication protocol.
[0010] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: displaying a user interface for a real-time communication application via the display generation component, the user interface including user-interactive graphical user interface elements corresponding to a link to a real-time communication session; receiving user input via the one or more input devices corresponding to selection of user-interactive graphical user interface elements; and, in response to receiving user input: initiating a process for sharing a link to the real-time communication session, wherein the process for sharing a link to the real-time communication session includes displaying a sharing user interface via the display generation component, the sharing user interface including multiple optional options for sharing the link to the real-time communication session, including simultaneously displaying: a first optional option for sharing a link to the real-time communication session via a first communication protocol; and a second optional option for sharing a link to the real-time communication session via a second communication protocol different from the first communication protocol.
[0011] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: displaying a user interface for a real-time communication application via the display generation component, the user interface including user-interactive graphical user interface elements corresponding to a link to a real-time communication session; receiving user input via the one or more input devices corresponding to selection of user-interactive graphical user interface elements; and, in response to receiving user input: initiating a process for sharing a link to the real-time communication session, wherein the process for sharing a link to the real-time communication session includes displaying a sharing user interface via the display generation component, the sharing user interface including multiple optional options for sharing the link to the real-time communication session, including simultaneously displaying: a first optional option for sharing a link to the real-time communication session via a first communication protocol; and a second optional option for sharing a link to the real-time communication session via a second communication protocol different from the first communication protocol.
[0012] According to some embodiments, a computer system is described. The computer system communicates with a display generation component and one or more input devices. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a user interface for a real-time communication application via the display generation component, the user interface including user-interactive graphical user interface elements corresponding to a link to a real-time communication session; receiving user input via the one or more input devices corresponding to selection of user-interactive graphical user interface elements; and in response to receiving user input: initiating a process for sharing a link to the real-time communication session, wherein the process for sharing a link to the real-time communication session includes displaying a sharing user interface via the display generation component, the sharing user interface including a plurality of optional options for sharing a link to the real-time communication session, including simultaneously displaying: a first optional option for sharing a link to the real-time communication session via a first communication protocol; and a second optional option for sharing a link to the real-time communication session via a second communication protocol different from the first communication protocol.
[0013] According to some embodiments, a computer system is described. The computer system communicates with a display generation component and one or more input devices. The computer system includes: means for displaying a user interface of a real-time communication application via the display generation component, the user interface including user-interactive graphical user interface elements corresponding to a link to a real-time communication session; means for receiving user input via one or more input devices corresponding to selection of the user-interactive graphical user interface elements; and means for performing the following operation in response to receiving the user input: initiating a process for sharing a link to the real-time communication session, wherein the process for sharing a link to the real-time communication session includes displaying a sharing user interface via the display generation component, the sharing user interface including a plurality of optional options for sharing a link to the real-time communication session, including simultaneously displaying: a first optional option for sharing a link to the real-time communication session via a first communication protocol; and a second optional option for sharing a link to the real-time communication session via a second communication protocol different from the first communication protocol.
[0014] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: displaying a user interface for a real-time communication application via the display generation component, the user interface including user-interactive graphical user interface elements corresponding to a link to a real-time communication session; receiving user input via the one or more input devices corresponding to selection of user-interactive graphical user interface elements; and, in response to receiving user input: initiating a process for sharing a link to the real-time communication session, wherein the process for sharing a link to the real-time communication session includes displaying a sharing user interface via the display generation component, the sharing user interface including multiple optional options for sharing the link to the real-time communication session, including simultaneously displaying: a first optional option for sharing a link to the real-time communication session via a first communication protocol; and a second optional option for sharing a link to the real-time communication session via a second communication protocol different from the first communication protocol.
[0015] According to some embodiments, a method is described. The method is performed at a computer system communicating with a display generation component and one or more input devices. The method includes: detecting one or more first user inputs corresponding to a selection of a user to be invited to participate in a real-time communication session via the one or more input devices; after detecting the one or more first user inputs, simultaneously displaying via the display generation component: optional user interface elements corresponding to a link to the real-time communication session; and a visual indication of the user; while simultaneously displaying the optional user interface elements and the user's visual indication, detecting second user input corresponding to a selection of the optional user interface elements via the one or more input devices; and in response to detecting the second user input: initiating a process for sending a link to the user; and displaying a user interface for participating in a real-time communication session with the user.
[0016] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting one or more first user inputs via the one or more input devices corresponding to a selection of a user to be invited to participate in a real-time communication session; after detecting the one or more first user inputs, simultaneously displaying via the display generation component: an optional user interface element corresponding to a link to the real-time communication session; and a visual indication of the user; while simultaneously displaying the optional user interface element and the user's visual indication, detecting second user input via the one or more input devices corresponding to a selection of the optional user interface element; and in response to detecting the second user input: initiating a process for sending a link to the user; and displaying a user interface for participating in a real-time communication session with the user.
[0017] According to some embodiments, a transient computer-readable storage medium is described. This transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting one or more first user inputs via the one or more input devices corresponding to a selection of a user to be invited to participate in a real-time communication session; after detecting the one or more first user inputs, simultaneously displaying via the display generation component: an optional user interface element corresponding to a link to the real-time communication session; and a visual indication of the user; while simultaneously displaying the optional user interface element and the user's visual indication, detecting a second user input via the one or more input devices corresponding to a selection of the optional user interface element; and in response to detecting the second user input: initiating a process for sending a link to the user; and displaying a user interface for participating in a real-time communication session with the user.
[0018] According to some embodiments, a computer system is described. The computer system communicates with a display generation component and one or more input devices. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: detecting one or more first user inputs corresponding to a selection of a user to be invited to participate in a real-time communication session via the one or more input devices; after detecting the one or more first user inputs, simultaneously displaying via the display generation component: optional user interface elements corresponding to a link to the real-time communication session; and a visual indication of the user; while simultaneously displaying the optional user interface elements and the user's visual indication, detecting second user input corresponding to a selection of the optional user interface elements via the one or more input devices; and in response to detecting the second user input: initiating a process for sending a link to the user; and displaying a user interface for participating in a real-time communication session with the user.
[0019] According to some embodiments, a computer system is described. The computer system communicates with a display generation component and one or more input devices. The computer system includes: means for detecting one or more first user inputs corresponding to a selection of a user to be invited to participate in a real-time communication session via one or more input devices; means for performing the following operations: after detecting one or more first user inputs, simultaneously displaying via the display generation component: optional user interface elements corresponding to a link to the real-time communication session; and visual indications for the user; means for performing the following operations: while simultaneously displaying the optional user interface elements and the user's visual indications, detecting second user input corresponding to a selection of the optional user interface elements via one or more input devices; and means for performing the following operations: in response to detecting the second user input: initiating a process for sending a link to the user; and displaying a user interface for participating in a real-time communication session with the user.
[0020] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting one or more first user inputs via the one or more input devices corresponding to a selection of a user to be invited to participate in a real-time communication session; after detecting the one or more first user inputs, simultaneously displaying via the display generation component: optional user interface elements corresponding to a link to the real-time communication session; and a visual indication of the user; while simultaneously displaying the optional user interface elements and the user's visual indication, detecting second user input via the one or more input devices corresponding to a selection of the optional user interface elements; and in response to detecting the second user input: initiating a process for sending a link to the user; and displaying a user interface for participating in a real-time communication session with the user.
[0021] According to some embodiments, a method is described. The method is performed at a computer system communicating with a display generation component and one or more input devices. The method includes: while the computer system displays a user interface via the display generation component a visual representation of an active real-time communication session including a first user, a second user, different from the first user, attempts to join the real-time communication session; and after the second user attempts to join the real-time communication session, displaying a visual representation of the second user's attempt to join the real-time communication session in the user interface, wherein displaying the visual representation of the second user's attempt to join the real-time communication session includes: based on determining that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option selectable to determine whether the second user is allowed to participate in the real-time communication session; and based on determining that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without displaying the option selectable to determine whether the second user is allowed to participate in the real-time communication session.
[0022] According to some implementation schemes, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: when the computer system displays a user interface via the display generation component showing a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication session; and after the second user attempts to join the real-time communication session, displaying a visual representation of the second user's attempt to join the real-time communication session in the user interface, wherein displaying the visual representation of the second user's attempt to join the real-time communication session includes: based on determining that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option to determine whether to allow the second user to participate in the real-time communication session; and based on determining that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without displaying the option to determine whether to allow the second user to participate in the real-time communication session.
[0023] According to some implementation schemes, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: when the computer system displays a user interface via the display generation component showing a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication session; and after the second user attempts to join the real-time communication session, displaying a visual representation of the second user's attempt to join the real-time communication session in the user interface, wherein displaying the visual representation of the second user's attempt to join the real-time communication session includes: based on determining that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option to determine whether to allow the second user to participate in the real-time communication session; and based on determining that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without displaying the option to determine whether to allow the second user to participate in the real-time communication session.
[0024] According to some implementation schemes, a computer system is described. The computer system communicates with a display generation component and one or more input devices. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: when the computer system displays a user interface via a display generation component that shows a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication session; and after the second user attempts to join the real-time communication session, displays a visual representation of the second user's attempt to join the real-time communication session in the user interface, wherein displaying the visual representation of the second user's attempt to join the real-time communication session includes: based on determining that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option selectable to determine whether to allow the second user to participate in the real-time communication session; and based on determining that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without displaying the option selectable to determine whether to allow the second user to participate in the real-time communication session.
[0025] According to some embodiments, a computer system is described. The computer system communicates with a display generation component and one or more input devices. The computer system includes: means for performing the following operations: when the computer system displays a user interface via the display generation component that includes a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication session; and means for performing the following operations: after the second user attempts to join the real-time communication session, displaying a visual representation of the second user's attempt to join the real-time communication session in the user interface, wherein displaying the visual representation of the second user's attempt to join the real-time communication session includes: based on determining that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option selectable to determine whether to allow the second user to participate in the real-time communication session; and based on determining that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without displaying the option selectable to determine whether to allow the second user to participate in the real-time communication session.
[0026] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: when the computer system displays a user interface via the display generation component showing a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication session; and after the second user attempts to join the real-time communication session, displaying a visual representation of the second user's attempt to join the real-time communication session in the user interface, wherein displaying the visual representation of the second user's attempt to join the real-time communication session includes: based on determining that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option to determine whether to allow the second user to participate in the real-time communication session; and based on determining that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without displaying the option to determine whether to allow the second user to participate in the real-time communication session.
[0027] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.
[0028] Therefore, devices are provided with faster and more efficient methods and interfaces for managing real-time communication sessions, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods used for managing real-time communication sessions. Attached Figure Description
[0029] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.
[0030] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.
[0031] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.
[0032] Figure 2A portable multi-functional device with a touchscreen is shown according to some embodiments.
[0033] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.
[0034] Figure 4A An exemplary user interface for a menu on an application on a portable multifunction device, according to some implementation schemes, is shown.
[0035] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is shown according to some embodiments.
[0036] Figure 5A A personal electronic device according to some implementation schemes is shown.
[0037] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.
[0038] Figure 5C An exemplary diagram of a communication session between electronic devices according to some implementation schemes is shown.
[0039] Figures 6A to 6V An exemplary user interface for creating and / or sharing links to real-time communication sessions is shown, according to some implementation schemes.
[0040] Figures 7A to 7H This is a flowchart illustrating a method for sharing a link to a real-time communication session according to some implementation schemes.
[0041] Figures 8A to 8G This is a flowchart illustrating a method for inviting participants to a real-time communication session and joining the real-time communication session, according to some implementation schemes.
[0042] Figures 9A to 9H An exemplary user interface for joining a real-time communication session is shown according to some implementation schemes.
[0043] Figure 10 This is a flowchart illustrating a method for allowing participants to join a real-time communication session according to some implementation schemes. Detailed Implementation
[0044] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.
[0045] There is a need for electronic devices that provide efficient methods and interfaces for managing real-time communication sessions. For example, there is a need for electronic devices that facilitate the creation of links to real-time communication sessions and provide multiple sharing options for sharing links to real-time communication sessions. Another example is the need for electronic devices that facilitate users' ability to invite participants to real-time communication sessions. Furthermore, there is a need for electronic devices that can restrict certain users from joining real-time communication sessions and provide users with the ability to authorize and / or deny access to real-time communication sessions. Such technologies can reduce the cognitive burden on users participating in and / or hosting real-time communication sessions, thereby improving productivity. In addition, such technologies can reduce processor power and battery power that would otherwise be wasted on redundant user input.
[0046] under Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5C A description of an exemplary device for performing management event notifications is provided. Figures 6A to 6V An exemplary user interface is shown for creating and / or sharing links to real-time communication sessions. Figures 7A to 7H This is a flowchart illustrating a method for sharing a link to a real-time communication session according to some implementation schemes. Figures 8A to 8G This is a flowchart illustrating a method for inviting participants to a real-time communication session and joining the real-time communication session. Figures 6A to 6V The user interface in the document is used to illustrate the process described below, including Figures 7A to 7H and Figures 8A to 8G The process in. Figures 9A to 9H An exemplary user interface for joining a real-time communication session is shown. Figure 10 This is a flowchart illustrating a method for allowing participants to join a real-time communication session according to some implementation schemes. Figures 9A to 9H The user interface in the document is used to illustrate the process described below, including Figure 10 The process in.
[0047] Furthermore, in methods described herein where one or more steps depend on the satisfaction of one or more conditions, it should be understood that the method may be repeated in multiple repetitions such that, during the repetitions, all conditions determining the steps in the method are satisfied in different repetitions of the method. For example, if a method requires performing a first step (if a condition is satisfied) and a second step (if a condition is not satisfied), those skilled in the art will know that the stated steps are repeated until both conditions are satisfied and not satisfied (in no particular order). Thus, a method described as having one or more steps depending on the satisfaction of one or more conditions can be rewritten as a method that repeats until each condition described in the method is satisfied. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing discretionary operations based on the satisfaction of the corresponding one or more conditions, and thus to determine whether possible conditions have been satisfied without explicitly repeating the steps of the method until all conditions determining the steps in the method are satisfied. Those skilled in the art will also understand that, similar to methods having discretionary steps, a system or computer-readable storage medium may repeat the steps of the method multiple times as needed to ensure that all discretionary steps have been performed.
[0048] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first touch may be named a second touch and similarly, a second touch may be named a first touch, without departing from the scope of the various described embodiments. Both the first touch and the second touch are touches, but they are not the same touch.
[0049] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0050] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "at," or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if determination..." or "if detection [the stated condition or event]" may optionally be interpreted as meaning "in response to determination..." or "in response to detection [the stated condition or event]."
[0051] This document describes implementations of electronic devices, user interfaces for such devices, and associated processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functionalities such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, those from Apple Inc. (Cupertino, California). Devices, iPod Equipment, and Device. Optionally, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the electronic device is a computer system that communicates with a display generating component (e.g., via wireless or wired communication). The display generating component is configured to provide visual output, such as a display via a CRT monitor, a display via an LED monitor, or a display via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. As used herein, “display” content includes displaying content (e.g., video data rendered or decoded by display controller 156) by transmitting data (e.g., image data or video data) to an integrated or external display generating component via a wired or wireless connection to visually generate content.
[0052] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.
[0053] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.
[0054] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.
[0055] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.
[0056] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values that includes at least four different values and more typically hundreds of different values (e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or variation of the contact area detected on the touch-sensitive surface, the capacitance and / or variation of the touch-sensitive surface near the contact, and / or the resistance and / or variation of the touch-sensitive surface near the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold (e.g., the intensity threshold is described in units corresponding to the substitute measurement) has been exceeded. In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold (e.g., the intensity threshold is a pressure threshold measured in units of pressure) has been exceeded. Using the intensity of the contact as an attribute of user input allows the user to access additional device functions that would otherwise be inaccessible to the user on a smaller device with limited physical space, which is used (e.g., on a touch-sensitive display) to display an indication and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).
[0057] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement does not move. For example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or sensed by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user.
[0058] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.
[0059] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls other components of device 100 to access memory 102.
[0060] Peripheral interface 118 can be used to couple the device's input and output peripherals to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs (such as computer programs (e.g., including instructions)) and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.
[0061] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals and vice versa, and communicates with communication networks and other communication devices via these electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks and other devices, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as via near-field communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed Downlink Packet Access (HSDPA), High-Speed Uplink Packet Access (HSUPA), Evolution, Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE...). 802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.
[0062] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset with both output (e.g., a single-ear or dual-ear headphone) and input (e.g., a microphone).
[0063] I / O subsystem 106 couples input / output peripherals on device 100, such as touchscreen 112 and other input control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / send electrical signals to the other input control device 116. The other input control device 116 optionally includes physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click dials, etc. In some embodiments, input controller 160 is optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing device such as mouse. One or more buttons (e.g., Figure 2 Optionally, 208) includes an increase / decrease button for volume control of speaker 111 and / or microphone 113. The one or more buttons optionally include a push-button (e.g., Figure 2(Ref. 206 in the original text). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication, via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking user gestures (e.g., hand gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system.
[0064] A quick press of the push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of the push button (e.g., 206) optionally powers the device 100 on or off. The function of one or more buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.
[0065] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.
[0066] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.
[0067] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that from Apple Inc. (Cupertino, California). and iPod The technology used.
[0068] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles a multi-touch touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.
[0069] The touch-sensitive display in some embodiments of the touchscreen 112 is described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, “Mode-Based Graphical User Interfaces For Touch Sensitive Input”. (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, “Virtual Input Device Placement On A Touch Screen User Interface”; (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, “Operation Of A Computer With A Touch Screen Interface”; (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard”; and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, “Multi-Functional Hand-Held Device”. The full text of all these applications is incorporated herein by reference.
[0070] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally use any suitable object or accessory such as a stylus, finger, etc., to interact with touchscreen 112. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.
[0071] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touchscreen, does not display visual output. Optionally, the touchpad is a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.
[0072] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.
[0073] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 is shown coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensor within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.
[0074] The device 100 optionally also includes one or more depth camera sensors 175. Figure 1A A depth camera sensor is shown coupled to a depth camera controller 169 in I / O subsystem 106. Depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with imaging module 143 (also referred to as a camera module), depth camera sensor 175 may optionally be used to determine depth maps of different portions of an image captured by imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information are optionally acquired for video conferencing while the user views other video conferencing participants on a touchscreen display, and selfies with depth map data are captured. In some embodiments, depth camera sensor 175 is located at the rear of the device, or both the rear and front of device 100. In some embodiments, the position of depth camera sensor 175 may be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.
[0075] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1AA contact strength sensor is shown coupled to a strength sensor controller 159 in I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.
[0076] The device 100 optionally also includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally perform as described in the following U.S. patent applications: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, the proximity sensor is turned off and the touchscreen 112 is disabled when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).
[0077] The device 100 may optionally also include one or more tactile output generators 167. Figure 1AA haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components for converting electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.
[0078] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally perform as described in the following U.S. Patent Publications: 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and 20060017692, entitled "Methods And Apparatuses For Operating A Portable DeviceBased On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.
[0079] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application program (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the figure. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, indicating what applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and position information relating to the device's position and / or orientation.
[0080] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
[0081] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is connected to… (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.
[0082] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.
[0083] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).
[0084] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.
[0085] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.
[0086] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like to specify the graphic to be displayed, and, if necessary, also receives coordinate data and other graphic attribute data, and then generates screen image data for output to the display controller 156.
[0087] The haptic feedback module 133 includes various software components for generating instructions which are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.
[0088] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).
[0089] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to phone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).
[0090] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:
[0091] • Contacts module 137 (sometimes called address book or contact list);
[0092] • Telephone module 138;
[0093] • Video conferencing module 139;
[0094] • Email client module 140;
[0095] • Instant Messaging (IM) module 141;
[0096] Fitness support module 142;
[0097] • Camera module 143 for still images and / or video images;
[0098] • Image management module 144;
[0099] • Video player module;
[0100] Music player module;
[0101] • Browser module 147;
[0102] • Calendar module 148;
[0103] • Widget module 149, which optionally includes one or more of the following: weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets acquired by the user, and user-created widgets 149-6;
[0104] • Widget creator module 150 for creating user-created widgets 149-6;
[0105] • Search module 151;
[0106] • Video and music player module 152, which combines a video player module and a music player module;
[0107] • Notes module 153;
[0108] • Map module 154; and / or
[0109] • Online video module 155.
[0110] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.
[0111] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.
[0112] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify entered telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.
[0113] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.
[0114] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.
[0115] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing a received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).
[0116] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (executive devices); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.
[0117] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.
[0118] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.
[0119] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.
[0120] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.
[0121] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally a microapplication downloaded and used by a user (e.g., weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a user-created microapplication (e.g., user-created widget 149-6). In some embodiments, the widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! widgets).
[0122] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 can optionally be used by the user to create widgets (e.g., to convert user-specified portions of a webpage into widgets).
[0123] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.
[0124] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).
[0125] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the note-taking module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.
[0126] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is optionally used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.
[0127] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following actions: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Further descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, both of which are incorporated herein by reference in their entirety.
[0128] Each of the above modules and applications corresponds to an executable set of instructions for performing one or more of the functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods as described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 optionally stores subgroups of the aforementioned modules and data structures. Additionally, memory 102 optionally stores other modules and data structures not described above.
[0129] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on device 100 can be optionally reduced.
[0130] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, home menu, or root menu. In such embodiments, a "menu button" is implemented using a touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.
[0131] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 137 to 151, 155, 380 to 390).
[0132] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.
[0133] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.
[0134] Event monitor 171 receives event information from peripheral device interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripheral device interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral device interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.
[0135] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other implementations, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).
[0136] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.
[0137] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.
[0138] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.
[0139] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.
[0140] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.
[0141] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.
[0142] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.
[0143] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.
[0144] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).
[0145] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device orientation).
[0146] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event (187) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on the displayed object for a predetermined duration, movement of the touch on the touch-sensitive display 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.
[0147] In some implementations, event definition 187 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.
[0148] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.
[0149] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.
[0150] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.
[0151] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some implementations, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with that flag retrieves the flag and executes a predefined process.
[0152] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and executes a predetermined process.
[0153] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on touch-sensitive display.
[0154] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.
[0155] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.
[0156] Figure 2A portable multifunction device 100 with a touchscreen 112 is shown according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.
[0157] Device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.
[0158] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.
[0159] Figure 3This is a block diagram of an exemplary multi-functional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the reference above). Figure 1A The tactile output generator 167 and sensor 359 (e.g., optical sensor, accelerometer, proximity sensor, touch sensor and / or contact intensity sensor (similar to the one mentioned above)) are described. Figure 1A The contact strength sensor 165 is described above. The memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU 310. In some embodiments, the memory 370 stores information related to the portable multifunction device 100. Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.
[0160] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each of the modules described above corresponds to a set of instructions for performing the functions described above. The modules or computer programs described above (e.g., instruction sets or including instructions) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules, and therefore various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subgroup of the modules and data structures described above. In addition, memory 370 optionally stores additional modules and data structures not described above.
[0161] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.
[0162] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is shown. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof:
[0163] • Signal strength indicator 402 for wireless communications such as cellular signals and Wi-Fi signals;
[0164] • Time 404;
[0165] Bluetooth indicator 405;
[0166] • Battery status indicator 406;
[0167] • Tray tray 408 with icons for commonly used applications, such as:
[0168] ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemail messages;
[0169] ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails;
[0170] ○ The icon 420 labeled "Browser" in browser module 147; and
[0171] ○ The icon 422 marked "iPod" for the video and music player module 152 (also known as the iPod module 152, a trademark of Apple Inc.); and
[0172] • Icons of other applications, such as:
[0173] ○The icon 424 of the IM module 141 marked as "Message";
[0174] ○The calendar module 148 has an icon 426 labeled "Calendar";
[0175] ○ The icon 428 of the image management module 144 is labeled "Photo".
[0176] ○ The icon 430 of camera module 143, which is labeled "camera";
[0177] ○ The icon 432 of the online video module 155, which is labeled "Online Video";
[0178] ○ The icon 434 labeled "Stock Market" in the Stock Market widget 149-2;
[0179] ○The icon 436 of the map module 154 that is labeled "map";
[0180] ○The weather widget 149-1 has icon 438 labeled "weather";
[0181] ○ The alarm clock widget 149-4 has an icon 440 labeled "clock";
[0182] ○ The icon 442 of the fitness support module 142 is labeled "fitness support";
[0183] ○ The icon 444 labeled "Notes" in Notes module 153; and
[0184] ○ The icon 446, labeled "Settings," is used to set the settings of the device 100 and its various applications 136.
[0185] It should be pointed out that, Figure 4A The icon labels shown are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.
[0186] Figure 4B A touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)) is shown. Figure 3 Devices such as tablets or touchpads (e.g., 355) Figure 3An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.
[0187] While some examples of input on a reference touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations the device detects input on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450)). Figure 4B The spindle corresponding to 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to the corresponding position on the display (e.g., in the example). Figure 4B In the middle, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, when the touch-sensitive surface (e.g., ...) Figure 4B 451) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on that touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may optionally be used for other user interfaces described herein.
[0188] Additionally, while the examples below are primarily given with reference to finger input (e.g., finger touch, single-finger tap, finger swipe), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the swipe path (e.g., instead of movement of the touch). Similarly, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Likewise, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.
[0189] Figure 5AAn exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include components relative to devices 100 and 300 (e.g., Figures 1A to 4B Some or all of the features described herein. In some embodiments, device 500 has a touch-sensitive display 504, referred to below as touchscreen 504. As an alternative to or complement to touchscreen 504, device 500 has a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., touch). The one or more intensity sensors of touchscreen 504 (or touch-sensitive surface) can provide output data representing the intensity of the touch. The user interface of device 500 can respond to touches based on the intensity of the touch, meaning that touches of different intensities can invoke different user interface operations on device 500.
[0190] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related patent applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application,” published under WIPO Patent Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships,” published under WIPO Patent Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.
[0191] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) may be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.
[0192] Figure 5B An exemplary personal electronic device 500 is depicted. In some embodiments, device 500 may include a reference retrieval system. Figure 1A , Figure 1B and Figure 3 Some or all of the components described herein. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive components 522 and optionally have an intensity sensor 524 (e.g., a contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 may optionally be a rotatable input device or a pressable input device and a rotatable input device. In some examples, input mechanism 508 may optionally be a button.
[0193] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.
[0194] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause the computer processors to perform the techniques described below, including processes 700, 800, and 1000. Figures 7A to 7H , Figures 8A to 8G and Figure 10A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.
[0195] As used herein, the term "power indication" refers to an indicator optionally displayed on devices 100, 300, and / or 500 ( Figure 1A , Figure 3 and Figures 5A to 5C A user-interactive graphical user interface object on a display screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.
[0196] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of enabling direct interaction with user interface elements on the touchscreen display) Figure 1A The touch-sensitive display system 112 or Figure 4AIn some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).
[0197] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted away, before or after contact begins to move, before contact ends, before or after contact intensity is detected to increase and / or before or after contact intensity decreases). The characteristic intensity of the contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half maximum value of the contact intensity, the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some implementations, the feature intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a feature intensity not exceeding the first threshold results in a first action, contact with a feature intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a feature intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the feature intensity and one or more thresholds is used to determine whether to perform one or more actions (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.
[0198] In some implementations, a portion of the gesture is identified to determine the characteristic intensity. For example, a touch-sensitive surface optionally receives a series of swipe contacts that transition from a starting position to an ending position, where the contact intensity increases. In this example, the characteristic intensity of the contact at the ending position is optionally based only on a portion of the series of swipe contacts, rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some implementations, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: unweighted moving average smoothing algorithm, triangular smoothing algorithm, median filter smoothing algorithm, and / or exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contact to achieve the purpose of determining the characteristic intensity.
[0199] Optionally, the contact intensity on a touch-sensitive surface is characterized relative to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device performs an operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device performs an operation different from the operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, when a contact with a characteristic intensity lower than the light press intensity threshold (e.g., and higher than the nominal contact detection intensity threshold, where contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector based on the movement of the contact on the touch-sensitive surface without performing the operation associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent across different groups of user interface figures.
[0200] An increase in contact intensity from below a light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in contact intensity from below a deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in contact intensity from below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in contact intensity from above a contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a contact being lifted off the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.
[0201] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the intensity of the contact (or multiple contacts) increases to above a press input intensity threshold. In some embodiments, the corresponding operation is performed in response to detecting that the intensity of the corresponding contact increases to above a press input intensity threshold (e.g., a "downward stroke" of the corresponding press input). In some embodiments, the press input includes the intensity of the corresponding contact increasing to above a press input intensity threshold and the intensity of the contact subsequently decreasing to below the press input intensity threshold, and the corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the press input threshold (e.g., an "upward stroke" of the corresponding press input).
[0202] In some implementations, the device employs intensity hysteresis to avoid unintended inputs sometimes referred to as "jitter," wherein the device defines or selects a hysteresis intensity threshold that has a predefined relationship with a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the press input intensity threshold). Therefore, in some implementations, a press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of that contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., an "upstroke" of the corresponding press input). Similarly, in some implementations, a press input is detected only when the device detects that the contact intensity increases from an intensity equal to or below the hysteresis intensity threshold to an intensity equal to or above the press input intensity threshold and optionally the contact intensity subsequently decreases to an intensity equal to or below the hysteresis intensity threshold, and a corresponding operation is performed in response to detecting a press input (e.g., an increase or decrease in contact intensity depending on the environment).
[0203] For ease of explanation, optionally, the description of an operation triggered in response to a press input associated with a press input strength threshold or in response to a gesture including a press input is provided in response to detecting any of the following conditions: the contact strength increases to above the press input strength threshold, the contact strength increases from below a hysteresis strength threshold to above the press input strength threshold, the contact strength decreases to below the press input strength threshold, and / or the contact strength decreases to below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in the example where the operation is described as being performed in response to detecting a decrease in contact strength below the press input strength threshold, the operation is optionally performed in response to detecting a decrease in contact strength below a hysteresis strength threshold corresponding to and less than the press input strength threshold.
[0204] Figure 5CAn exemplary diagram depicts a communication session between electronic devices 500A, 500B, and 500C. Devices 500A, 500B, and 500C are similar to electronic device 500, and each device shares one or more data connections 510 (such as an internet connection, Wi-Fi connection, cellular connection, short-range communication connection, and / or any other such data connection or network) to facilitate real-time communication of audio and / or video data between the respective devices for a sustained period of time. In some embodiments, the exemplary communication session may include a shared data session, whereby data is transmitted from one or more electronic devices to other electronic devices to enable simultaneous output of corresponding content at the electronic devices. In some embodiments, the exemplary communication session may include a video conferencing session, whereby audio and / or video data are transmitted between devices 500A, 500B, and 500C, enabling users of the respective devices to communicate in real time using the electronic devices.
[0205] exist Figure 5C In this design, device 500A represents an electronic device associated with user A. Device 500A (via data connection 510) communicates with devices 500B and 500C, which are associated with users B and C, respectively. Device 500A includes a camera 501A for capturing video data of the communication session, and a display 504A (e.g., a touchscreen) for displaying content associated with the communication session. Device 500A also includes other components such as a microphone (e.g., 113) for recording audio of the communication session and a speaker (e.g., 111) for outputting the audio of the communication session.
[0206] Device 500A displays a communication UI 520A via a display 504A. This communication UI is a user interface used to facilitate communication sessions (e.g., video conferencing sessions) between device 500B and device 500C. The communication UI 520A includes video feeds 525-1A and 525-2A. Video feed 525-1A is a representation of video data captured at device 500B (e.g., using camera 501B) and transmitted from device 500B to devices 500A and 500C during a communication session. Video feed 525-2A is a representation of video data captured at device 500C (e.g., using camera 501C) and transmitted from device 500C to devices 500A and 500B during a communication session.
[0207] The communication UI 520A includes a camera preview 550A, which is a representation of video data captured by camera 501A at device 500A. Camera preview 550A indicates to user A the expected video feed to be displayed at corresponding devices 500B and 500C.
[0208] The communication UI 520A includes one or more controls 555A for controlling one or more aspects of a communication session. For example, controls 555A may include controls for muting audio in the communication session, changing the camera view of the communication session (e.g., changing the camera used to capture video of the communication session, adjusting zoom values), terminating the communication session, applying visual effects to the camera view of the communication session, and activating one or more modes associated with the communication session. In some embodiments, one or more controls 555A are optionally displayed in the communication UI 520A. In some embodiments, one or more controls 555A are displayed separately from the camera preview 550A. In some embodiments, one or more controls 555A are displayed to cover at least a portion of the camera preview 550A.
[0209] exist Figure 5C In this context, device 500B represents an electronic device associated with user B, who communicates with devices 500A and 500C (via data connection 510). Device 500B includes a camera 501B for capturing video data of the communication session, and a display 504B (e.g., a touchscreen) for displaying content associated with the communication session. Device 500B also includes other components such as a microphone (e.g., 113) for recording audio of the communication session and a speaker (e.g., 111) for outputting the audio of the communication session.
[0210] Device 500B displays a communication UI 520B similar to the communication UI 520A of device 500A via touchscreen 504B. Communication UI 520B includes video feeds 525-1B and 525-2B. Video feed 525-1B is a representation of video data captured at device 500A (e.g., using camera 501A) and transmitted from device 500A to devices 500B and 500C during a communication session. Video feed 525-2B is a representation of video data captured at device 500C (e.g., using camera 501C) and transmitted from device 500C to devices 500A and 500B during a communication session. Communication UI 520B also includes: a camera preview 550B, which is a representation of video data captured at device 500B via camera 501B; and one or more controls 555B similar to controls 555A, which are used to control one or more aspects of the communication session. Camera preview 550B indicates to user B the expected video feed that user B will see displayed at the corresponding devices 500A and 500C.
[0211] exist Figure 5CIn this context, device 500C represents an electronic device associated with user C, who communicates with devices 500A and 500B (via data connection 510). Device 500C includes a camera 501C for capturing video data of the communication session, and a display 504C (e.g., a touchscreen) for displaying content associated with the communication session. Device 500C also includes other components such as a microphone (e.g., 113) for recording audio of the communication session and a speaker (e.g., 111) for outputting the audio of the communication session.
[0212] Device 500C displays a communication UI 520C, similar to the communication UI 520A of device 500A and the communication UI 520B of device 500B, via a touchscreen 504C. The communication UI 520C includes video feeds 525-1C and 525-2C. Video feed 525-1C is a representation of video data captured at device 500B (e.g., using camera 501B) and transmitted from device 500B to devices 500A and 500C during a communication session. Video feed 525-2C is a representation of video data captured at device 500A (e.g., using camera 501A) and transmitted from device 500A to devices 500B and 500C during a communication session. The communication UI 520C also includes: a camera preview 550C, which is a representation of video data captured at device 500C via camera 501C; and one or more controls 555C similar to controls 555A and 555B, which are used to control one or more aspects of the communication session. The camera preview 550C indicates to user C the expected video feed to be displayed at the corresponding devices 500A and 500B.
[0213] Although Figure 5C The diagram depicts a communication session between three electronic devices, but such a session can be established between two or more electronic devices, and the number of devices participating in the session can change as electronic devices join or leave. For example, if one electronic device leaves the communication session, audio and video data from the device that has stopped participating are no longer represented on the participating devices. For instance, if device 500B stops participating in the communication session, there is no data connection 510 between devices 500A and 500C, and there is no data connection 510 between devices 500C and 500B. Furthermore, device 500A does not include video feed 525-1A, and device 500C does not include video feed 525-1C. Similarly, if a device joins the communication session, a connection is established between the joining device and the existing devices, and video and audio data are shared among all devices, enabling each device to output data transmitted from other devices.
[0214] Figure 5C The implementation scheme depicted in the diagram represents a communication session between multiple electronic devices, including Figures 6A to 6V and Figures 9A to 9H The exemplary communication session is depicted in the image. In some implementations, Figures 6A to 6V and Figures 9A to 9H The communication session depicted in the figure includes two or more electronic devices, even if other electronic devices involved in the communication session are not depicted in the figure.
[0215] Now let’s turn our attention to the implementation of user interfaces (“UIs”) and associated processes on electronic devices (such as portable multifunction devices 100, 300 or 500).
[0216] Figures 6A to 6V Exemplary user interfaces for real-time communication sessions (e.g., audio communication sessions and / or video communication sessions) according to some implementation schemes are shown. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 7A to 7H and Figures 8A to 8G The process in.
[0217] Figure 6A A first user interface 604 of a real-time communication application (e.g., "RTCApp") of electronic device 600 is shown via display 602. The first user interface 604 includes a list and / or schedule of real-time communication sessions that the user of the electronic device has participated in (e.g., previously or currently), has been invited to participate in (e.g., the user was invited to participate but has not yet joined a real-time communication session), and / or has been invited to participate in (e.g., the real-time communication session is in progress and / or scheduled at a time later than the current time). Figure 6A In this configuration, the first user interface 604 arranges user interface objects 606a to 606g into regions 604a, 604b, and 604c of the user interface 604 corresponding to different time periods. The first region 604a includes user interface objects 606a to 606c, which correspond to real-time communication sessions that occurred and / or were scheduled on the current day (e.g., today). The second region 604b includes user interface objects 604d to 604f, which correspond to real-time communication sessions that occurred and / or were scheduled during the current week, not the current day. The third region 604c includes user interface object 606g, which corresponds to real-time communication sessions that occurred and / or were scheduled during the previous week (e.g., the week before the current week). Therefore, the user of the electronic device 600 can quickly view and / or find real-time communication sessions based on when they occurred, were scheduled, and / or were scheduled.
[0218] In some implementations, the first user interface 604 includes a user interface object corresponding to a live communication session that the user of the electronic device 600 was invited to but did not participate in. For example, the electronic device 600 receives an invitation and / or request to join a live communication session (e.g., a call, text, link, and / or calendar invitation), but the electronic device 600 does not detect user input corresponding to accepting the invitation and / or a request to initiate a live communication session (e.g., the user rejects the call, the user does not answer the call, the user does not select a link and / or another user interface object to initiate a live communication session).
[0219] exist Figure 6A At this location, the first user interface 604 also includes a link-sharing user interface object 608 and an add-participant user interface object 610. As described below, the electronic device 600 initiates a process for sharing a link to a live communication session in response to detecting user input corresponding to a selection of the link-sharing user interface object 608. In some embodiments, the process for sharing a link to a live communication session includes, for example, creating a link to the live communication session and / or displaying a user interface that enables the electronic device 600 to share the link. Furthermore, the electronic device 600 displays a second user interface 612 that allows the user to add an invitee to the live communication session in response to detecting a selection of the add-participant user interface object 610. Figure 6A At this point, electronic device 600 detects user input 650a corresponding to the selection of the user interface object 610 for adding a participant. In response to detecting user input 650a, electronic device 600 displays a second user interface 612, such as... Figure 6B As shown. Alternatively, in Figure 6A At this point, electronic device 600 detects user input 650b corresponding to the selection of a linked shared user interface object 608. In response to detecting user input 650b, electronic device 600 displays shared user interface 626, such as... Figure 6I As shown.
[0220] exist Figure 6B At this location, the second user interface 612 includes an invitee area 612a, a suggested invitee area 612b, an audio user interface object 612c, a real-time communication user interface object 612d, and a keyboard 612e (e.g., a graphical and / or virtual keyboard). Figure 6BIn the current state, the invitee area 612a is blank, indicating that the electronic device 600 has not yet detected user input corresponding to the selection and / or identification of an invitee for the live communication session. The electronic device 600 displays an audio user interface object 612c and a live communication user interface object 612d with an inactive appearance (such as reduced brightness, grayscale color scheme, and / or reduced size). For example, because the electronic device 600 has not detected user input corresponding to the selection of a contactable user to invite to the live communication session, the electronic device 600 displays an audio user interface object 612c and a live communication user interface object 612d with an inactive appearance to indicate that the user cannot initiate a live communication session without selecting an invitee.
[0221] It is suggested that the invitee area 612b include contact user interface objects 614a to 614h corresponding to contactable users stored in or associated with electronic device 600. Contact user interface objects 614a to 614h may include information such as telephone numbers, email addresses, and / or usernames associated with identifiers of entities (such as people, groups of people, and / or businesses). In some embodiments, it is suggested that the invitee area 612b display contact user interface objects 614a to 614h corresponding to users who are most frequently contacted (e.g., sending messages, making calls, and / or otherwise interacting via electronic device 600), most recently contacted, and / or most likely to be contacted, based on the usage patterns of electronic device 600. Usage patterns may include users who are frequently contacted, for example, at specific times of day, specific times of week, or specific times of month. Figure 6B At this point, the second user interface 612 includes eight contact user interface objects 614a to 614h. In some embodiments, the second user interface 612 includes more or fewer than eight contact user interface objects based on the number of contactable users stored by the electronic device 600 and / or based on the number of suggested contactable users determined by the electronic device 600.
[0222] exist Figure 6B At this point, electronic device 600 detects user input 650c that selects key 612f (e.g., the letter "l") on keyboard 612e. After detecting user input 650c, electronic device 600 displays a list of contactable users 618, such as... Figure 6C As shown.
[0223] exist Figure 6CAt this point, electronic device 600 has detected additional user input (e.g., user input corresponding to the "a" and "u" keys on keyboard 612e) and displays a list of contactable users 618 based on user input 650c and the additional user input. Electronic device 600 displays the list of contactable users 618 as an overlay on a second user interface 612, and specifically as an overlay over the suggested invitee area 612b, the audio user interface object 612c, and the real-time communication user interface object 612d (e.g., displayed on top of it). The list of contactable users 618 includes contactable users stored by electronic device 600 based on user input 650c corresponding to input indicator 619 (e.g., the character "lau") and the additional user input. For example, electronic device 600 uses user input 650c and additional user input as search terms (e.g., search terms corresponding to alphanumeric and / or emoji characters received via user input) to perform a search of electronic device 600's database of contactable users (and / or the database of contactable users associated with accounts logged into electronic device 600). Therefore, the list of contactable users 618 includes contact user interface objects 618a to 618e corresponding to identifiers of contactable users stored in the database of contactable users who match and / or partially match the search terms.
[0224] In some implementations, electronic device 600 arranges the contact user interface objects 618a to 618e of the contactable user list 618 based on one or more criteria. For example, in some implementations, electronic device 600 arranges the contact user interface objects 618a to 618e based on a comparison of search terms with identifiers (e.g., names, usernames, email addresses, and / or other identifiers) of contactable users stored in a contactable user database. For example, a first contact user interface object representing a first contactable user that includes an identifier that exactly matches the search term is listed before (e.g., above) a second contact user interface object representing a second contactable user that includes an identifier that partially matches the search term.
[0225] In some implementations, electronic device 600 arranges contact user interface objects 618a to 618e based on a comparison between search terms and identifiers of contactable users, as well as based on information associated with the contactable users. In some such implementations, electronic device 600 determines a subset of contact user interface objects to be displayed based on a comparison between search terms and identifiers of contactable users stored in a database of contactable users. Furthermore, electronic device 600 arranges the subset of contact user interface objects based on information associated with the corresponding contactable users of the subset of contact user interface objects. For example, in some implementations, the information associated with the corresponding contactable user includes a type of communication address, such as available communication patterns associated with a telephone number, email address, and / or account username. In some such implementations, contact user interface objects associated with contactable users having a first type of communication address (such as a telephone number and / or a messaging / chat application account) are listed before (e.g., above) contact user interface objects associated with contactable users having a second type of communication address (such as an email address and / or a non-messaging / chat application account) that is different from the first type of communication address. In some embodiments, the information associated with a corresponding contactable user includes the operating system of the device associated with the communication address, the applications installed on the device associated with the communication address, and / or the accounts checked in on the device associated with the communication address. In some embodiments, electronic device 600 displays (e.g., above) a contactable user associated with a device having a first operating system, a first application, and / or checked in to a first type of account before (e.g., above) a contactable user associated with a device having a first operating system, a first application, and / or checked in to a first type of account. In some embodiments, the information associated with a corresponding contactable user includes any combination of the type of communication address, the operating system of the device associated with the communication address, the applications installed on the device associated with the communication address, and / or the accounts checked in to the device associated with the communication address.
[0226] exist Figure 6CAt this point, the contact user interface object 618a corresponding to Lauren Jones is listed above the contact user interface object 618e corresponding to Bruno Lars, because the electronic device 600 determines that the information associated with Lauren Jones includes a phone number and the information associated with Bruno Lars includes an email address. In some embodiments, the electronic device 600 displays the contact user interface object associated with the phone number before the contact user interface object associated with the email address (and not the phone number), because the electronic device 600 can initiate communication with the phone number via a variety of different communication protocols (such as telephone calls and electronic messaging), while the electronic device 600 can typically initiate communication with the email address via one communication protocol. In some embodiments, even when the electronic device 600 detects one or more user inputs corresponding to the letters "lar" (instead of "lau"), the electronic device 600 displays the contact user interface object 618a before (e.g., above) the contact user interface object 618e, because the electronic device 600 has access to Lauren Jones' phone number but not Bruno Lars' phone number. Therefore, when the contactable user list 618 is based on a comparison between a search term and the identifiers of contactable users in the contactable user database, the contact list 618 arranges the contact user interface objects 618a to 618e based on the available communication modes and / or device types associated with the respective contactable users.
[0227] exist Figure 6C At this point, electronic device 600 detects user input 650d corresponding to the contact user interface object 618a (e.g., a tap gesture and / or user input selecting a contactable user as an invitee in a live communication session). In response to detecting user input 650d, electronic device 618a displays a second user interface 612, such as... Figure 6D As shown.
[0228] exist Figure 6DAt this location, electronic device 600 displays a first contactable user indicator 612g in the inviter area 612a of the second user interface 612. The first contactable user indicator 612g corresponds to the contactable user associated with the contact user interface object 618a, namely Lauren Jones. The first contactable user indicator 612g includes a first appearance (e.g., a bold appearance and / or text of the contact user interface object 620a with a first color, such as blue). In some embodiments, electronic device 600 displays the first contactable user indicator 612g with the first appearance when it determines that information associated with the contactable user represented by the first contactable user indicator 612g indicates that the device associated with the contactable user includes a real-time communication application (e.g., a specific real-time communication application associated with the first user interface 604 and / or the second user interface 612). In some implementations, when electronic device 600 determines that information associated with a contactable user represented by a first contactable user indicator 612g indicates that the device associated with the contactable user includes specific operating system software, electronic device 600 displays the first contactable user indicator 612g as having a first appearance.
[0229] exist Figure 6D At this point, electronic device 600 displays a suggested invitee area 612b, an audio user interface object 612c, and a real-time communication user interface object 612d. In response to detecting user input 650d, electronic device 600a displays the audio user interface object 612c and the real-time communication user interface object 612d with an active appearance (such as normal brightness, a non-grayscale color scheme, and / or an enlarged size). Thus, electronic device 600 indicates that a real-time communication session with Lauren Jones can be initiated because Lauren Jones has been added as an invitee. As explained below, electronic device 600 displays the audio user interface object 612c and the real-time communication user interface object 612d based on information determining that Lauren Jones is associated with meeting a first set of criteria. The first set of criteria includes the inclusion of a specific real-time communication application (e.g., RTCApp) on the device associated with Lauren Jones. For example, information accessible to electronic device 600 indicates that a real-time communication session with Lauren Jones can be initiated via a specific real-time communication application because the device associated with Lauren Jones has the same specific real-time communication application.
[0230] In some cases, a user can invite multiple contactable users to participate in a real-time communication session. Figure 6DAt this point, electronic device 600 detects user input 650e corresponding to a key 612h (e.g., the letter "i") on keyboard 612e. After detecting user input 650e, electronic device 600 displays a list of contactable users 620, such as... Figure 6E As shown.
[0231] exist Figure 6E At this point, electronic device 600 has detected additional user input (e.g., user input corresponding to the "v" and "y" keys on keyboard 612e) and displays a list of contactable users 620 based on user input 650e and the additional user input. Electronic device 600 displays the list of contactable users 620 as an overlay on a second user interface 612, and specifically as an overlay over the suggested invitee area 612b, the audio user interface object 612c, and the real-time communication user interface object 612d (e.g., displayed on top of it). The list of contactable users 620 includes contactable users stored by electronic device 600 based on user input 650e corresponding to input indicator 621 (e.g., the character "ivy") and the additional user input. For example, electronic device 600 uses user input 650e and the additional user input (e.g., displayed as input indicator 621) as search terms to perform a search of electronic device 600's database of contactable users. Therefore, the list of contactable users 620 includes contact user interface objects 620a to 620c corresponding to the identifiers of contactable users stored in the database of contactable users and those that match and / or partially match search terms.
[0232] As described above, the electronic device 600 arranges the contact user interface objects 620a to 620c of the contact list 620 based on information associated with contactable users. For example, in Figure 6E At this point, electronic device 600 displays contact user interface object 620a corresponding to Evan Ivor before (e.g., above) contact user interface object 620c corresponding to Ivy Smith, even though Ivy Smith includes an identifier that exactly matches the search term “ivy” (e.g., first name “Ivy”) and Evan Ivor (e.g., last name “Ivor”) includes an identifier that only partially matches the search term. Figure 6E In one implementation, the contact user interface object 620a is displayed before the contact user interface object 620c because the electronic device 600 has access to information indicating that the device associated with the contactable user Evan Ivor includes a specific real-time communication application (e.g., the real-time communication application associated with the first user interface 604 and the second user interface 612).
[0233] In addition, Figure 6EAt this location, electronic device 600 displays a contact user interface object 620a having a first appearance (e.g., a bold appearance and / or text of a contact user interface object 620a having a first color (such as blue)), and displays contact user interface objects 620b and 620c having a second appearance different from the first appearance (e.g., a non-bold appearance and / or text of contact user interface objects 620b and 620c having a second color (such as green). Therefore, electronic device 600 displays contact user interface objects 620a to 620c having the first and / or second appearances to indicate information about a specific contactable user represented by contact user interface objects 620a to 620c.
[0234] In some implementations, when electronic device 600 determines that information associated with a contactable user represented by a contact user interface object indicates that the device associated with the contactable user includes a real-time communication application (e.g., a specific real-time communication application associated with the first user interface 604 and the second user interface 612), electronic device 600 displays a contact user interface object with a first appearance (e.g., contact user interface object 620a). Similarly, when electronic device 600 determines that information associated with a contactable user represented by a contact user interface object indicates that the device associated with the contactable user does not include a real-time communication application (e.g., a specific real-time communication application associated with the first user interface 604 and the second user interface 612), electronic device 600 displays a contact user interface object with a second appearance (e.g., contact user interface objects 620b and 620c).
[0235] In some implementations, when electronic device 600 determines that information associated with a contactable user represented by a contact user interface object indicates that the device associated with the contactable user includes specific operating system software, electronic device 600 displays a contact user interface object having a first appearance (e.g., contact user interface object 620a). Similarly, when electronic device 600 determines that information associated with a contactable user represented by a contact user interface object indicates that the device associated with the contact does not include specific operating system software, electronic device 600 displays a contact user interface object having a second appearance (e.g., contact user interface objects 620b and 620c).
[0236] In some implementations, when electronic device 600 determines that information associated with a contactable user represented by a contact user interface object indicates that the contactable user has a specific account (e.g., an account associated with a specific instant messaging application) and / or is checked into a specific account, electronic device 600 displays a contact user interface object with a first appearance (e.g., contact user interface object 620a). Similarly, when electronic device 600 determines that information associated with a contactable user represented by a contact user interface object indicates that the contactable user does not have a specific account and / or is not checked into a specific account, electronic device 600 displays a contact user interface object with a second appearance (e.g., contact user interface objects 620b and 620c).
[0237] exist Figure 6E At this point, electronic device 600 detects user input 650f corresponding to the selection of a contact user interface object 620c. In response to detecting user input 650f, electronic device 600 displays a second user interface 612, such as... Figure 6F As shown.
[0238] exist Figure 6F At this location, electronic device 600 displays a second user interface 612 in invitee area 612a, having a second contactable user indicator 612i. The second contactable user indicator 612i corresponds to the contactable user associated with contact user interface object 620c, namely Ivy Smith. Figure 6F At this point, the electronic device 600 displays an appearance different from the first appearance (e.g., as shown in the image). Figure 6D The electronic device 600 displays a first contactable user indicator 612g with a second appearance (e.g., a non-bold appearance and / or text of a contact user interface object 620a with a first color, such as green), and displays a second contactable user indicator 612i with the second appearance. In some embodiments, when the electronic device 600 determines that information associated with at least one contactable user represented by the displayed contactable user indicator indicates that the device associated with that contactable user does not include a real-time communication application (e.g., a specific real-time communication application associated with the first user interface 604 and the second user interface 612), the electronic device 600 displays each contactable user indicator with the second appearance (e.g., instead of the first appearance). In some embodiments, when the electronic device 600 determines that information associated with at least one contactable user represented by the displayed contactable user indicator indicates that the device associated with that contactable user does not include specific operating system software, the electronic device 600 displays each contactable user indicator with the second appearance (e.g., instead of the first appearance).
[0239] exist Figure 6FAt this point, electronic device 600 displays (e.g., redisplays) the suggested invitee area 612b, but does not display the audio user interface object 612c and the live communication user interface object 612d. Instead, electronic device 600 displays the continue user interface object 612j. Electronic device 600 does not display the audio user interface object 612c and the live communication user interface object 612d because electronic device 600 determines that the information associated with the contactable user represented by the second contactable user indicator 612i indicates that the contactable user's device does not include the live communication application (and optionally, does not include the account (or specific account) associated with the live communication application). Even if electronic device 600 determines that the information associated with the contactable user indicated by the first contactable user indicator 612g indicates that the contactable user's device includes a live communication application, electronic device 600 does not display the audio user interface object 610c and the live communication user interface object 612d because the device of one of the contactable users included in inviter area 612a does not include a live communication application (e.g., electronic device 600 cannot initiate a live communication session with two invitees included in inviter area 612a because at least one invitee does not include a live communication application).
[0240] For example, the device associated with Ivy Smith does not include a real-time communication application, and therefore electronic device 600 cannot initiate real-time communication with Ivy Smith via the real-time communication application (e.g., electronic device 600 cannot send and / or transmit data via the real-time communication application that causes the device associated with Ivy Smith to output a notification). Instead, electronic device 600 displays a continue user interface object 610j, which, when selected via user input, causes electronic device 600 to generate and / or receive a link to the real-time communication session. The link to the real-time communication session allows IvySmith to join the real-time communication session without using a device that includes a real-time communication application (e.g., IvySmith can join the real-time communication session via a web browser on a device that does not include a real-time communication application).
[0241] exist Figure 6F At this point, electronic device 600 detects user input 650g corresponding to the selection of the continue user interface object 612j. In response to detecting user input 650g, electronic device 600 displays message transmission user interface 622, such as... Figure 6G As shown. In response to detecting user input 650g, electronic device 600 generates and / or receives a link to a real-time communication session, which can be used by a device without a real-time communication application to join the real-time communication session.
[0242] exist Figure 6FThe message sending user interface 622 includes a recipient area 622a, a message area 622b, a composing area 622c, a sending user interface object 622d, and a keyboard 622e. Figure 6F At this point, the electronic device 600 displays the messaging user interface 622 as an overlay on the second user interface 612. In some embodiments, the electronic device 600 launches a messaging application associated with the messaging user interface 622 and replaces the display of the second user interface 612 with the messaging user interface 622.
[0243] In response to the detection of user input 650g, electronic device 600 automatically populates recipient area 622a using a first contactable user indicator 622f corresponding to a first contactable user indicator 612g (e.g., Lauren Jones) and a second contactable user indicator 622g corresponding to a second contactable user indicator 622i (e.g., Ivy Smith). For example, the user of electronic device 600 does not need to provide additional user input to add recipients who will receive messages sent via messaging user interface 622. Furthermore, electronic device 600 automatically populates writing area 622c using linked user interface object 624a and draft message 624b. Writing area 622c of messaging user interface 622 displays character and / or graphical user interface objects corresponding to the detected user input but not yet sent via electronic device 600 to one or more recipient devices. For example, writing area 622c displays a draft message that can ultimately be sent by electronic device 600 to the recipient in response to the detection of user input corresponding to sending user interface object 622d. Therefore, electronic device 600 automatically populates the writing area 622c using the linked user interface object 624a and the draft message 624b, allowing the user to review the recipient area 622a, review the writing area 622c, and / or edit the draft message 624b (e.g., via one or more user inputs corresponding to the keyboard 622e) before causing electronic device 600 to send a message. In some embodiments, the linked user interface object 624a is a hyperlink to a unique Uniform Resource Locator (URI) used for a real-time communication session. The linked user interface object 624a enables a user who selects the linked user interface object 624a to participate in a real-time communication session with other users who have access to the URI associated with the linked user interface object 624a.
[0244] exist Figure 6FAt this point, message area 622b is blank, indicating that a message has not yet been sent from electronic device 600 to the first electronic device (different from electronic device 600) associated with Lauren Jones and / or the second electronic device (different from electronic device 600) associated with Ivy Smith. In some embodiments, electronic device 600 displays messages previously sent and / or received between electronic device 600, the first electronic device, and the second electronic device in message area 622b.
[0245] exist Figure 6F At this point, electronic device 600 detects and sends user input 650h corresponding to user interface object 622d. In response to detecting user input 650h, electronic device 600 initiates a process for sending a message including linked user interface object 624a and draft message 624b to the recipient corresponding to the first contactable user indicator 622f and the second contactable user indicator 622g (e.g., Lauren Jones and Ivy Smith). Furthermore, in response to detecting user input 650h, electronic device 600 displays the message including linked user interface object 624a and draft message 624b in message area 622b. As described below, after detecting user input 650h, electronic device 600 displays user interface object 606h on the first user interface 604, as shown below. Figure 6L As shown.
[0246] In some implementations, in response to detecting user input 650f (e.g. Figure 6E As shown), the electronic device 600 displays a second user interface 612, which includes a shared user interface object 612k and a message user interface object 612l, but does not include a continue user interface object 612j (as shown). Figure 6F (As shown). For example, in Figure 6H At this location, the electronic device 600 displays a second contactable user indicator 612i in the inviter area 612a of the second user interface 612. The electronic device 600 displays a first contactable user indicator 612g and a second contactable user indicator 612i having a second appearance (e.g., a non-bold appearance and / or text of a contact user interface object 620a with a first color (such as green)). This second appearance indicates that at least one contactable user associated with the first contactable user indicator 612g and / or the second contactable user indicator 612i is associated with a device that does not include a real-time communication application.
[0247] Furthermore, electronic device 600 does not display audio user interface object 612c and real-time communication user interface object 612d. Instead, electronic device 600 displays shared user interface object 612k and messaging user interface object 612l (e.g., in the same area and / or the same location as audio user interface object 612c and real-time communication user interface object 612c of the second user interface 612). Electronic device 600 does not display audio user interface object 612c and real-time communication user interface object 612d because electronic device 600 determines that the information associated with the contactable user represented by the second contactable user indicator 612i indicates that the contactable user's device does not include the real-time communication application (and optionally, does not include the account associated with the real-time communication application). Even if electronic device 600 determines that the information associated with the contactable user represented by the first contactable user indicator 612g indicates that the contactable user's device includes the real-time communication application, electronic device 600 does not display audio user interface object 610c and real-time communication user interface object 612d because the device of one of the contactable users included in inviter area 612a does not include the real-time communication application.
[0248] exist Figure 6H At this point, electronic device 600 detects user input 650i corresponding to the selection of message user interface object 612l. In response to detecting user input 650i, electronic device 600 displays message transmission user interface 622, such as... Figure 6G As shown. Alternatively, electronic device 600 detects user input 650j corresponding to the selection of shared user interface object 612k. In response to detecting user input 650j, electronic device 600 displays shared user interface 626, as shown. Figure 6I As shown. Alternatively, as described above, electronic device 600 displays shared user interface 626 in response to detecting user input 650b corresponding to selection of linked shared user interface object 608, as... Figure 6A As shown.
[0249] exist Figure 6IAt this location, the shared user interface 626 includes a first sharing area 626a providing one or more sharing options for a first group of invitees and a second sharing area 626b providing one or more sharing options for a second group of invitees different from the first group. For example, the first sharing area 626a includes different sharing options for sharing a link to a live communications session with invitees (e.g., selected users represented by a first contactable user indicator 612g and / or a second contactable user indicator 612i of invitee area 612a) associated with devices that do not include a live communications application and / or do not include specific operating system software. The second sharing area 626b includes one or more sharing options (e.g., one or more sharing options that may be selected in response to detecting user input corresponding to a third shared user interface object 630) for sharing a link to a live communications session with invitees (or, in some embodiments, sharing invitations and / or notifications associated with a link to a live communications session), associated with devices that include a live communications application and / or include specific operating system software.
[0250] exist Figure 6I At this location, the first shared area 626a includes a second shared user interface object 628a, a copied user interface object 628b, and a second message sending user interface object 628c. Figure 6I At this point, electronic device 600 detects user input 650k selecting the second shared user interface object 628a. In response to detecting user input 650k, electronic device 600 displays an indicator 629 near and / or above the second shared user interface object 628a, indicating that the second shared user interface object has been selected. When the second shared user interface object 628a is selected and electronic device 600 detects user input corresponding to the continued user interface object 631, electronic device 600 displays the second shared user interface 632, as shown. Figure 6J As shown.
[0251] When the copy user interface object 628b is selected and the electronic device 600 detects user input corresponding to the continue user interface object 631, the electronic device 600 copies and / or stores (e.g., temporarily stores) the data associated with the link to the live communication session. After the electronic device 600 copies the data associated with the link to the live communication session, the electronic device 600 may paste the link to the live communication session in response to the detection of further user input (e.g., touch and hold gesture). For example, the electronic device 600 may paste the link to the live communication session by displaying text and / or user interface objects in a writing area for drafting messages. When the second messaging user interface object 628c is selected and the electronic device 600 detects user input corresponding to the continue user interface object 631, the electronic device 600 displays the messaging user interface 622, such as... Figure 6G As shown.
[0252] The second shared area 626b includes a third shared user interface object 630 that indicates sharing options for sending invitations (e.g., notifications and / or links to live messaging sessions) to a second group of invitees instead of the first group. The second group of invitees includes contactable users associated with devices including a live messaging application and / or devices including specific operating system software that enables the device to receive invitations to live messaging sessions, which do not necessarily include the link (e.g., a Uniform Resource Locator). For example, in response to detecting user input corresponding to the third shared user interface object 630, the electronic device 600 displays sharing options for sending invitations to the second group of invitees, which may include sharing options different from those available via the first shared area 626a and / or the second shared user interface 632. In some embodiments, the electronic device 600 shares the invitation to the second group of invitees via a notification from the live messaging application (e.g., the invitee's device outputs a notification associated with the live messaging application). In some embodiments, the electronic device 600 shares the invitation to the second group of invitees by calling the device associated with the second group of invitees. For example, an electronic device can call devices associated with a second group of invitees by communicating with the invitee devices over a network, causing the invitee devices to ring and / or output notifications related to the communication from the electronic device 600. In some implementations, the electronic device 600 uses the same sharing options (e.g., communication protocols) as the first group of invitees to share the invitation with the second group of invitees.
[0253] exist Figure 6IAt this point, the electronic device detects user input 650l corresponding to the selection of the continued user interface object 631. In response to detecting user input 650l and determining that the second shared user interface object 628a has been selected (e.g., via user input 650k), the electronic device 600 displays the second shared user interface 632, as shown. Figure 6J As shown.
[0254] exist Figure 6J At this point, the electronic device 600 displays the second shared user interface 632 as an overlay on the shared user interface 626. In some embodiments, the electronic device 600 replaces the display of the shared user interface 626 with the display of the second shared user interface 632. Figure 6J At this location, the second sharing user interface 632 includes sharing option user interface objects 632a to 632e and a copy user interface object 632f. The sharing option user interface objects 632a to 632e correspond to different sharing technologies, applications, and / or communication modes for sending a link to a real-time communication session to a first group of invitees (and optionally, a second group of invitees). The first sharing option user interface object 632a corresponds to an option for sharing the link to the real-time communication session via direct wireless communication (e.g., short-range communication connections such as Bluetooth, Wi-Fi, and / or Zigbee, which send the link directly from electronic device 600 to one or more invitee devices without an intermediary device). For example, the first option user interface object 632a enables the link to the real-time communication session to be shared with an external device (e.g., an invitee's external device) via a direct wireless communication protocol. The second sharing option user interface object 632b corresponds to a first messaging application (e.g., a first chat and / or text messaging application) configured to share the link to the real-time communication session via electronic messaging (e.g., asynchronous messaging, such as text messaging). In some implementations, the first messaging application is the messaging application associated with the messaging user interface 622. The third sharing option user interface object 632c corresponds to a second messaging application (e.g., a second chat and / or text messaging application) different from the first messaging application, which is configured to share a link to the real-time communication session via electronic messaging (e.g., asynchronous messaging, such as text messages and / or chat messages). The fourth sharing option user interface object 632d corresponds to an email application configured to share a link to the real-time communication session via email.
[0255] exist Figure 6JAt this point, electronic device 600 partially displays a fifth sharing option user interface object 632e. In response to detecting user input (e.g., a left swipe gesture) on the second sharing user interface 632, electronic device 600 pans sharing option user interface objects 632a to 632e to display (e.g., fully display) the fifth sharing option user interface object 632e and / or display one or more additional sharing option user interface objects. In some embodiments, the fifth sharing option user interface object 632e corresponds to another application that enables electronic device 600 to share a link to a real-time communication session via electronic messaging, short-range communication, email, notifications, and / or another suitable communication protocol.
[0256] exist Figure 6J At this point, electronic device 600 detects user input 650m corresponding to the selection of the third shared option user interface object 632c. In response to detecting user input 650m, electronic device 600 displays the chat user interface 634 of the second messaging application, such as... Figure 6K As shown.
[0257] exist Figure 6K At this location, the chat user interface 634 includes a draft message 634a, a link 634b, and a send user interface object 634c. The draft message 634a is an automatically generated (e.g., generic) message indicating that the link 634b corresponds to a real-time communication session. The electronic device 600 can change the content of the draft message 634b in response to detecting one or more user inputs directed to the keyboard 634d of the chat user interface 634. For example, a user of the electronic device 600 can draft a personalized message to send along with the link 634b. The link 634b is a unique Uniform Resource Locator corresponding to a virtual room used for the real-time communication session. Figure 6K At this point, electronic device 600 detects and sends user input 650n corresponding to user interface object 634c. In response to detecting user input 650n, electronic device 600 initiates a process for sending a message including draft message 634a and link 634b via a second messaging application.
[0258] via a second messaging application (such as Figure 6K (as shown) after sending a message and / or via a messaging application (such as...) Figure 6G After sending the message (as shown), the electronic device 600 displays the first user interface 604, such as... Figure 6L As shown. In some implementations, in response to detecting user input 650h (e.g. Figure 6G (as shown) and / or in response to detecting user input 650n (e.g.) Figure 6K As shown), the electronic device 600 displays a real-time communication user interface 636, such as... Figure 6M As shown. Therefore, in such embodiments, electronic device 600 initiates a real-time communication session in response to the detection that a message including a link to a real-time communication session (e.g., a link to user interface object 624a and / or link 634b) has been sent (e.g., via electronic device 600).
[0259] exist Figure 6L At the same time, electronic device 600 displays a first user interface 604 immediately after sending a link to the real-time communication session. Electronic device 600 displays a first user interface 604 having a user interface object 606h corresponding to the sent link to the real-time communication session (e.g., link user interface object 624a and / or link 634b). Furthermore, electronic device 600 displays an area 604d on the first user interface 604 corresponding to the current time (e.g., 10:09). Figure 6L As shown, region 604d is displayed before (e.g., above) regions 604a to 604c, indicating that user interface object 606h corresponds to the real-time communication session closest to the present time and / or a real-time communication session is in progress. As described below, electronic device 600 may change and / or adjust the appearance of user interface object 606h (and optionally, other user interface objects corresponding to the ongoing real-time communication session) in response to the detection of one or more events.
[0260] exist Figure 6L At this point, electronic device 600 detects user input 650o corresponding to the selection of user interface object 606h. In response to detecting user input 650o, electronic device 600 displays real-time communication user interface 636, such as... Figure 6M As shown. In some embodiments, electronic device 600 displays real-time communication user interface 636 in response to detecting user input 650o corresponding to selection of user interface object 606h (e.g., when displaying first user interface 604), and in some embodiments, electronic device 600 displays real-time communication user interface 636 in response to detecting user input 650h and / or user input 650n corresponding to a request for sending a message including a link to a real-time communication session.
[0261] exist Figure 6MAt this location, the real-time communication user interface 636 includes a first participant indicator 638 corresponding to a first invitee of the real-time communication session (e.g., the user associated with contactable user indicator 612g; Lauren Jones), a second participant indicator 640 corresponding to a second invitee of the real-time communication session (e.g., the user associated with contactable user indicator 612i; Ivy Smith), and a user indicator 642 corresponding to the field of view of the camera of the electronic device 600 (which is associated with the user of the electronic device 600). Figure 6M At the same time, the first participant indicator 638 includes a visual indicator 638a (e.g., an avatar, photo, and / or another visual representation) and an identifier 638b (e.g., name, nickname, username, and / or email address) for the first invitee. Similarly, the second participant indicator 640 includes a visual indicator 640a (e.g., an avatar, photo, and / or another visual representation) and an identifier 640b (e.g., name, nickname, username, and / or email address) for the second invitee. Since the first and second invitees have not yet joined the live communication session, the electronic device 600 displays visual indicators 638a and 640a respectively. As described below, when the first and / or second invitees join the live communication session, the electronic device 600 does not display visual indicators 638a and / or 640a respectively (or removes their display), but instead displays live images of the first and / or second invitees (e.g., snapshots and / or video feeds).
[0262] Furthermore, the first participant indicator 638 includes a messaging user interface object 638c, and the second participant indicator 640 includes a messaging user interface object 640c. In response to detecting user input corresponding to the messaging user interface object 638c and / or the messaging user interface object 640c, the electronic device 600 displays a messaging user interface that enables the electronic device 600 to send electronic messages (e.g., text messages) to a first invitee (e.g., a device associated with the first invitee) and / or a second invitee (e.g., a device associated with the second invitee).
[0263] Furthermore, in Figure 6MAt this point, the first participant indicator 638 includes a call user interface object 638d (e.g., "ringing"), and the second participant indicator 640 includes a status indicator 640d (e.g., "invitation sent"). As explained above, the first invitee (e.g., Lauren Jones) is a user associated with a device having a real-time communication application and / or a device having specific operating system software. Therefore, the electronic device 600 displays the call user interface object 638d, which, when selected via user input, causes the electronic device 600 to call the first invitee's device (e.g., initiate real-time communication with the first invitee's device). Since the second invitee (e.g., Ivy Smith) includes a user associated with a device that does not have a real-time communication application and / or does not include specific operating system software, the electronic device 600 cannot call the second invitee's device via the real-time communication application. Therefore, electronic device 600 displays status indicator 640d, which indicates that electronic device 600 has sent a link to a real-time communication session to the second invitee (e.g., the second invitee's device) (and / or initiated a process for sending the link), without enabling electronic device 600 to call the second invitee via the real-time communication application.
[0264] exist Figure 6N At that point, electronic device 600 displays first user interface 604 a second time after the first time following the connection sent to the real-time communication session. Figure 6N In response to detecting that a first invitee (e.g., Lauren Jones) and a second invitee (e.g., Ivy Smith) have joined a live communication session, electronic device 600 displays a user interface object 606h with an updated appearance. For example, user interface object 606h includes a participant identifier 642a (e.g., a textual indication of a participant who has joined the live communication session) and a participant icon 642b (e.g., a visual indication of a participant who has joined the live communication session and / or the number of participants). In response to detecting that one or more invitees have joined the live communication session, electronic device 600 displays user interface object 606h, which has an identifier for the joined invitee via participant identifier 642a and a visual indication of the identity and / or number of joined participants via participant icon 642b. Therefore, the user of electronic device 600 can quickly and easily (e.g., without additional input) identify the participants currently participating in the live communication session without having to join the live communication session.
[0265] exist Figure 6NAt this point, electronic device 600 detects user input 650p corresponding to the selection of user interface object 606h. In response to detecting user input 650p, electronic device 600 displays real-time communication user interface 636, such as... Figure 6O As shown.
[0266] Since some invitees have already joined the live communication session, electronic device 600 displays a live communication user interface 636 with a first participant representation 644 and a second participant representation 646. The first participant representation 644 includes a live image and / or video feed captured via a device associated with the first invitee (e.g., Lauren Jones), and the first participant identifier 644a of the first invitee (e.g., name, nickname, username, and / or email address). Similarly, the second participant representation 646 includes a live image and / or video feed captured via a device associated with the second invitee (e.g., Ivy Smith), and the second participant identifier 646a of the second invitee (e.g., name, nickname, username, and / or email address).
[0267] As described above, electronic device 600 determines that the second invitee and / or participant (e.g., Ivy Smith) is not associated with a device that includes an instant messaging application and / or specific operating system software. Therefore, electronic device 600 displays a remove user interface object 648 on the second participant representation 646. In response to detecting user input corresponding to the remove user interface object 648, electronic device 600 causes the second invitee and / or participant to be removed from the instant messaging session (e.g., electronic device 600 causes the device of the second invitee and / or the second participant to be disconnected from the instant messaging session). Furthermore, electronic device 600 determines that the first invitee and / or participant (e.g., Lauren Jones) is associated with a device that includes an instant messaging application and / or specific operating system software. Therefore, electronic device 600 does not display the remove user interface object 648 on the first participant representation 644. In some embodiments, electronic device 600 does not display the remove user interface object 648 for any participant in the instant messaging session. In some embodiments, electronic device 600 displays the remove user interface object 648 for all participants in the instant messaging session. References below... Figures 9A to 9H and Figure 10 Further description of additional technologies related to the ability to remove users from a real-time communication session and / or allow users to participate in a real-time communication session.
[0268] In some cases, users may wish to refer to the above. Figures 6A to 6K The different methods described in the real-time communication application are used to send links to the real-time communication session. Figures 6P to 6RAn implementation scheme is shown in which an electronic device 600 generates and / or sends a link to a real-time communication session via a messaging application. Furthermore, Figures 6R to 6V An implementation of an electronic device 600 generating a link to a real-time communication session via a calendar application is shown.
[0269] exist Figure 6P At this location, electronic device 600 displays a messaging user interface 652 (e.g., a messaging user interface identical to and / or different from messaging user interface 622). Messaging user interface 652 includes a recipient indicator 652a, a message area 652b, a composing area 652c, a suggested content area 652d, and a keyboard 652e. Figure 6P At this location, the recipient indicator 652a indicates that a message sent via the messaging application will be sent to a single recipient (e.g., Gina). In some embodiments, the recipient indicator 652a may include more than one recipient. Furthermore, the message area 652b includes messages previously sent and received between the recipient and the user of the electronic device 600.
[0270] exist Figure 6P At this location, electronic device 600 detects one or more user inputs corresponding to keyboard 652e and displays the content of a draft message (e.g., "Tuesday evening video chat") in writing area 652c. In response to detecting one or more user inputs, electronic device 600 compares the content with one or more predefined characters indicating the user's intent to schedule and / or initiate a real-time communication session. For example, in Figure 6PIn the draft message, the content includes the word "video chat." Electronic device 600 determines that the word "video chat" matches one or more predefined characters and displays a real-time communication user interface object 654 in the suggested content area 652d. The real-time communication user interface object 654 corresponds to a link to a real-time communication session generated by and / or received via electronic device 600. Furthermore, the real-time communication user interface object 654 corresponds to a link to a real-time communication session scheduled for a time corresponding to the content of the draft message in the writing area 652c. For example, the draft message content includes the word "Tuesday evening," indicating the user's intention to schedule a real-time communication session for an upcoming Tuesday evening. In some embodiments, electronic device 600 generates links to schedule the real-time communication session for a time on an upcoming Tuesday that corresponds to evening (e.g., evening based on the current time zone of electronic device 600), such as 5 PM, 6 PM, and / or 7 PM. Therefore, a user of electronic device 600 can type in a draft message that mentions communication and / or meetings at a specific time (e.g., via a live communication session), and electronic device 600 displays a live communication user interface object 654 that suggests the user schedule the live communication session at a time corresponding to the content of the draft message.
[0271] exist Figure 6P At this point, electronic device 600 detects user input 650q corresponding to the selection of real-time communication user interface object 654. In response to detecting user input 650q, electronic device 600 schedules a real-time communication session (e.g., adds and / or displays a user interface object corresponding to the scheduled real-time communication session on the first user interface 604), sends a message with a link corresponding to the real-time communication user interface object 654, and displays message 656 in the message area 652b of the message delivery user interface 652, such as... Figure 6Q As shown.
[0272] exist Figure 6Q In this context, message 656 includes a link 656a and a message portion 656b. Link 656a is a Uniform Resource Locator (URL) corresponding to a scheduled real-time communication session (e.g., an upcoming Tuesday evening). The recipient's device displays link 656a in its corresponding messaging application. In some implementations, the recipient may select link 656a before the scheduled time, but the real-time communication session will not be initiated until the scheduled time. Figure 6P In this context, message portion 656b of message 656 includes an automatically generated (e.g., default) message. In some implementations, in response to selecting a real-time communication user interface object 654, the electronic device 600 enables the user to change and / or modify the automatically generated message before sending message 656 to the recipient.
[0273] In some cases, the content of a draft message in area 652c corresponds to an invitation to a scheduled real-time communication session. For example, in Figure 6R At this location, electronic device 600 detects one or more user inputs corresponding to keyboard 652e and displays the content of a draft message (e.g., "Tuesday evening video call") in writing area 652c. In response to detecting one or more user inputs, electronic device 600 compares the content with a set of criteria (e.g., one or more predefined characters indicating the user's intent to schedule and / or initiate a real-time communication session). Figure 6R In the draft message, the content includes the word "video call". Electronic device 600 determines that the word "video call" matches one or more predefined characters and displays a first real-time communication user interface object 658 and a second real-time communication user interface object 660 in the suggested content area 652d.
[0274] The first real-time communication user interface object 658 corresponds to a link to a new real-time communication session generated by and / or received via the electronic device 600. Figure 6R At this point, electronic device 600 determines that the content of the draft message indicates a time (e.g., within 30 minutes, within 1 hour, and / or within 2 hours) in a previously scheduled live communication session. Therefore, electronic device 600 displays a second live communication user interface object 660 corresponding to the previously scheduled live communication session at and / or near the time indicated by the content of the draft message. Thus, the user of electronic device 600 can quickly and easily invite the recipient to a previously scheduled live communication session (e.g., via the second live communication user interface object 660) and / or schedule a new live communication session with the recipient (e.g., via the first live communication user interface object 658).
[0275] As mentioned above, Figures 6S to 6V An implementation scheme is shown in which an electronic device 600 generates a link to a real-time communication session via a calendar application. Figure 6S At this location, electronic device 600 displays a calendar user interface 662, including a calendar 662a and a new event user interface object 662b. Figure 6S At this point, electronic device 600 detects user input 650r corresponding to the selection of a new event user interface object 662b. In response to detecting user input 650r, electronic device 600 displays the scheduling user interface 664 of the calendar application, such as... Figure 6T As shown.
[0276] The scheduling user interface 664 of the calendar application enables the electronic device 600 to create new events (e.g., appointments, meetings, and / or another scheduled event), which the electronic device 600 adds to the calendar 662a of the calendar user interface 662 (e.g., a user interface object corresponding to the new event displayed on calendar 662a). The scheduling user interface 664 includes scheduling option user interface objects 664a to 664h, which, when selected via user input, enable the user of the electronic device 600 to specify and / or customize details associated with the new event, such as timing information associated with the new event, the location of the new event, invitees to the new event, and / or the communication mode of the new event (e.g., live, video conferencing, audio conferencing, and / or another communication mode).
[0277] exist Figure 6T At this point, electronic device 600 detects user input 650s corresponding to the selection of a real-time communication session user interface object 664b (e.g., for video conferencing, audio conferencing, or video and / or audio conferencing). In response to detecting user input 650s, electronic device 600 displays the real-time communication session user interface 666, as shown below. Figure 6U As shown.
[0278] exist Figure 6UAt this location, the real-time communication session user interface 666 includes real-time communication session option user interface objects 666a to 666f. The first real-time communication session option user interface object 666a corresponds to a non-real-time communication session option, which, when selected via user input, causes a new event to be designated as a non-real-time communication session (e.g., a live meeting and / or an appointment and / or an audio conference and / or an appointment). Furthermore, real-time communication session option user interface objects 666b to 666e correspond to real-time communication session applications that can be used to conduct and / or initiate real-time communication sessions for new events. In some embodiments, the real-time communication sessions of one or more of the option user interface objects 666b to 666e correspond to real-time communication session applications installed on the electronic device 600 (e.g., applications that have been downloaded and enabled for use on the electronic device 600). In some embodiments, one or more of the real-time communication session option user interface objects 666b to 666e correspond to real-time communication session applications installed on the electronic device 600 to which the user of the electronic device 600 has checked in (e.g., the user has logged into an account associated with the corresponding real-time communication session application). Real-time communication session user interface object 666f corresponds to customized link options for the real-time communication session. In response to detecting user input corresponding to real-time communication session user interface object 666f, electronic device 600 prompts the user to enter a link to the real-time communication session (e.g., link user interface objects 624a and / or link 634b). In some embodiments, the link to the real-time communication session corresponds to a real-time communication session application that is not represented by and / or associated with these real-time communication session user interface objects 666b to 666e.
[0279] exist Figure 6U At this point, electronic device 600 detects user input 650t corresponding to a selection of a real-time communication session option user interface object 666d (e.g., corresponding to a real-time communication session application called "Boom," which is different from the aforementioned real-time communication application "RTCApp"). In response to detecting user input 650t, electronic device 600 displays a real-time communication session details user interface 668, such as... Figure 6V As shown.
[0280] exist Figure 6V At this location, the real-time communication session details user interface 668 includes user interface objects 668a to 668c corresponding to various options and / or details of the real-time communication session, which is configured to be executed via a real-time communication session application (e.g., "Boom") corresponding to the real-time communication session options user interface object 666d. Figure 6VAt this location, user interface object 668a corresponds to a one-time room for a real-time communication session (e.g., video conferencing) associated with the new event. In response to detecting user input corresponding to user interface object 668a, electronic device 600 creates, schedules, and / or designates a new one-time virtual room for the real-time communication session associated with the new event. In some embodiments, the new one-time virtual room cannot be used for additional real-time communication sessions occurring after the real-time communication session associated with the new event.
[0281] exist Figure 6V At this location, user interface object 668b corresponds to a personal room for a real-time communication session (e.g., video conferencing) associated with the new event. In response to detecting user input corresponding to user interface object 668b, electronic device 600 schedules and / or assigns a virtual room associated with the user's account (e.g., an account logged into a real-time communication session application associated with the real-time communication session option user interface object 666d (e.g., "Boom")) for the real-time communication session associated with the new event. In some embodiments, the virtual room can be used for additional real-time communication sessions that occur after the real-time communication session associated with the new event.
[0282] In addition, Figure 6V At this location, user interface object 668c corresponds to a password for real-time communication associated with the new event. In response to detecting user input corresponding to user interface object 668c, electronic device 600 generates a password and / or enables the user to enter a password that restricts access to the real-time communication session associated with the new event. For example, when user interface object 668c is selected, an invitee and / or user attempting to join the real-time communication session will not be allowed to participate in the real-time communication session unless the invitee and / or user enters the correct password. In some embodiments, the invitation to the real-time communication session associated with the new event includes a password, allowing an invitee authorized by the user of electronic device 600 (e.g., selected via user input) to join the real-time communication session by entering the password. Furthermore, unauthorized invitees without a password and / or who enter an incorrect password are prevented from joining the real-time communication session.
[0283] Although Figure 6VA real-time communication session details user interface 668 with user interface objects 668a to 668c is shown, but in some embodiments, the real-time communication session details user interface 668 includes more than three or fewer user interface objects. In some embodiments, the real-time communication session details user interface 668 includes user interface objects specific to and / or otherwise corresponding to options unique to the respective real-time communication session application. Therefore, the real-time communication session details user interface 668 may include different user interface objects corresponding to different options and / or details of the real-time communication session, depending on which real-time communication session user interface object 666b to 666e is selected.
[0284] Figures 7A to 7H This is a flowchart illustrating a method for sharing a link to a real-time communication session using a computer system, according to some embodiments. Method 700 is performed at a computer system (e.g., 100, 300, 500, 600, 900, 902, 904, and / or 906) (e.g., an electronic device; a smart device, such as a smartphone or smartwatch; a mobile device; a wearable device), which communicates with a display generation component (e.g., 602) and one or more input devices (e.g., 602). Some operations in method 700 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.
[0285] As described below, method 700 provides an intuitive way to share a link to a real-time communication session. This method reduces the cognitive burden on users when sharing a link to a real-time communication session, thereby creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to share a link to a real-time communication session more quickly and efficiently, saving power and increasing the time interval between battery charges.
[0286] A computer system (e.g., 600) displays (702) via a display generation component (e.g., 603) a user interface (e.g., 604 and / or 612) of a real-time communication application (e.g., an application of a computer system that enables voice and / or video conferencing between a user associated with the computer system and one or more users associated with a corresponding external computer system) (e.g., the main screen user interface of the video communication application, a user interface displayed in response to user input corresponding to a request for selection and an external computer system (e.g., an external computer system without a specific video communication application installed), and / or a user interface displayed in response to a request to initiate a video communication session), the user interface including links to the real-time communication session (e.g., 624a, 634b, and / or 910). a) (e.g., such a link that, in response to user input corresponding to a selection of the link, launches a real-time communication application and / or a different application (e.g., an application associated with a web browser) that enables a user associated with an external computer system to participate in a real-time communication session with the user associated with the computer system) (e.g., a user-interactive graphical user interface element (e.g., 608, 612j, and / or 612k) corresponding to a link that initiates the creation of a real-time communication session between a user associated with the computer system and one or more users associated with a corresponding external computer system selected via a user interface and / or another user interface) (e.g., buttons and / or power indicators displayed in response to user input corresponding to a selection of the user associated with the external computer system).
[0287] The computer system (e.g., 600) receives (704) user input (e.g., 650b, 650g and / or 650j) corresponding to the selection of user interactive graphical user interface elements (e.g., 608, 612j and / or 612k) via one or more input devices (e.g., 602) (e.g., tap gesture).
[0288] In response to receiving user input, the computer system (e.g., 600) initiates (706) a process for sharing a link (e.g., 624a, 634b, and / or 910a) to a real-time communication session, wherein the process for sharing the link to the real-time communication session includes displaying, via a display generation component, a plurality of optional options (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e, and / or 632f) for sharing the link (e.g., 624a, 634b, and / or 910a) to the real-time communication session (e.g., corresponding to different sharing options, technologies, and / or applications for sharing the link). A shared user interface (e.g., 626 and / or 632) (e.g., a pop-up window overlaid on the user interface, a shared user interface partially overlaid on the user interface, and / or a shared user interface replacing the user interface) including simultaneously displaying: a first optional option (e.g., 632a, 632b, 632c, 632d, 632e, and / or 632f) for sharing a link (e.g., 624a, 634b, and / or 910a) to a real-time communication session via a first communication protocol (e.g., SMS service, email, short-range communication, direct wireless communication technology, file-sharing service, and / or social media application) (708). And a second optional option (e.g., 632a, 632b, 632c, 632d, 632e, and / or 632f) for sharing a link to a real-time communication session via a second communication protocol different from the first communication protocol (e.g., SMS service, email, short-range communication, direct wireless communication technology, file-sharing service, and / or social media application) (e.g., 624a, 634b, and / or 910a). In some embodiments, each of the plurality of optional options corresponds to a corresponding communication protocol and / or application (e.g., real-time communication application, messaging application, communication application, email application, and / or direct wireless communication technology, such as Bluetooth), which is configured to send a link (e.g., sending data associated with the link from the computer system to the external computer system, causing the external computer system to display the link) to a user associated with an external computer system different from the computer system (e.g., a user selected via one or more user inputs detected when the computer system displays a user interface).
[0289] Including a first optional option for sharing a link to a real-time communication session via a first communication protocol and a second optional option for sharing a link to a real-time communication session via a second communication protocol provides the user with multiple options for sharing a link to a real-time communication session and reduces the amount of input required for the user to share the link. For example, the user can select the option for sharing a link to a real-time communication session without having to navigate from the user interface of the real-time communication application to another application. Reducing the amount of input required to perform an operation enhances the operability of the device and makes the user device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling the user to use the device more quickly and efficiently.
[0290] In some implementations, the computer system (e.g., 600) detects (712) a second user input (e.g., 650m) corresponding to the selection of one of a plurality of optional options (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e and / or 632f) (e.g., 632c). (e.g., a tapping gesture, such as a click gesture). And in response to the detection of a second user input (714): based on determining that the second user input corresponds to a selection of a first optional option (e.g., 632a, 632b, 632c, 632d, 632e and / or 632f), the computer system (e.g., 600) initiates (716) a link (e.g., 624a, 634b and / or 910a) to be shared to a real-time communication session via a first communication protocol (e.g., short message service, email, short-range communication, direct wireless communication technology, file-sharing service and / or social media application) (e.g., sent, transmitted and / or transmitted via the first communication protocol). The process of sharing a link; and the process of a computer system (e.g., 600) initiating (718) a link (e.g., 624a, 634b and / or 910a) for sharing to a real-time communication session via a second communication protocol (e.g., SMS service, email, short-range communication, direct wireless communication technology, file-sharing service and / or social media application) (e.g., sending, transmitting and / or sharing the link via the second communication protocol) based on determining that the second user input corresponds to a selection of a second optional option (e.g., 632a, 632b, 632c, 632d, 632e and / or 632f).
[0291] The inclusion of a first optional option for sharing a link to a real-time communication session via a first communication protocol and a second optional option for sharing a link to a real-time communication session via a second communication protocol allows the user to select the preferred communication protocol for sharing the link to the real-time communication session and reduces the amount of input required for the user to select the communication protocol and / or share the link. For example, the user can select the option for sharing the link to the real-time communication session without having to navigate from the user interface of the real-time communication application to another application. Reducing the amount of input required to perform operations enhances the operability of the device and makes the user device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.
[0292] In some implementations, multiple alternatives (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e and / or 632f) correspond to multiple communication protocols (e.g., short message service, email, short-range communication, direct wireless communication technology, file sharing service and / or social media application) (in some implementations, each alternative corresponds to a corresponding communication protocol), which include email (e.g., 632d) (e.g., an application of a computer system that shares a link to a real-time communication session via an electronic message (e.g., email) over a network) and one or more real-time communication protocols (e.g., 632b and / or 632c) (e.g., an application of a computer system that shares a link to a real-time communication session via a short message service, instant messaging and / or messaging service and / or social media service).
[0293] Including multiple optional options corresponding to various communication protocols allows users to select a preferred communication protocol without having to navigate to another application associated with that preferred protocol, thereby reducing the amount of input required for users to share links. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0294] In some implementations, multiple alternatives (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e, and / or 632f) correspond to multiple communication protocols (e.g., short message service, email, short-range communication, direct wireless communication technologies, file-sharing services, and / or social media applications) (in some implementations, each alternative corresponds to a corresponding communication protocol). These communication protocols include short message service protocols (e.g., 632b and / or 632c) (e.g., applications of computer systems that share a link to a real-time communication session via a network in short messages) and one or more messaging protocols (e.g., 632b and / or 632c) (e.g., applications of computer systems that share a link to a real-time communication session via instant messaging, messaging services, and / or social media services that do not use short message service protocols). In some implementations, the short message service protocol corresponds to a first application installed on the computer system, and the one or more messaging protocols correspond to one or more second applications installed on the computer system.
[0295] Including multiple optional options corresponding to various communication protocols allows users to select a preferred communication protocol without having to navigate to another application associated with that preferred protocol, thereby reducing the amount of input required for users to share links. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0296] In some implementations, multiple alternatives (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e and / or 632f) correspond to multiple communication protocols (e.g., short message service, email, short-range communication, direct wireless communication technology, file sharing service and / or social media application) (in some implementations, each alternative corresponds to a corresponding communication protocol), these communication protocols include a first messaging communication protocol (e.g., 632b) (e.g., a first application of a computer system shared via a link to a real-time communication session through a short message service, instant messaging and / or messaging service and / or social media service) and a second messaging communication protocol (e.g., 632c) (e.g., a second application of a computer system, different from the first application, shared via a link to a real-time communication session through a short message service, instant messaging and / or messaging service and / or social media service).
[0297] Including multiple optional options corresponding to various communication protocols allows users to select a preferred communication protocol without having to navigate to another application associated with that preferred protocol, thereby reducing the amount of input required for users to share links. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0298] In some implementations, the process of initiating a link for sharing to a real-time communication session (e.g., 624a and / or 634b) includes: displaying one or more user-interactive graphical user interface objects (e.g., 614a to 614h) via a display generation component (e.g., 602) (e.g., visual indications of users and / or contacts whose telephone numbers, email addresses, usernames, and / or other contact information associated with the respective users and / or contacts), the one or more user-interactive graphical user interface objects representing one or more previously contacted (e.g., favorite, recently contacted, or frequently contacted) users (e.g., users selected by users associated with the computer system to receive communications from recent and / or within a threshold time period from the current time (e.g., last week, last day, last hour, last 30 minutes)), and, when selected, selecting the one or more previously contacted users with whom to share the link to the real-time communication session (e.g., 624a and / or 634b).
[0299] Displaying one or more user-interactive graphical user interface objects representing one or more previously contacted users allows users to quickly select the desired recipient to share a link to a real-time communication session without providing additional input corresponding to the identifier of the desired recipient, thus reducing the amount of input required to select a recipient. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0300] In some implementations, one or more of a plurality of alternative options (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e and / or 632f) correspond to a direct wireless sharing protocol (e.g., 632a) (e.g., establishing short-range communication with an external computer system (e.g., Bluetooth connection, Zigbee connection and / or Wi-Fi connection) to transmit, send and / or share links to a real-time communication session without traversing intermediate devices such as servers and / or external networks (e.g., the public Internet) to transmit, send and / or share links to an external computer system).
[0301] Including multiple optional options corresponding to various communication protocols allows users to select a preferred communication protocol without having to navigate to another application associated with that preferred protocol, thereby reducing the amount of input required for users to share links. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0302] In some implementations, a computer system (e.g., 600) receives (720) a third user input (e.g., 650g and / or 650i) corresponding to a selection of a messaging user-interactive graphical user interface object (e.g., 612j, 612l, and / or 632b) (e.g., a first optional option, a second optional option, and / or a message invitation display) via one or more input devices (e.g., 602) (e.g., a tap gesture, such as a click gesture); and in response to receiving the third user input, the computer system (e.g., 600) displays (722) the messaging application's message transmission via a display generation component (e.g., 602). A messaging user interface (e.g., 622, 634, and / or 908) (e.g., a messaging user interface including a virtual keyboard, a writing and / or input area, a message area, and one or more indicators of the message recipient), wherein the messaging user interface (e.g., 622, 634, and / or 908) includes a message writing area (e.g., 622c) (e.g., an area of the messaging user interface that displays text, images, data, and / or other information to be sent to the recipient, before detecting user input that causes a message to be sent), the message writing area including links to a real-time communication session (e.g., 624a, 634b, and / or 910a).
[0303] Displaying a messaging user-interactive graphical user interface (GUI) object in a real-time communication application allows users to quickly access the messaging application by selecting the GUI object, without having to provide additional user input to navigate to the messaging application. This reduces the amount of input required for users to share links. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.
[0304] In some implementations, after a link to a real-time communication session (e.g., 624a, 634b, and / or 910a) is shared (e.g., after a user selection indicating a sending capability that causes a message including the link to the real-time communication session to be sent via a messaging application and / or in response to that user selection), the computer system (e.g., 600) displays (724) a second user interface (e.g., 604) of the real-time communication application (e.g., a home user interface and / or default user interface displayed when the real-time communication application is launched), wherein the second user interface (e.g., 604) simultaneously displays: a first user interface object (e.g., 606h) (726) representing the real-time communication session corresponding to the shared link (e.g., 624a, 634b, and / or 910a) (e.g., including information and / or details about the real-time communication session (such as the scheduled time of the real-time communication session, etc.)). Visual indications and / or capability representations of the type of real-time communication session (e.g., audio and / or video), the invitees and / or users who have joined the real-time communication session (in some embodiments, user input corresponding to the selection of a first user interface object causes the computer system to display a real-time communication session user interface for participating in the real-time communication session); and a second user interface object (e.g., 606a-606g) (728) representing a previous real-time communication session that occurred before the current time (e.g., visual indications and / or capability representations corresponding to a previous real-time communication session that the user associated with the computer system joined and / or was invited to join, and which includes information and details about the previous real-time communication session) (in some embodiments, user input corresponding to the selection of a second user interface object initiates a new real-time communication session with the same invitees as the previous real-time communication session).
[0305] The first user interface object, displaying the real-time communication session corresponding to the shared link, allows the user to quickly select and initiate a real-time communication session after the link is shared, thereby reducing the amount of input required for the user to participate in the session. Furthermore, the first user interface object provides improved feedback to the user by confirming that the link to the real-time communication session has been shared and that the session has been scheduled. Reducing the amount of input required to perform operations and providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0306] In some implementations, displaying the second user interface (e.g., 604) includes an animation of the first user interface object (e.g., 606h) appearing in a list of real-time communication sessions (e.g., 606a to 606h), adjusting and / or changing the appearance of the first user interface object over time in response to detecting an event (such as an invitee joining the real-time communication session). In some implementations, the animation of the first user interface object directs the user's attention to the first user interface object, thereby enabling the user to quickly locate the first user interface object and / or enabling the user to understand that the first user interface object corresponds to a real-time communication session associated with a shared link.
[0307] Animating the appearance of the first user interface object representing the real-time communication session corresponding to the shared link provides the user with improved feedback regarding the status of the real-time communication session. Furthermore, animating the appearance of the first user interface object representing the real-time communication session corresponding to the shared link provides the user with improved feedback regarding the first user interface object's representation of the real-time communication session corresponding to the shared link. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.
[0308] In some implementations, a first user interface object (e.g., 606h) is displayed in a first area (e.g., 604d) of a second user interface (e.g., 604) corresponding to a first time period (e.g., the first area corresponding to the current hour, a time occurring after the current time, or the same day). (e.g., the first area of the second user interface is positioned on top of a display generation component corresponding to one or more ongoing real-time communication sessions and / or one or more real-time communication sessions scheduled for times later than the current time; the first user interface object and any additional user interface objects representing ongoing real-time communication sessions and / or real-time communication sessions scheduled for times later than the current time are listed chronologically in the first area (e.g., user interface objects representing real-time communication sessions scheduled for times closest to the current time are listed first and / or above other user interface objects representing real-time communication sessions scheduled for times further from the current time).) and / or the first area of the second user interface is positioned relative to the real-time communication sessions occurring before the current time and / or scheduled before the current time. Above the second area of the second user interface corresponding to the communication session, and the second user interface objects (e.g., 606a to 606g) are displayed in the second area (e.g., 604a, 604b and / or 604c) of the second user interface (e.g., the second area corresponding to the day, the previous day and / or the week) corresponding to the second time period (e.g., 604) different from the first time period (e.g., the second area is positioned below the first area of the second user interface such that the second user interface objects are positioned below and / or after the first user interface objects, the second area and the first area do not overlap each other, the second user interface objects displayed in the second area are positioned adjacent to the third user interface objects representing the second previous real-time communication session (e.g., sharing a boundary with them), and / or the second user interface objects, the third user interface objects and / or any additional user interface objects representing additional previous real-time communication sessions are listed in the second area in chronological order (e.g., the user interface objects representing the most recent previous real-time communication session are listed first and / or above other user interface objects representing less recent previous real-time communication sessions)).
[0309] Arranging the first and second user interface objects of the second user interface into one or more areas corresponding to the relevant time period allows users to quickly and efficiently identify the relevant user interface objects corresponding to a specific real-time communication session without having to scroll through a list of user interface objects to find them, thereby reducing the amount of input required for users to initiate real-time communication sessions. Reducing the amount of input required to perform operations enhances the operability of the device and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.
[0310] In some implementations, a computer system (e.g., 600) (e.g., when a first user interface object is displayed in a first area of a second user interface) receives (730) a fourth user input (e.g., 650o and / or 650p) corresponding to a selection of a first user interface object (e.g., 606h) (e.g., a user interface object corresponding to a real-time communication session associated with a link) via one or more input devices (e.g., 602). In response to receiving the fourth user input (e.g., 650o and / or 650p), the computer system (e.g., 600) displays (732) a representation of the real-time communication session (e.g., 636 and / or 916a), which includes a representation of the real-time contributions of one or more participants in the real-time communication session (e.g., 638, 638, 642, 644, 646, 916a, 916b, 916c and / or 916d) (e.g., a third user interface of a real-time communication application corresponding to the real-time communication session) (e.g., including a representation of the contributions of one or more participants in the real-time communication session associated with the computer system). The user interface (e.g., a visual representation of the user (e.g., a real-time image and / or real-time video feed captured by a camera of a computer system) and one or more visual representations of a user invited to participate in a real-time communication session (e.g., a placeholder image and / or avatar of the corresponding user when the corresponding user has not yet joined the real-time communication session, and / or a real-time image and / or real-time video feed of the corresponding user when the corresponding user has joined the real-time communication session)) (e.g., a user interface that enables the user to communicate with one or more other users associated with an external computer system (e.g., visually and / or audibly)).
[0311] The display of a third user interface for the real-time communication application in response to receiving a fourth user input allows users to initiate real-time communication sessions without additional input for adding and / or selecting one or more invitees, thus reducing the amount of input required for users to participate in real-time communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0312] In some implementations, after displaying a first user interface object (e.g., 606h) representing a real-time communication session corresponding to a shared link (e.g., 624a, 634b, and / or 910a) (e.g., when displaying a second user interface of the real-time communication application), the computer system (e.g., 600) detects (734) that a second user (e.g., a user) has joined the real-time communication session (e.g., a user different from the user associated with the computer system). After detecting that the second user has joined the real-time communication session, the computer system (e.g., 600) detects (736) a request to display the first user interface object (e.g., 606h) (e.g., causing the computer system 600 to display input to the first user interface 604). In response to a detected request to display a first user interface object (e.g., 606h), the computer system (e.g., 600) displays (738) a first user interface object (e.g., 606h) with an updated appearance, wherein a first portion (e.g., 642a) (e.g., text and / or image) of the first user interface object (e.g., 606h) indicates that a second user has joined a live communication session (e.g., adjusting and / or changing the appearance of the first user interface object to visually indicate the identity of one or more users who have joined the live communication session, such that the user associated with the computer system is aware and / or understands that the user is currently participating in a live communication session, and / or adjusting and / or changing the appearance of the first user interface object to visually indicate the number of users who have joined the live communication session).
[0313] Updating the appearance of the first part of the first user interface object representing the real-time communication session corresponding to the shared link provides the user with improved feedback on the status of the real-time communication session. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.
[0314] In some implementations, after displaying a first user interface object (e.g., 606h) representing a real-time communication session corresponding to a shared link (e.g., 624a, 634b, and / or 910a) (e.g., when displaying a second user interface of the real-time communication application), the computer system (e.g., 600) detects (740) that a third user (e.g., the user and / or the second user) has joined the real-time communication session (e.g., the user selects a link and / or otherwise selects a user interface object (e.g., displayed in the user interface of the real-time communication application) to initiate the real-time communication session). After detecting that a third user has joined the real-time communication session, the computer system (e.g., 600) detects (742) a request to display the first user interface object (e.g., 606h) (e.g., causing the computer system 600 to display input to the first user interface 604). In response to a detected request to display a first user interface object (e.g., 606h), a computer system (e.g., 600) displays (744) a first user interface object having a visual icon (e.g., 642b) on a second portion of the first user interface object (e.g., 606h), wherein the visual icon (e.g., 642b) indicates that a third user has joined a live communication session (e.g., a text indicator including the user's identifier (e.g., name, username, phone number, and / or email address) and indicating that the user has joined a live communication session, a visual indicator including an image and / or icon associated with the user and indicating that the user has joined a live communication session, and / or a visual indicator including an image and / or icon indicating the number of users who have joined a live communication session).
[0315] The updated visual icon of the second part of the first user interface object, representing the real-time communication session corresponding to the shared link, provides the user with improved feedback on the status of the real-time communication session. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.
[0316] In some implementations, when a second messaging user interface (e.g., 652) of a second messaging application (e.g., including a keyboard (e.g., a virtual keyboard), a writing area, a session area, and / or a user interface indicating one or more participants in a session) of a second messaging application is displayed via a display generation component (e.g., 602), a computer system (e.g., 600) receives (746) one or more user inputs (e.g., one or more user inputs selecting keys of keyboard 652) corresponding to the content of a message configured to be sent via the messaging application to one or more participants in a messaging session (e.g., one or more user inputs selecting keys of keyboard 652) (e.g., one or more inputs corresponding to the selection of keys of keyboard (e.g., a virtual keyboard) and / or user interface objects of the messaging user interface in the writing area of the messaging area, wherein the computer system is configured to send a message including text, images, and / or data in response to additional user input). In response to receiving one or more user inputs (748): based on the determination that the content includes one or more predefined characters (e.g., alphanumeric characters and / or emoji characters), the computer system (e.g., 600) displays (750) an optional user interface object (e.g., 654, 658, and / or 660) via a display generation component (e.g., an image, object, and / or data corresponding to a link to a real-time communication session to be included in a message configured to be sent via a messaging application of the computer system). This optional user interface object, when selected via user input, is configured to initiate a process for sending a second link (e.g., 656a and / or 910a) to one or more participants in the messaging session to a second real-time communication session (e.g., displaying the second link and / or objects associated with the second link in the writing area of the messaging user interface, such that the second link can be sent in a message via the messaging user interface of the computer system (e.g., in response to user input for sending a message)). In some embodiments, the computer system sends a second link to one or more participants in the messaging session to a second real-time communication session in response to detecting user input corresponding to a selection of an optional user interface object.
[0317] Displaying real-time communication user interface objects allows users to generate and / or send links from messaging applications to real-time communication sessions without navigating to the real-time communication application, thereby reducing the amount of input required for users to schedule and / or create real-time communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0318] In some implementations, when displaying optional user interface objects (e.g., 654, 658, and / or 660), the computer system (e.g., 600) receives (752) a fifth user input (e.g., 650q) corresponding to the selection of the optional user interface objects (e.g., 654, 658, and / or 660) (e.g., a tap gesture, such as a click gesture). In response to receiving a fifth user input (e.g., 650q) corresponding to a selection of an optional user interface object (e.g., 654, 658, and / or 660), the computer system (e.g., 600) displays (754) a second link (e.g., 652b and / or 652c) in a second writing area (e.g., 652b and / or 652c) of a second messaging user interface (e.g., 652, 658, and / or 660) of the second messaging application (e.g., including an area containing text, characters, objects, images, and / or other data sent in a message via the messaging application of the computer system in response to user input (e.g., user input corresponding to a sending user interface object)) to a second real-time communication session (e.g., 656a and / or 910a) (e.g., a Uniform Resource Locator and / or an alphanumeric representation of an object corresponding to the second link).
[0319] Displaying a second link in the second composing area of the second messaging user interface in response to the selection of a real-time communication user interface object allows the user to send links from the second messaging application to the real-time communication session without navigating to the real-time communication application, thereby reducing the amount of input required for the user to schedule and / or create a real-time communication session. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0320] In some implementations, after receiving a fifth user input (e.g., 650q) corresponding to a selection of an optional user interface object (e.g., 654, 658, and / or 660) (in some implementations, in response to detecting user input corresponding to a selection of a sending user interface object for a messaging user interface), the computer system (e.g., 600) (e.g., in response to a request to display a list of recent and upcoming real-time communication sessions) displays (756) a fourth user interface (e.g., 604) of the real-time communication application via a display generation component (e.g., 602). The real-time communication application displays the home user interface and / or default user interface of the real-time communication application, wherein the fourth user interface of the real-time communication application (e.g., 604) includes a third user interface object (e.g., 606h) corresponding to a second link to the second real-time communication session (e.g., 656a and / or 910a) (e.g., a display of one or more visual indications (e.g., text indications and / or images corresponding to the invitee, participant, and / or user of the second real-time communication and / or the scheduled time of the second real-time communication session) of the second link to the second real-time communication session).
[0321] The fourth user interface of a real-time communication application, which displays third user interface objects, enables users to quickly initiate real-time communication sessions, thereby reducing the amount of input required for users to participate in such sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0322] In some implementations, a second link to a second real-time communication session (e.g., 656a and / or 910a) corresponds to a previously scheduled real-time communication session (e.g., 660), which is scheduled before receiving one or more user inputs corresponding to the content of a message configured to be sent via a second messaging application (e.g., the second link to the second real-time communication session is for a previously scheduled real-time communication session that is scheduled to occur at the current time or after the current time; the second link to the second real-time communication session is generated and / or shared with another user before receiving one or more user inputs corresponding to the content of a message configured to be sent via the second messaging application).
[0323] Displaying a first user interface object corresponding to a previously scheduled live communication session allows the user to access an existing link to the previously scheduled live communication session from a second messaging application without navigating to the live communication application, thereby reducing the amount of input required for the user to schedule and / or create live communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0324] In some implementations, in response to detecting one or more user inputs corresponding to the content of a message configured to be sent to one or more participants in a messaging session via a messaging application, a computer system (e.g., 600) displays (758) via a display generation component (e.g., 602): an optional user interface object (e.g., 660) (760), which, when selected via user input, is configured to initiate a process for sending a second link (e.g., 656a and / or 910a) to a second real-time communication session; and a second optional user interface object (e.g., 658) (762) (e.g., optional user interface elements different from the first real-time communication user interface object; images, objects, and / or data corresponding to the link to the real-time communication session to be included in a message configured to be sent via a messaging application of the computer system), which, when selected via user input, is configured to generate a third link to a third real-time communication session (e.g., displaying the third link and / or objects associated with the third link in the writing area of the messaging user interface, such that the third link can be sent in a message via the messaging user interface of the computer system (e.g., in response to user input for sending a message)). The third link to the third real-time communication session corresponds to a new real-time communication that does not correspond to a previously scheduled real-time communication session (e.g., a third link to the third real-time communication session was not previously generated; a third real-time communication session was not previously scheduled; and / or the third link to the third real-time communication session corresponds to a newly generated and scheduled real-time communication session).
[0325] Displaying a first real-time communication user interface object corresponding to a previously scheduled real-time communication session and a second real-time communication user interface object corresponding to a new real-time communication session allows the user to choose between an existing link from the messaging application to the previously scheduled real-time communication session or a new link to the new real-time communication session without navigating to the real-time communication application. This reduces the amount of input required for the user to schedule and / or create real-time communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.
[0326] In some implementations, the computer system (e.g., 600) displays (764) a calendar user interface (e.g., 662) of a calendar application via a display generation component (e.g., 602), which includes a scheduling user interface object (e.g., 662b) (e.g., buttons and / or display representations that, when selected via user input, cause the computer system to initiate a process for creating and / or scheduling appointments, meetings, and / or events). The computer system (e.g., 600) receives (766) a sixth user input (e.g., 650r) corresponding to the selection of the scheduling user interface object (e.g., 662b) via one or more input devices (e.g., 602) (e.g., a tap gesture, such as a click gesture). In response to receiving a sixth user input (e.g., 650r) (768), the computer system (e.g., 600) displays (770) an event scheduling user interface (e.g., 664) including a video conferencing user interface object (e.g., 664b) (e.g., a user interface that enables the user to input and / or specify details associated with a new appointment, meeting, and / or event) via a display generation component. The video conferencing user interface object displays a video conferencing user interface (e.g., 666) (e.g., a user interface that enables the user to select a video conferencing application and / or specify settings and / or details of the video conference associated with a new appointment, meeting, and / or event) when selected via user input.
[0327] The event scheduling user interface, which displays video conferencing user interface objects, allows users to schedule live communication sessions via a calendar application without having to navigate to the live communication application, thus reducing the amount of input required for users to schedule and / or create live communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0328] In some implementations, when an event scheduling user interface (e.g., 664) is displayed via a display generation component (e.g., 602), a computer system (e.g., 600) receives (772) a seventh user input (e.g., 650s) corresponding to a selection of a video conferencing user interface object (e.g., 664b) via one or more input devices (e.g., a tap gesture, such as a click gesture). In response to receiving the seventh user input (e.g., 650s) (774), the computer system (e.g., 600) displays (776) a video conferencing user interface (e.g., 666) via the display generation component, including multiple video conferencing options (e.g., 666b to 666e) (e.g., a user interface that enables a user to select a video conferencing application and / or specify settings and / or details of a video conference associated with a new appointment, meeting, and / or event), which correspond to a respective application for participating in a real-time communication session (e.g., buttons and / or indications corresponding to different applications and / or software configured to enable a user associated with the computer system to participate in a video conference (e.g., a real-time communication session).
[0329] The video conferencing user interface, which displays multiple video conferencing options, allows users to schedule live communication sessions via their preferred live communication application without having to navigate to it, thus reducing the amount of input required for scheduling and / or creating live communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0330] In some implementations, multiple video conferencing options (e.g., 666b to 666e) include a first video conferencing option (e.g., 666d) corresponding to a first application of a computer system that meets one or more criteria (e.g., meeting one or more criteria when the application and / or software is installed, downloaded, and / or otherwise linked to the computer system (e.g., the computer system is checked into an account for the application and / or software) and / or when the application and / or software is configured to enable users of the computer system to participate in video conferencing (e.g., a real-time communication session)) and a second video conferencing option (666b) corresponding to a second application of a computer system that meets one or more criteria (e.g., the second application of the computer system is different from the first application of the computer system; meeting one or more criteria when the application and / or software is installed, downloaded, and / or otherwise linked to the computer system (e.g., the computer system is checked into an account for the application and / or software) and / or when the application and / or software is configured to enable users of the computer system to participate in video conferencing (e.g., a real-time communication session)).
[0331] Displaying multiple video conferencing options corresponding to applications on the computer system and meeting one or more standards allows users to schedule real-time communication sessions via applications already downloaded and / or installed on the computer system. This eliminates the need for users to provide additional user input to install and / or download new applications and / or software on the computer system. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.
[0332] In some implementations, when a video conferencing user interface (e.g., 666) is displayed via a display generation component (e.g., 602), a computer system (e.g., 600) receives (778) an eighth user input (e.g., 650t) corresponding to the selection of a first video conferencing option (e.g., 666d) among a plurality of optional video conferencing options (e.g., 666b to 666e). This input may be a tap gesture, such as a click gesture. In response to receiving an eighth user input (e.g., 650t) (780), the computer system (e.g., 600) displays (782) a plurality of optional setting options (e.g., 668a to 668c) via a display generation component (e.g., 602) (e.g., enablement indicators and / or buttons that allow the user to select, specify, and / or input various settings, options, and / or features of a real-time communication session to be executed via a first application of the computer system), wherein the plurality of optional setting options (e.g., 668a to 668c) correspond to the available settings of the first application of the computer system (e.g., 600) (e.g., the plurality of setting options are specific to the first application of the computer system and / or generated by the first application, such as an option for a one-time room for a real-time communication session, an option for a personal room associated with the user's account for a real-time communication session, and / or a password option that allows the user to restrict access to the real-time communication session to a user who enters a predefined password when joining the real-time communication session and / or before joining the real-time communication session).
[0333] The video conferencing settings user interface, which displays multiple optional settings, allows users to specify and / or select settings for live communication sessions without having to navigate to the live communication application, thus reducing the amount of input required for users to schedule and / or create live communication sessions. Reducing the amount of input required to perform operations enhances device operability and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0334] It should be noted that the above is relative to method 700 (e.g., Figures 7A to 7H The details of the process described herein also apply in a similar manner to the methods described below. For example, methods 800 and 1000 optionally include one or more features of the various methods described above with reference to method 700. For example, the computer system of method 700 can be used to invite participants to and join a real-time communication session and / or to allow participants to join a real-time communication session. For the sake of brevity, these details will not be repeated below.
[0335] Figures 8A to 8G This is a flowchart illustrating a method for inviting and joining a real-time communication session using an electronic device, according to some embodiments. Method 800 is performed at a computer system (e.g., 100, 300, 500, 600, 900, 902, 904 and / or 906) (e.g., an electronic device; a smart device, such as a smartphone or smartwatch; a mobile device; a wearable device), which communicates with a display generation component (e.g., 602) and one or more input devices (e.g., 602). Some operations in method 800 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.
[0336] As described below, method 800 provides an intuitive way to invite and join a real-time communication session. This method reduces the cognitive burden on users when inviting and / or joining a real-time communication session, thus creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to invite and / or join real-time communication sessions more quickly and efficiently, saving power and increasing the time interval between battery charges.
[0337] The computer system (e.g., 600) detects (802) one or more first user inputs (e.g., 650c, 650d, 650e and / or 650f) corresponding to the selection of a user (e.g., 650c, 650d, 650e and / or 650f) on a user interface object (e.g., displayed in response to one or more first user inputs) to invite a user to participate in a real-time communication session (e.g., a voice and / or video conference between at least a user associated with the computer system and a user associated with an external computer system) via one or more input devices (e.g., 602) to invite a user to participate in a real-time communication session (e.g., a user interface object (e.g., displayed in response to one or more first user inputs) to a contact stored in the memory of the computer system and / or a user phone number associated with an external computer system associated with the user) (e.g., a tap gesture on a user interface object associated with the user and / or one or more inputs associated with the user's name and / or phone number on a keyboard displayed via a display generating component).
[0338] After detecting one or more first user inputs (e.g., 650c, 650d, 650e, and / or 650f) (e.g., in response to detecting the one or more first user inputs) (e.g., when a user types a user interface object associated with a contact and / or the user and / or in response to selecting a user interface object associated with a contact and / or the user), the computer system (e.g., 600) simultaneously displays (804) via a display generation component (e.g., 602): a link to a real-time communication session (e.g., 624a, 634b, and / or 910a) (e.g., such a link, in response to user input corresponding to the selection of that link, enables communication with an external computer system). The system includes optional user interface elements (e.g., 612j, 612k, and / or 612l) corresponding to real-time communication applications and / or different applications (e.g., applications associated with web browsers) that enable users to participate in real-time communication sessions with users associated with the computer system (e.g., 612j, 612k, and / or 612l) (806) (e.g., enable indicators and / or buttons prompting the user to continue setting up the real-time communication session after selecting one or more users associated with the corresponding external computer system and / or enable indicators and / or buttons that send messages and / or data including links associated with the real-time communication session when selected) and visual indications for the user (e.g., 612g and / or 612i) (808).
[0339] When simultaneously displaying optional user interface elements (e.g., 612j, 612k and / or 612l) and user visual indications (e.g., 612g and / or 612i), the computer system (e.g., 600) detects (810) via one or more input devices (e.g., 602) a second user input (e.g., 650g, 650i and / or 650j) corresponding to the selection of optional user interface elements (e.g., 612j, 612k and / or 612l) (e.g., a tapping gesture).
[0340] In response to (812) detecting a second user input (e.g., 650g, 650i, and / or 650j), the computer system (e.g., 600) initiates (814) a process for sending a link to the user (e.g., 624a, 634b, and / or 910a) (e.g., displaying a shared user interface (e.g., a messaging user interface, a communication application user interface, and / or an email user interface) that enables the user associated with the computer system to send a link to a user associated with an external computer system and / or an additional user associated with the corresponding external computer system) and displays (816) a user interface for participating in a real-time communication session with the user (e.g., 636 and / or 916a) (e.g., automatically starting the real-time communication session by transitioning from the shared user interface to the real-time communication user interface after sending the link). In some embodiments, initiating a real-time communication session is performed in response to receiving a second user input. In some embodiments, initiating a real-time communication session displays a real-time communication user interface including placeholder user interface objects corresponding to the user associated with the external computer system. In response to a link to a real-time communication session by user activation associated with an external computer system (e.g., user input detected by the external computer system), a placeholder user interface object is replaced with an image and / or representation of the user associated with the external computer system (e.g., received from the camera of the external computer system).
[0341] In response to the detection of a second user input, a user interface for participating in a real-time communication session with the user is displayed. This initiates the real-time communication session by sending a link to the user, without requiring the user associated with the computer system to navigate to another user interface and / or provide additional input. Reducing the amount of input required to perform an operation enhances the operability of the device and makes the user device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.
[0342] In some implementations, before simultaneously displaying optional user interface elements (e.g., 612j, 612k, and / or 612l) and visual indications for the user (e.g., 612g and / or 612i) (e.g., upon detecting one or more inputs corresponding to the selection of a user to be invited to participate in a live communication session), the computer system (e.g., 600) displays (818) a user selection user interface (e.g., 612) via a display generation component (e.g., 602) (e.g., the user interface displayed upon detecting one or more inputs corresponding to the selection of a user to be invited to participate in a live communication session; the user interface that enables the user to select and / or input the user and / or additional users to be invited to participate in the live communication session). Based on the determination that any user selected as an invitee for a real-time communication session does not meet the first set of criteria (e.g., the first set of criteria includes that the identifiers associated with the respective user (e.g., any user selected as an invitee for a real-time communication session) such as username, email address, phone number, and / or other identifiers are not associated with the real-time communication application configured to initiate the real-time communication session and / or that the external computer system associated with the respective user (e.g., any user selected as an invitee for a real-time communication session) does not include one or more requirements of a specific application (e.g., a specific real-time video communication application), the user selects a user interface (e.g., 612) including initiating a user interface object (e.g., 612d) (820) (e.g., a button and / or The indication may optionally include an initiating user interface object (such as a visual indication) that causes a real-time communication session to be initiated in response to user selection. This initiating user interface object is configured to initiate a real-time communication session (e.g., without a separate message sending link) when selected via user input (e.g., and any additional users selected / invited to the real-time communication session) (e.g., when all users selected as invitees to the real-time communication session are associated with a corresponding external computer system including a specific real-time communication application (e.g., an account installed on the corresponding external computer system and / or logged into and / or otherwise associated with the specific real-time communication application). In some embodiments, in response to detecting user input corresponding to the initiating user interface object, the computer system calls the user's receiving device (e.g., causes the receiving device to ring for a predetermined period of time) without sending an asynchronous message with a link to the real-time communication session to the user's receiving device.Based on the fact that at least one user selected as an invitee for a real-time communication session meets a first set of criteria (e.g., at least one user selected as an invitee for a real-time communication session is not associated with an external computer system that includes a specific real-time communication application) (e.g., identifiers associated with at least one user (such as username, email address, phone number, and / or other identifiers) are not associated with a real-time communication application configured to initiate a real-time communication session, and / or the external computer system associated with at least one user does not include a specific application (e.g., a specific real-time video communication application)), the user selects a user interface (e.g., 612) that includes linked-share user interface objects (e.g., 612j, 612k, and / or 612l) (822) (e.g., buttons and / or indicator lights, which optionally include linked-share objects). A user interface object enables the sharing of a link to a live communications session in response to user selection (such as a visual indication). This link-sharing user interface object is configured, when selected via user input, to initiate a process for sharing a link to a live communications session with one or more other users (e.g., a user interface including optional user interface elements and visual indications for the user, such as a messaging user interface of a messaging application). (For example, the link-sharing user interface object is displayed when at least one user selected as an invitee to a live communications session is associated with a corresponding external computer system that does not include a specific live communications application (e.g., installed on the corresponding external computer system and / or logged into an account of the specific live communications application and / or otherwise associated with an account of the specific live communications application).
[0343] Based on whether the selected invitee meets the first set of criteria, the user interface object that initiates the session and / or the link-sharing user interface object provides relevant options for inviting users to participate in the live communication session, thereby reducing the amount of input required to send links to users. Furthermore, based on whether the selected invitee meets the first set of criteria, the user interface object that initiates the session and / or the link-sharing user interface object provides improved feedback regarding the selected user. Reducing the amount of input required to perform operations and providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0344] In some implementations, when displaying a user selection user interface (e.g., 612) including an initiating user interface object (e.g., 612d) (e.g., the user selected as an invitee in a live communication session and any other user meet this set of criteria), a computer system (e.g., 600) detects (824) one or more third user inputs (e.g., 650e and / or 650f) corresponding to the selection of a second user to be invited to participate in the live communication session (e.g., a second user different from the user) (e.g., one or more tap gestures), wherein the second user meets the first set of criteria (e.g., the identifiers associated with the second user (such as username, email address, phone number and / or other identifiers) are not associated with a live communication application configured to initiate a live communication session, and / or the external computer system associated with the second user does not include a specific application (e.g., a specific live video communication application)). In response to the detection of one or more third-user inputs (e.g., 650e and / or 650f), the computer system (e.g., 600) stops displaying (826) the initiating user interface object (e.g., 612d) and displays a linked-shared user interface object (e.g., 612j, 612k, and / or 612l) (e.g., replacing the display of the initiating user interface object with the linked-shared user interface object). In some embodiments, the computer system replaces the display of the initiating user interface object with the linked-shared user interface object when any user selected as an invitee of the real-time communication session meets the first set of criteria (e.g., any user selected as an invitee of the real-time communication session is associated with an external computer system that does not include the specific real-time communication application). The computer system displays the initiating user interface object when the user invited to participate in the real-time communication session and all other users do not meet the first set of criteria.
[0345] When the second user meets the first set of criteria, the display of the initiating user interface object is replaced with a link-sharing user interface object. This provides relevant options for inviting the user to participate in a real-time communication session based on the first and second users, thereby reducing the amount of input required to send links to both users. Furthermore, replacing the initiating user interface object with a link-sharing user interface object provides improved feedback regarding the selected second user. Reducing the amount of input required to perform operations and providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.
[0346] In some implementations, the user selection user interface (e.g., 612) including the user interface object that initiates the user interface (e.g., 612d) (e.g., a user selected as an invitee in a live communication session and any other user) includes a second visual indication of the user with a first visual characteristic (e.g., 612g) (e.g., an indication of the user's identifier (such as name, username, email address, phone number and / or other identifier) with a first font and / or text color (such as blue)). In some implementations, in response to detecting one or more third user inputs (e.g., 650e and / or 650f) corresponding to the selection of a second user (828): the computer system (e.g., 600) displays (830) a second visual indication (e.g., 612g) of the user having a second visual characteristic different from the first visual characteristic (e.g., an indication of the user's identifier (such as name, username, email address, telephone number and / or other identifier) having a second font and / or text color (such as green); and the computer system (e.g., 600) displays the third visual indication (e.g., 612i) of the second user having the second visual characteristic (e.g., 612i) (e.g., an indication of the second user's identifier (such as name, username, email address, telephone number and / or other identifier) having a second font and / or text color (such as green)) simultaneously with the user's second visual indication (e.g., 612g) (832).
[0347] When a second user meets the first set of criteria, changing the visual characteristics of the user's second visual indication from the first visual characteristic to the second visual characteristic provides feedback on the selected second user and on the improved communication protocol used to communicate with both the first and second users. Providing improved feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and effectively.
[0348] In some implementations, before simultaneously displaying optional user interface elements (e.g., 612j, 612k, and / or 612l) and visual indications for the user (e.g., 612g and / or 612i), the computer system (e.g., 600) displays (834) a second user selection user interface (e.g., 612) via a display generation component (e.g., 602). This includes, for example, one or more user interface objects corresponding to selectable contacts and / or users to invite to a live communication session, a keyboard (e.g., a virtual keyboard) for searching for contacts and / or users to invite to a live communication session, and / or visually indicating the invitee of the user and / or other user who has been selected as an invitee for the live communication session. The second user selection user interface simultaneously displays: a messaging user interface object (e.g., 612k and / or 612l) (836) (e.g., a first user interface object corresponding to a link shared in an electronic message (e.g., a short message service message) via a messaging application of a computer system to a real-time communication session), which, when selected by user input, causes the display of a messaging application (e.g., 624a, 634b and / or 910a) for sharing a link to a real-time communication session (e.g., enabling electronic messages (e.g., short message service messages) to reach one or more recipients (e.g., via name, phone number, username, etc.). A messaging user interface (e.g., 622 and / or 908) of a computer system (including an application and / or software for the recipient identified by an email address and / or another suitable identifier) (e.g., a message area with one or more visual indications of the message being sent and / or received, a keyboard (e.g., a virtual keyboard) for selecting the content of the message, a writing area for drafting the content of the message, and / or an indication of one or more recipients of the message) (e.g., the computer system is configured to send electronic messages (e.g., short message service messages) with links to real-time communication sessions via a network (e.g., the Internet, cellular networks, and / or personal networks)); and shared user interface objects (e.g., 612j). 838) (e.g., a second user interface object corresponding to a link shared to a real-time communication session via one or more communication protocols), which causes a shared user interface (e.g., 626 and / or 632) to be displayed when selected by user input, the shared user interface including multiple optional options (e.g., 628a, 628b, 628c, 630, 632a, 632b, 632c, 632d, 632e and / or 632f) for sharing links to a real-time communication session via different communication protocols (624a, 634b and / or 910a) (e.g., power indicators and / or buttons corresponding to different sharing options, technologies and / or applications for sharing the link).
[0349] Simultaneously displaying both messaging and sharing user interface objects allows users to select a preferred communication protocol without navigating to another application associated with that protocol, thus reducing the amount of input required for user-shared links. Reducing the amount of input required to perform operations enhances device operability and makes the user interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0350] In some implementations, the sharing user interface includes (e.g., 626 and / or 632): a sharing option (e.g., 628a) selectable to share a link to a live communication session via one or more communication protocols (e.g., SMS service, email, short-range communication, direct wireless communication technology, file-sharing service, and / or social media application) (e.g., 624a, 634a, and / or 910a); and a copy option (e.g., 628b) selectable to copy to a link in the live communication session (e.g., 624a and / or 634b) (e.g., 624a and / or 634b). For example, storing (e.g., temporary storage) data associated with a link to a real-time communication session, which can be reproduced (e.g., pasted) in response to the detection of user input; and messaging options (e.g., 628c) that can be selected to share the link to the real-time communication session via a messaging application (e.g., 624a and / or 634b) (e.g., such optional options, when selected via user input, cause the computer system to display a messaging user interface of a messaging application configured to send electronic messages with a link to the real-time communication session).
[0351] Including sharing, copying, and messaging options on the shared user interface allows users to select preferred sharing options without navigating to another application associated with a preferred communication protocol, thus reducing the amount of input required for users to share links. Reducing the amount of input required to perform operations enhances device operability and makes the user interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.
[0352] In some implementations, the computer system (e.g., 600) detects (840) a fourth user input (e.g., 650j) corresponding to a selection of a sharing option (e.g., 612k) (e.g., a tapping gesture, such as a click gesture). In response to the detection of the fourth user input (e.g., 650j), the computer system (e.g., 600) displays (842) a second sharing user interface (e.g., 626) via a display generation component (e.g., 602) (e.g., a sharing user interface including multiple optional options for sharing a link to a real-time communication session via one or more communication protocols (e.g., enable symbols and / or buttons corresponding to different sharing options, technologies, and / or applications for sharing the link)). This second sharing user interface simultaneously displays: for meeting a second set of criteria (e.g., identifiers associated with each user (such as username, email address, phone number, and / or other identifiers) not associated with a real-time communication application configured to initiate a real-time communication session, and...). A first optional option (e.g., 628a to 628c) (844) for sharing a link to a real-time communication session with users whose external computer systems associated with each user do not include a specific application (e.g., a specific real-time video communication application); and a second optional option (e.g., 630) (846) for sharing a link to a real-time communication session with users who do not meet the second set of criteria (e.g., the first set of criteria; that the identifiers associated with each user (such as username, email address, telephone number and / or other identifiers) are associated with a real-time communication application configured to initiate a real-time communication session, and / or that the external computer systems associated with each user include a specific application (e.g., a specific real-time video communication application)).
[0353] Simultaneously displaying a first and a second optional option for sharing a link with different types of users allows users to share the link via two communication protocols at the same time, thereby reducing the amount of input required to send the link to the user. Reducing the amount of input required to perform an operation enhances the operability of the device and makes the user device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.
[0354] In some implementations, the computer system (e.g., 600) detects (848) a fifth user input (e.g., 650k) (e.g., a tapping gesture, such as a click gesture), which corresponds to the selection of a first optional option (e.g., 628a) for sharing a link to a real-time communication session with a user who meets a second set of criteria. In response to the detection of a fifth user input (e.g., 650k), the computer system (e.g., 600) displays (850) a third shared user interface (e.g., 632) via a display generation component (e.g., a shared user interface including multiple optional options (e.g., power indicators and / or buttons corresponding to different sharing options, technologies, and / or applications for sharing the link) for a link to a real-time communication session via one or more communication protocols), wherein displaying the third shared user interface (e.g., 632) includes simultaneously displaying: a third optional option (e.g., 632a to 632f) (852) for sharing a link to a real-time communication session via a first communication protocol (e.g., SMS service, email, short-range communication, direct wireless communication technology, file-sharing service, and / or social media application); and a fourth optional option (e.g., 632a to 632f) (854) for sharing a link to a real-time communication session via a second communication protocol different from the first communication protocol (e.g., SMS service, email, short-range communication, direct wireless communication technology, file-sharing service, and / or social media application).
[0355] The inclusion of a third optional option for sharing a link to a real-time communication session via a first communication protocol and a fourth optional option for sharing a link to a real-time communication session via a second communication protocol different from the first communication protocol provides the user with multiple options for sharing a link to a real-time communication session, and reduces the amount of input required for the user to share the link. Reducing the amount of input required to perform operations enhances the operability of the device and makes the user device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.
[0356] In some implementations, before simultaneously displaying optional user interface elements (e.g., 612j, 612k, and / or 612l) and visual indications for the user (e.g., 612g and / or 612i), the computer system (e.g., 600) displays (856) a user list (e.g....
Claims
1. A computer system communicating with a display generation component and one or more input devices, the computer system comprising: One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: While the computer system displays a user interface via the display generation component that shows a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication. as well as After the second user attempts to join the real-time communication session, a visual representation of the second user's attempt to join the real-time communication session is displayed in the user interface, wherein the visual representation of the second user's attempt to join the real-time communication session includes: Based on the determination that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session in order to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option that can be selected to determine whether to allow the second user to participate in the real-time communication session; and Based on the determination that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without showing an option to be selected to determine whether to allow the second user to participate in the real-time communication session and without displaying a removal user interface object, which causes the second user to be removed from the real-time communication session when selected via user input; and When displaying the option to select whether to allow the second user to participate in the real-time communication session, user input corresponding to a request to allow the second user to participate in the real-time communication session is detected; and In response to detecting user input corresponding to the request to allow the second user to participate in the real-time communication session, the following is displayed via the display generation unit: The representation of the second user; and The removal of the user interface object causes the second user to be removed from the real-time communication session when selected via user input.
2. The computer system according to claim 1, wherein: The first user is designated as the host of the real-time communication session, and the computer system displays an option that can be selected to determine whether to allow the second user to participate in the real-time communication session, and the option that can be selected to determine whether to allow the second user to participate in the real-time communication session is not displayed on other devices of other participants in the real-time communication session.
3. The computer system of claim 1, wherein the first user is not designated as the host of the real-time communication session.
4. The computer system of claim 1, wherein the one or more programs further include instructions for performing the following operations: When the representation of the second user is displayed in the user interface, data instructing a third user participating in the real-time communication session to remove the second user from the real-time communication session is received; and In response to receiving an instruction from the third user participating in the real-time communication session to remove the second user's data from the real-time communication session, the display of the second user's representation is stopped.
5. The computer system of claim 1, wherein the visual representation of the attempt by the second user to join the real-time communication session, including the option that can be selected to determine whether to allow the second user to participate in the real-time communication session, comprises: The visual representation of the second user's attempt to join the real-time communication session, including the option that can be selected to determine whether to allow the second user to participate in the real-time communication session, is displayed without displaying the representation of the second user.
6. The computer system of claim 1, wherein the one or more criteria include a second requirement that the first user has participated in the real-time communication session for at least a threshold amount of time in order to satisfy the one or more criteria.
7. The computer system according to claim 1, wherein: The second user attempts to join the real-time communication session by selecting a link corresponding to the real-time communication session. The second user is allowed to join the real-time communication session, and A fifth user who meets the second set of criteria can remove the second user from the real-time communication.
8. The computer system of claim 7, wherein the second set of standards includes the fifth user connecting to the real-time communication session using a specific real-time communication application to satisfy a second requirement of the second set of standards.
9. The computer system of claim 8, wherein the second set of criteria includes the fifth user having participated in the real-time communication session for at least a threshold time period in order to satisfy a third requirement of the second set of criteria.
10. The computer system of claim 1, wherein the representation of the second user includes a visual indication of the identifier name of the second user.
11. The computer system of claim 1, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is satisfied when the identifier of the second user is included in the list of verified invitees associated with the real-time communication session and the identifier of the second user is associated with an account of a specific type.
12. The computer system of claim 1, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is satisfied when the second user's email address is included in a list of verified invitees associated with the real-time communication session and the second user's email address includes a predefined domain name.
13. The computer system of claim 1, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the identifier of the second user is included in the list of verified invitees associated with the real-time communication session and the identifier of the second user is not associated with a particular type of account.
14. The computer system of claim 1, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the second user attempts to join the real-time communication session via a link to the real-time communication session and the identifier of the second user is not associated with a specific type of account.
15. The computer system of claim 1, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not met when the identifier of the second user is not included in the list of verified invitees associated with the real-time communication session.
16. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices, the one or more programs including instructions for: While the computer system displays a user interface via the display generation component that shows a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication. as well as After the second user attempts to join the real-time communication session, a visual representation of the second user's attempt to join the real-time communication session is displayed in the user interface, wherein the visual representation of the second user's attempt to join the real-time communication session includes: Based on the determination that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session in order to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option that can be selected to determine whether to allow the second user to participate in the real-time communication session; and Based on the determination that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without showing an option to be selected to determine whether to allow the second user to participate in the real-time communication session and without displaying a removal user interface object, which causes the second user to be removed from the real-time communication session when selected via user input; and When displaying the option to select whether to allow the second user to participate in the real-time communication session, user input corresponding to a request to allow the second user to participate in the real-time communication session is detected; and In response to detecting user input corresponding to the request to allow the second user to participate in the real-time communication session, the following is displayed via the display generation unit: The representation of the second user; and The removal of the user interface object causes the second user to be removed from the real-time communication session when selected via user input.
17. The non-transitory computer-readable storage medium according to claim 16, wherein: The first user is designated as the host of the real-time communication session, and the computer system displays an option that can be selected to determine whether to allow the second user to participate in the real-time communication session, and the option that can be selected to determine whether to allow the second user to participate in the real-time communication session is not displayed on other devices of other participants in the real-time communication session.
18. The non-transitory computer-readable storage medium of claim 16, wherein the first user is not designated as the host of the real-time communication session.
19. The non-transitory computer-readable storage medium of claim 16, wherein the one or more programs further include instructions for: When the representation of the second user is displayed in the user interface, data instructing a third user participating in the real-time communication session to remove the second user from the real-time communication session is received; and In response to receiving an instruction from the third user participating in the real-time communication session to remove the second user's data from the real-time communication session, the display of the second user's representation is stopped.
20. The non-transitory computer-readable storage medium of claim 16, wherein the visual representation of the attempt by the second user to join the real-time communication session, including the option selectable to determine whether to allow the second user to participate in the real-time communication session, comprises: The visual representation of the second user's attempt to join the real-time communication session, including the option that can be selected to determine whether to allow the second user to participate in the real-time communication session, is displayed without displaying the representation of the second user.
21. The non-transitory computer-readable storage medium of claim 16, wherein the one or more criteria include a second requirement that the first user has participated in the real-time communication session for at least a threshold amount of time to satisfy the one or more criteria.
22. The non-transitory computer-readable storage medium according to claim 16, wherein: The second user attempts to join the real-time communication session by selecting a link corresponding to the real-time communication session. The second user is allowed to join the real-time communication session, and A fifth user who meets the second set of criteria can remove the second user from the real-time communication.
23. The non-transitory computer-readable storage medium of claim 22, wherein the second set of standards includes the fifth user connecting to the real-time communication session using a specific real-time communication application to satisfy a second requirement of the second set of standards.
24. The non-transitory computer-readable storage medium of claim 23, wherein the second set of criteria includes a third requirement that the fifth user has participated in the real-time communication session for at least a threshold time period in order to satisfy the second set of criteria.
25. The non-transitory computer-readable storage medium of claim 16, wherein the representation of the second user includes a visual indication of the identifier name of the second user.
26. The non-transitory computer-readable storage medium of claim 16, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is satisfied when the identifier of the second user is included in a list of verified invitees associated with the real-time communication session and the identifier of the second user is associated with an account of a specific type.
27. The non-transitory computer-readable storage medium of claim 16, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is satisfied when the second user's email address is included in a list of verified invitees associated with the real-time communication session and the second user's email address includes a predefined domain name.
28. The non-transitory computer-readable storage medium of claim 16, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the identifier of the second user is included in the list of verified invitees associated with the real-time communication session and the identifier of the second user is not associated with a particular type of account.
29. The non-transitory computer-readable storage medium of claim 16, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the second user attempts to join the real-time communication session via a link to the real-time communication session and the identifier of the second user is not associated with a specific type of account.
30. The non-transitory computer-readable storage medium of claim 16, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the identifier of the second user is not included in the list of verified invitees associated with the real-time communication session.
31. A method comprising: At the computer system that communicates with the display generation component and one or more input devices: While the computer system displays a user interface via the display generation component that shows a visual representation of an active real-time communication session of a first user, a second user, different from the first user, attempts to join the real-time communication. as well as After the second user attempts to join the real-time communication session, a visual representation of the second user's attempt to join the real-time communication session is displayed in the user interface, wherein the visual representation of the second user's attempt to join the real-time communication session includes: Based on the determination that one or more criteria have been met, wherein the one or more criteria include the requirement that the second user's identity has not yet been verified as an invitee of the real-time communication session in order to meet the requirements of the one or more criteria, the visual representation of the second user's attempt to join the real-time communication session includes an option that can be selected to determine whether to allow the second user to participate in the real-time communication session; and Based on the determination that the second user's identity has been verified as an invitee of the real-time communication session, the visual representation of the second user's attempt to join the real-time communication session includes a representation of the second user, which is displayed without showing an option to be selected to determine whether to allow the second user to participate in the real-time communication session and without displaying a removal user interface object, which causes the second user to be removed from the real-time communication session when selected via user input; and When displaying the option to select whether to allow the second user to participate in the real-time communication session, user input corresponding to a request to allow the second user to participate in the real-time communication session is detected; and In response to detecting user input corresponding to the request to allow the second user to participate in the real-time communication session, the following is displayed via the display generation unit: The representation of the second user; and The removal of the user interface object causes the second user to be removed from the real-time communication session when selected via user input.
32. The method according to claim 31, wherein: The first user is designated as the host of the real-time communication session, and the computer system displays an option that can be selected to determine whether to allow the second user to participate in the real-time communication session, and the option that can be selected to determine whether to allow the second user to participate in the real-time communication session is not displayed on other devices of other participants in the real-time communication session.
33. The method of claim 31, wherein the first user is not designated as the host of the real-time communication session.
34. The method of claim 31, further comprising: When the representation of the second user is displayed in the user interface, data indicating that a third user participating in the real-time communication session removes the second user from the real-time communication session is received; as well as In response to receiving an instruction from the third user participating in the real-time communication session to remove the second user's data from the real-time communication session, the display of the second user's representation is stopped.
35. The method of claim 31, wherein the visual representation of the attempt by the second user to join the real-time communication session, including the option that can be selected to determine whether to allow the second user to participate in the real-time communication session, comprises: The visual representation of the second user's attempt to join the real-time communication session, including the option that can be selected to determine whether to allow the second user to participate in the real-time communication session, is displayed without displaying the representation of the second user.
36. The method of claim 31, wherein the one or more criteria include a second requirement that the first user has participated in the real-time communication session for at least a threshold amount of time in order to satisfy the one or more criteria.
37. The method according to claim 31, wherein: The second user attempts to join the real-time communication session by selecting a link corresponding to the real-time communication session. The second user is allowed to join the real-time communication session, and A fifth user who meets the second set of criteria can remove the second user from the real-time communication.
38. The method of claim 37, wherein the second set of criteria includes the fifth user connecting to the real-time communication session using a specific real-time communication application to satisfy a second requirement of the second set of criteria.
39. The method of claim 38, wherein the second set of criteria includes a third requirement that the fifth user has participated in the real-time communication session for at least a threshold time period in order to satisfy the second set of criteria.
40. The method of claim 31, wherein the representation of the second user includes a visual indication of the identifier name of the second user.
41. The method of claim 31, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is satisfied when the identifier of the second user is included in the list of verified invitees associated with the real-time communication session and the identifier of the second user is associated with an account of a specific type.
42. The method of claim 31, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is satisfied when the second user's email address is included in the list of verified invitees associated with the real-time communication session and the second user's email address includes a predefined domain name.
43. The method of claim 31, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the identifier of the second user is included in the list of verified invitees associated with the real-time communication session and the identifier of the second user is not associated with a particular type of account.
44. The method of claim 31, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not satisfied when the second user attempts to join the real-time communication session via a link to the real-time communication session and the second user's identifier is not associated with a specific type of account.
45. The method of claim 31, wherein the requirement that the second user's identity has been verified as an invitee of the real-time communication session is not met when the identifier of the second user is not included in the list of verified invitees associated with the real-time communication session.
Citation Information
Patent Citations
Method and apparatus for integrating manual input
US20020015024A1
Gestures for touch sensitive input devices
US20060026521A1
Gestures for touch sensitive input devices
US20060026536A1
Virtual input device placement on a touch screen user interface
US20060033724A1
Multipoint touchscreen
US20060097991A1