Devices, methods, and graphical user interfaces for automatically providing shared content to applications.
By automatically integrating shared content into relevant applications and providing linkable, highlighted content on touch-sensitive devices, the inefficiencies and power consumption issues of current methods are addressed, improving user experience and device efficiency.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2026-04-13
AI Technical Summary
Current methods for viewing shared content on electronic devices require significant user input, leading to inefficiency and increased power consumption, particularly in battery-operated devices.
Electronic devices with touch-sensitive displays and processors automatically identify and integrate shared content into relevant applications without additional user input, using selectable attribute affordances to link back to the source and highlight content of interest.
This approach reduces user input requirements, enhances discoverability of shared content, and conserves power by minimizing unnecessary device operations.
Smart Images

Figure 0007844609000001 
Figure 0007844609000002 
Figure 0007844609000003
Abstract
Description
Technical Field
[0001] (Related Application) This application is a continuation application of U.S. Patent Application No. 17 / 481,206, filed on September 21, 2021, titled "Devices, Methods, and Graphical User Interfaces for Automatically Providing Shared Content to Applications," which claims the benefit of U.S. Provisional Patent Application No. 63 / 189,654, filed on May 17, 2021.
[0002] This application is a continuation application of U.S. Patent Application No. 17 / 481,212, filed on September 21, 2021, titled "Devices, Methods, and Graphical User Interfaces for Automatically Providing Shared Content to Applications," which claims the benefit of U.S. Provisional Patent Application No. 63 / 189,654, filed on May 17, 2021.
[0003] (Technical Field) The present disclosure generally relates to electronic devices that share content between applications.
Background Art
[0004] In recent years, the personal use of electronic devices has increased dramatically. Examples of such electronic devices include smartphones, laptops, desktop computers, and wearable technology. Such devices are increasingly used for communication (e.g., instant messaging, email, or social media) and entertainment (e.g., music, television, podcasts, etc.). For example, smartphone users frequently receive communications containing content from various contacts, which may recommend content or otherwise identify content that the user might want to view or interact with later (for example, a user might receive a song recommendation from a friend and want to listen to the song after work). Current methods do not provide an efficient way for users to view this received content in a way that makes it easy to find later. For example, a user typically needs to open and read a message to discover that it contains shared content such as books, videos, photos, articles, podcasts, music, or links to web pages. These methods usually require a large amount of user input, posing a significant cognitive burden on the user, taking longer than necessary, and thereby wasting energy. This latter consideration is particularly important in battery-powered devices. [Overview of the project]
[0005] Therefore, there is a need for electronic devices with faster and more efficient methods and interfaces to improve the discoverability of shared content without requiring additional user input. Such methods and interfaces reduce the number, range, and / or types of user input, resulting in a more efficient human-machine interface. With respect to battery-operated devices, such methods and interfaces conserve power and extend the time between battery charges.
[0006] The drawbacks and other problems associated with the user interface of electronic devices described above are mitigated or eliminated by the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device such as a watch). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a “touchscreen” or “touchscreen display”). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger touch and gestures on the touch-sensitive surface. In some embodiments, the functions are optional and include image editing, drawing, presentation, word processing, spreadsheet creation, gameplay, making phone calls, video conferencing, sending emails, instant messaging, training support, digital photography, digital videography, web browsing, digital music playback, note-taking, and / or digital video playback. The executable instructions for performing those functions are optional and may be contained in a non-temporary computer-readable storage medium or in other computer program products configured to be executed by one or more processors.
[0007] According to some embodiments, the method is performed on an electronic device communicating with a display generation component and one or more input devices. The method includes receiving information identifying a content item as shared content by a messaging application among a plurality of applications, and, in response to receiving the information identifying the content item as shared content and in accordance with the determination that the content item is of a first type, automatically making the content item available in a first application among the plurality of applications that is associated with content of the first type. The method further includes automatically making the content item available in a second application among the plurality of applications that is associated with content of the second type, in response to receiving information identifying the content item as shared content and in accordance with the determination that the content item is of a second type.
[0008] According to some embodiments, the method is performed on an electronic device communicating with a display generation component and one or more input devices. The method includes receiving information from a messaging application among a plurality of applications that identifies a content item of a first type as shared content. The method further includes making the content item automatically available within the first application in response to receiving the information that identifies the content item as shared content, in accordance with a determination that the first application among the plurality of applications can open the content of the first type, and in accordance with a determination that it is permitted to provide the information that identifies the content item as shared content to the first application. The method further includes not making the content item automatically available within the first application in response to receiving the information that identifies the content item as shared content, in accordance with a determination that the first application among the plurality of applications can open the content of the first type, and in accordance with a determination that it is not permitted to provide the information that identifies the content item as shared content to the first application.
[0009] According to some embodiments, the method is performed on an electronic device communicating with a display generation component and one or more input devices. The method includes displaying a view of a first application among a plurality of applications, while displaying a representation of a content item and selectable attribute affordances that provide a link to a location in a second application where information identifying the content item as shared content is received.
[0010] According to some embodiments, the method is performed on an electronic device communicating with a display generation component and one or more input devices. The method includes displaying a representation of a content item in a first application of a plurality of applications. The method further includes receiving input to select a representation of a content item while displaying a representation of a content item in the first application, and, in response to receiving input to select a representation of a content item, activating controls for highlighting a first type of content item in a second application of the plurality of applications, which is associated with a first type of content.
[0011] According to some embodiments, the electronic device includes a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, one or more processors, and a memory storing one or more programs, the one or more programs being configured to be executed by the one or more processors, and the one or more programs including instructions to perform or cause to perform any of the operations described herein. According to some embodiments, a computer-readable storage medium stores instructions internally, and when executed by an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators, the instructions cause the device to perform or cause to perform any of the operations described herein. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, memory, and one or more processors for executing one or more programs stored in memory includes one or more elements displayed in any of the methods described herein, the elements being updated in response to input as described in any of the methods described herein. According to some embodiments, the electronic device includes a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, and means for performing or causing to perform any of the operations described herein. According to some embodiments, an information processing device for use in an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators includes means for performing or causing to perform any of the operations described herein.
[0012] Thus, an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, optionally one or more device orientation sensors, and optionally an audio system provides an improved method and interface for automatically providing shared content to applicable applications, thereby enhancing the effectiveness, efficiency, and user satisfaction of such a device. [Brief explanation of the drawing]
[0013] To better understand the various embodiments described, the following “Modes for Carrying Out the Invention” should be referenced in conjunction with the following drawings, and similar reference numbers throughout the following drawings refer to the corresponding parts.
[0014] [Figure 1A] This is a block diagram showing a portable multifunctional device having a touch-sensitive display, according to several embodiments.
[0015] [Figure 1B] This is a block diagram showing exemplary components for event handling according to several embodiments.
[0016] [Figure 2] Several embodiments of a portable multifunctional device having a touchscreen are shown.
[0017] [Figure 3] This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface, according to several embodiments.
[0018] [Figure 4A] The following are exemplary user interfaces for application menus on a portable multifunction device, according to several embodiments.
[0019] [Figure 4B] An exemplary user interface for a multifunctional device having a touch sensing surface separated from a display, according to some embodiments, is shown.
[0020] [Figure 5A1] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A2] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A3] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A4] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A5] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A6] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A7] An exemplary user interface for sharing music media content received in a messaging application with a music media application, according to some embodiments, is shown. [Figure 5A8]An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5B1] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5B2] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5B3] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5C1] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5C2] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5C3] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5C4] An exemplary user interface for sharing music media content received in a messaging application with a music media application according to some embodiments is shown. [Figure 5D1]This document illustrates an exemplary user interface for sharing music media content received in a messaging application with a music media application, according to several embodiments. [Figure 5D2] This document illustrates an exemplary user interface for sharing music media content received in a messaging application with a music media application, according to several embodiments. [Figure 5D3] This document illustrates an exemplary user interface for sharing music media content received in a messaging application with a music media application, according to several embodiments. [Figure 5D4] This document illustrates an exemplary user interface for sharing music media content received in a messaging application with a music media application, according to several embodiments. [Figure 5D5] This document illustrates an exemplary user interface for sharing music media content received in a messaging application with a music media application, according to several embodiments.
[0021] [Figure 5E1] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E2] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E3] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E4] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E5]The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E6] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E7] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments. [Figure 5E8] The following are exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments.
[0022] [Figure 5F1] Several embodiments illustrate exemplary user interfaces for eliminating ambiguity. [Figure 5F2] Several embodiments illustrate exemplary user interfaces for eliminating ambiguity.
[0023] [Figure 5G1A] This document presents an exemplary user interface for sharing received media content in a group message thread, according to several embodiments. [Figure 5G1B] This document presents an exemplary user interface for sharing received media content in a group message thread, according to several embodiments. [Figure 5G2] This document presents an exemplary user interface for sharing received media content in a group message thread, according to several embodiments. [Figure 5G3] This document presents an exemplary user interface for sharing received media content in a group message thread, according to several embodiments. [Figure 5G4]This document presents an exemplary user interface for sharing received media content in a group message thread, according to several embodiments.
[0024] [Figure 5H] This section illustrates an exemplary search user interface for displaying shared and highlighted content, according to several embodiments.
[0025] [Figure 5I1] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I2] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I3] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I4] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I5] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I6] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I7] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I8] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I9] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown. [Figure 5I10] Examples of various media applications that display shared content in a messaging application, according to several embodiments, are shown.
[0026] [Figure 6A] The following are exemplary user interfaces for initializing the sharing function according to several embodiments. [Figure 6B] The following are exemplary user interfaces for initializing the sharing function according to several embodiments. [Figure 6C] The following are exemplary user interfaces for initializing the sharing function according to several embodiments. [Figure 6D] The following are exemplary user interfaces for initializing the sharing function according to several embodiments.
[0027] [Figure 7A] The following are exemplary user interfaces for enabling and / or disabling shared media integration functionality for multiple applications, according to several embodiments. [Figure 7B] The following are exemplary user interfaces for enabling and / or disabling shared media integration functionality for multiple applications, according to several embodiments. [Figure 7C] The following are exemplary user interfaces for enabling and / or disabling shared media integration functionality for multiple applications, according to several embodiments. [Figure 7D] The following are exemplary user interfaces for enabling and / or disabling shared media integration functionality for multiple applications, according to several embodiments.
[0028] [Figure 8A]This is a flowchart illustrating a process for automatically making content shared in one or more messaging applications available within corresponding media applications, based on the content type, according to several embodiments. [Figure 8B] This is a flowchart illustrating a process for automatically making content shared in one or more messaging applications available within corresponding media applications, based on the content type, according to several embodiments. [Figure 8C] This is a flowchart illustrating a process for automatically making content shared in one or more messaging applications available within corresponding media applications, based on the content type, according to several embodiments. [Figure 8D] This is a flowchart illustrating a process for automatically making content shared in one or more messaging applications available within corresponding media applications, based on the content type, according to several embodiments.
[0029] [Figure 9A] This is a flowchart illustrating a process for automatically displaying shared content in a media application based on whether the sharing function is enabled, according to several embodiments. [Figure 9B] This is a flowchart illustrating a process for automatically displaying shared content in a media application based on whether the sharing function is enabled, according to several embodiments. [Figure 9C] This is a flowchart illustrating a process for automatically displaying shared content in a media application based on whether the sharing function is enabled, according to several embodiments. [Figure 9D] This is a flowchart illustrating a process for automatically displaying shared content in a media application based on whether the sharing function is enabled, according to several embodiments. [Figure 9E] This is a flowchart illustrating a process for automatically displaying shared content in a media application based on whether the sharing function is enabled, according to several embodiments. [Figure 9F] This is a flowchart illustrating a process for automatically displaying shared content in a media application based on whether the sharing function is enabled, according to several embodiments.
[0030] [Figure 10A] This is a flowchart of a process for displaying information about how content items were shared, according to several embodiments. [Figure 10B] This is a flowchart of a process for displaying information about how content items were shared, according to several embodiments. [Figure 10C] This is a flowchart of a process for displaying information about how content items were shared, according to several embodiments. [Figure 10D] This is a flowchart of a process for displaying information about how content items were shared, according to several embodiments. [Figure 10E] This is a flowchart of a process for displaying information about how content items were shared, according to several embodiments.
[0031] [Figure 11A] This is a flowchart of a process for highlighting the representation of a content item so that the highlighted representation of the content item can be used in another application, according to several embodiments. [Figure 11B] This is a flowchart of a process for highlighting the representation of a content item so that the highlighted representation of the content item can be used in another application, according to several embodiments. [Figure 11C]This is a flowchart of a process for highlighting the representation of a content item so that the highlighted representation of the content item can be used in another application, according to several embodiments. [Figure 11D] This is a flowchart of a process for highlighting the representation of a content item so that the highlighted representation of the content item can be used in another application, according to several embodiments. [Modes for carrying out the invention]
[0032] Many electronic devices have one or more communication applications and several other applications that provide different functions, including, but not limited to, entertainment, shopping, business, productivity, navigation, web browsing, and other types of functions. Users may frequently receive communications in one or more communication applications that include several different recommendations for various types of content. For example, different contacts of a user may send / share copies or links to music, books, photos, locations, podcasts, websites, news or articles, applications, etc. Current methods for viewing, listening to, playing, or otherwise consuming shared content require a series of user inputs that are navigated through various user interfaces (for example, a user may need to scroll through several screens of an incoming message). For example, a user may be required to open a communication application, individually search and open each conversation thread, review and search each message or communication contained therein, and individually select each link or copy of the content in order to interact with content shared by the user's contacts. Furthermore, these methods may require the user to make another series of inputs to respond to the contact and / or message containing the selected content after the user has selected the content and the electronic device has displayed the content. For example, a user may be required to reopen the messaging application, search the conversation and / or messages to find the location where the content was shared, and then respond at that location. This cumbersome process is repeated for each piece of shared content. Furthermore, if the same content is shared by two or more contacts, the user may be required to follow the same inefficient method of searching the messaging application, conversation threads, and messages to discover that a particular piece of content was shared by multiple people. Moreover, the user may have no way of knowing whether two or more contacts shared the same content until the search is started and / or completed.Therefore, discovering and consuming shared content using current methods can be cumbersome and inefficient.
[0033] In the embodiments described below, an improved method for discovering (or integrating) shared content into a device ecosystem is achieved by making different types of content shared in a messaging application automatically available (e.g., without requiring additional user input) within other applications where that type of content is typically served, rendered, viewed, played, or listened to. Furthermore, in the embodiments described below, an improved method for discovering and connecting to the source of a share (e.g., the content sender) is achieved by displaying selectable attribute affordances within a first application (e.g., a music, photo, or video application) that provide a link to the location within the communication application (e.g., a messaging application) where the content was shared. The selectable attribute affordances provide within the first application a mechanism for navigating to the location within the first application where information identifying the shared content item was first received. Furthermore, the selectable attribute affordances indicate who the contacts who shared the content are, or, if there are more than two, the number of contacts who shared the same content. Furthermore, improved methods for highlighting or pinning shared content of interest are achieved by providing user interface controls within the communication application for highlighting shared content in other applications. These improved methods provide an efficient way to display shared content within the corresponding application where it is typically consumed, thus reducing the number of user inputs required, as well as the corresponding cognitive burden that may arise from having to navigate through different applications and menu hierarchies.
[0034] In the following, Figures 1A, 1B, 2, and 3 provide a description of exemplary devices. Figures 4A-4B, 5A1-5D5, 5E1-5E8, 5F1-5F2, 5G1-5G4, 5H, 5I1-5I10, 6A-6D, and 7A-7D show exemplary user interfaces for automatically providing shared content to appropriate or applicable applications. Figures 8A-8D show flowcharts of methods for automatically making shared content in one or more messaging applications available in corresponding media applications based on the content type, according to several embodiments. Figures 9A-9F show flowcharts of methods for automatically displaying shared content in a media application based on whether sharing functionality is enabled. Figures 10A-10E show flowcharts of methods for displaying information about how a content item was shared. Figures 11A to 11D show flowcharts illustrating how to highlight content item representations so that the highlighted representations of content items are available in other applications. The user interfaces in Figures 5A1 to 5D5, 5E1 to 5E8, 5F1 to 5F2, 5G1 to 5G4, 5H, 5I1 to 5I10, 6A to 6D, and 7A to 7D are used to illustrate the processes in Figures 8A to 8D, 9A to 9F, 10A to 10E, and 11A to 11D. Example device
[0035] Herein, a detailed reference is made to embodiments shown in the accompanying drawings. The following detailed description includes numerous specific details to provide a complete understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other examples, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the aspects of the embodiments.
[0036] In this specification, terms such as "first," "second," etc., are used to describe various elements in several embodiments, but it will be understood that these elements should not be limited by those terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first and second contacts are contacts, but they are not the same contact unless the context explicitly indicates otherwise.
[0037] The terminology used in the descriptions of the various embodiments described herein is intended solely to describe specific embodiments and is not intended to be limiting. In the descriptions of the various embodiments and the accompanying claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless otherwise explicitly stated in the context. Furthermore, it should be understood that, as used herein, the term “and / or” refers to and includes any and all possible combinations of one or more of the enumerated items relating to the description. It will be further understood that, as used herein, the terms “includes,” “comprises,” and / or “comprising” specify the presence of the described features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.
[0038] Where used herein, the term "if" is interpreted, at its discretion and in context, to mean "when," "upon," "in response to determining," or "in response to detecting." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" are interpreted, at its discretion and in context, to mean "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]."
[0039] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, which also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptop or tablet computers having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad), are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device but a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad).
[0040] The following discussion describes electronic devices including displays and touch-sensitive surfaces. However, please understand that the definition of an electronic device is optional and may include one or more other physical user interface devices such as a physical keyboard, mouse, and / or joystick.
[0041] The device generally supports a variety of applications, including one or more of the following: note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.
[0042] Various applications running on this device optionally utilize at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, are optionally adjusted and / or modified on an application-by-application basis and / or within individual applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally supports a variety of applications with intuitive and transparent user interfaces for the user.
[0043] Here, we turn our attention to embodiments of portable devices having a touch-sensitive display. Figure 1A is a block diagram of a portable multifunction device 100 having a touch-sensitive display system 112 according to several embodiments. The touch-sensitive display system 112 may be referred to as a “touchscreen” for convenience, or simply as a touch-sensitive display. The device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (for example, a touch-sensitive surface such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 that generate tactile outputs on Device 100 (for example, on touch-sensitive surfaces such as the touch-sensitive display system 112 of Device 100 or the touchpad 355 of Device 300). These components optionally communicate via one or more communication buses or signal lines 103.
[0044] As used herein and in the claims, the term “tactile output” refers to the physical displacement of a device relative to its previous position, the physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or the displacement of a component relative to the center of mass of a device, which will be detected by the user through the user’s sense of touch. For example, in a situation where a device or component of a device is in contact with a touch-sensitive user’s surface (e.g., the user’s fingers, palm, or other part of their hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) may be optionally interpreted by the user as a “down-click” or “up-click” of a physical actuator button. In some cases, the user may perceive a tactile sensation such as a “down-click” or “up-click” even when there is no movement of a physical actuator button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user’s action. In another embodiment, movement of a touch-sensitive surface may be optionally interpreted or perceived by the user as "roughness" of the touch-sensitive surface, even if there is no change in the smoothness of the touch-sensitive surface. Such user interpretations of touch depend on the user's personal sensory perception, but there are many touch sensory perceptions common to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., "up-click," "down-click," "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the device or its components that produce the described sensory perception of a typical (or average) user.Providing haptic feedback to the user using tactile output improves device usability and makes the user-device interface more efficient (for example, by helping the user provide appropriate input when operating / interacting with the device and reducing user errors), and in addition, reduces power consumption and improves the device's battery life by enabling the user to use the device more quickly and efficiently.
[0045] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the motion waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.
[0046] When a device generates tactile outputs with various different tactile output patterns (for example, via one or more tactile output generators that move a movable mass to generate a tactile output), the tactile output can evoke various different tactile sensations when the user holds or touches the device. While the user's sensation is based on their perception of the tactile output, most users are able to identify changes in the waveform, frequency, and amplitude of the tactile output generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different actions are being performed. Therefore, in some situations, tactile outputs having tactile output patterns designed, selected, and / or developed to simulate the properties (e.g., size, material, weight, stiffness, smoothness, etc.), behavior (e.g., vibration, displacement, acceleration, rotation, expansion, etc.), and / or interactions (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or any combination of the above), provide the user with useful feedback that reduces input errors and increases the efficiency of the user's operation of the device. In addition, tactile outputs are optionally generated to correspond to feedback unrelated to simulated physical properties, such as input thresholds or object selection. In some situations, such tactile outputs provide the user with useful feedback that reduces input errors and increases the efficiency of the user's operation of the device.
[0047] In some embodiments, tactile output having a suitable tactile output pattern acts as a signal for the occurrence of a target event behind a scene in the user interface or device. Examples of target events include the activation of affordances provided on the device or within the user interface (e.g., physical buttons, virtual buttons, or toggle switches), the success or failure of a requested action, reaching or crossing a boundary within the user interface, entering a new state, switching the focus of input between objects, activating a new mode, reaching or exceeding an input threshold, and detecting or recognizing the type of input or gesture. In some embodiments, tactile output is provided to serve as a warning or alert for an upcoming event or result that will occur unless a redirection or interruption input is detected in a timely manner. Tactile output is also used in other contexts to improve the user experience, improve the accessibility of the device for users with visual or motor impairments or other accessibility needs, and / or improve the efficiency and functionality of the user interface and / or device. Tactile output, optionally accompanied by audio output and / or visible user interface changes, further enhances the user experience when the user interacts with the user interface and / or device, facilitates better communication of information regarding the state of the user interface and / or device, reduces input errors, and increases the efficiency of user operation of the device.
[0048] It should be understood that device 100 is merely an example of a portable multifunction device, and that device 100 may optionally have more or fewer components than those shown, may optionally be a combination of two or more components, or may optionally have different configurations or arrangements of those components. The various components shown in Figure 1A are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.
[0049] Memory 102 optionally includes high-speed random-access memory and optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by CPU(s) 120 and other components of device 100 such as peripheral interface 118 is optionally controlled by memory controller 122.
[0050] A peripheral device interface 118 is used to connect the device's input and output peripherals to the CPU(s) 120 and memory 102. One or more processors 120 operate or execute various software programs and / or instruction sets stored in memory 102 to perform various functions for device 100 and process data.
[0051] In some embodiments, the peripheral interface 118, the CPU(s) 120, and the memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.
[0052] The RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, which include, but are not limited to, antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC chipsets, subscriber identity module (SIM) cards, and memory. The RF circuit 108 optionally communicates wirelessly with networks such as the Internet, also known as the World Wide Web (WWW), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), as well as with other devices. Wireless communication is optional and includes Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), and Code Division Multiple Access (W-CDMA).Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), Instant Messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE)), Instant Messaging and Presence Services Using any of several communication standards, communication protocols, and communication technologies, including, but not limited to, a Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including a communication protocol not yet developed as of the filing date of this document.
[0053] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts this audio data into an electrical signal, and transmits this electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits this audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or RF circuit 108 by the peripheral interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., 212, Figure 2). The headset jack provides an interface between the audio circuit 110 and detachable audio input / output peripherals such as output-only headphones or headsets that have both output (e.g., headphones for one or both ears) and input (e.g., a microphone).
[0054] The I / O subsystem 106 connects input / output peripherals on device 100, such as the touch-sensitive display system 112 and other input or control devices 116, to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from and transmit electrical signals to the other input or control devices 116. The other input or control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) 160 are optionally connected to (or not connected to) any of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., 208, Figure 2) optionally include up / down buttons for controlling the volume of speaker 111 and / or microphone 113. One or more buttons optionally include push buttons (e.g., 206, Figure 2).
[0055] The touch-sensitive display system 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays a visual output to the user. This visual output optionally includes graphics, text, icons, animations, and any combination thereof (collectively, “graphics”). In some embodiments, some or all of the visual output corresponds to user interface objects. As used herein, the term “affordance” refers to a user-interactive graphical user interface object (for example, a graphical user interface object configured to respond to input directed toward the graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.
[0056] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or set of sensors that accept user input based on tactile and / or haptic contact. The touch-sensitive display system 112 and the display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of contact) on the touch-sensitive display system 112 and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.
[0057] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light-emitting polymer display) technology, or LED (light-emitting diode) technology, but other display technologies are used in other embodiments. The touch-sensitive display system 112 and the display controller 156 optionally use any of several currently known or future-developed touch-sensing technologies, including but not limited to capacitive technology, resistive technology, infrared technology, and surface acoustic wave technology, as well as other proximity sensor arrays or other elements for determining one or more contact points with the touch-sensitive display system 112, to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as that found in iPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, California.
[0058] The touch-sensitive display system 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the video resolution of the touchscreen is greater than 400 dpi (e.g., 500 dpi, 800 dpi, or higher). The user optionally touches the touch-sensitive display system 112 using any suitable object or attachment such as a stylus or finger. In some embodiments, the user interface is designed to function with finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of a finger on the touchscreen than that of a stylus. In some embodiments, the device translates coarse finger input into a precise pointer / cursor position or command to perform an action desired by the user.
[0059] In some embodiments, in addition to the touchscreen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touchscreen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.
[0060] Device 100 also includes a power system 162 that supplies power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with generating, managing, and distributing power within the portable device.
[0061] The device 100 also optionally includes one or more optical sensors 164. Figure 1A shows an optical sensor coupled with an optical sensor controller 158 in the I / O subsystem 106. The optical sensor(s) 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor(s) 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with the imaging module 143 (also called the camera module), the optical sensor(s) 164 optionally captures still images and / or video. In some embodiments, the optical sensor is located on the back of the device 100, opposite the touch-sensitive display system 112 on the front of the device, so that the touchscreen can be used as a viewfinder for acquiring still images and / or video images. In some embodiments, a separate light sensor is positioned on the front of the device so that an image of the user can be captured (for example, for selfies, or for video conferencing while the user is viewing other video conference participants on the touchscreen).
[0062] Device 100 also optionally includes one or more contact strength sensors 165. Figure 1A shows a contact strength sensor coupled with a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor(s) 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, pressure-power sensors, optical force sensors, capacitive touch-sensing surfaces, or other strength sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensing surface). The contact strength sensor(s) 165 receives contact strength information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact strength sensor is positioned juxtaposed with or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112). In some embodiments, at least one contact strength sensor is located on the back of Device 100, opposite the touchscreen display system 112 which is located on the front of Device 100.
[0063] The device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows a proximity sensor 166 coupled to a peripheral interface 118. Alternatively, the proximity sensor 166 is coupled to an input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is positioned near the user's ear (for example, when the user is making a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.
[0064] Device 100 also optionally includes one or more tactile output generators 167. Figure 1A shows a tactile output generator coupled with a tactile feedback controller 161 in the I / O subsystem 106. In some embodiments, the tactile output generator(s) 167 includes one or more electroacoustic devices such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). The tactile output generator(s) 167 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output on the device 100 that can be sensed by the user of the device 100. In some embodiments, at least one tactile output generator is positioned alongside or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112) and optionally generates a tactile output by moving the touch-sensing surface vertically (e.g., inward / outward from the surface of device 100) or horizontally (e.g., forward / backward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is positioned on the back of device 100, opposite the touch-sensing display system 112 which is located on the front of device 100.
[0065] The device 100 also optionally includes one or more accelerometers 168. Figure 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in a portrait or landscape view based on an analysis of data received from one or more accelerometers. The device 100 optionally includes, in addition to the accelerometers 168 (one or more), a magnetometer (not shown), and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the location and orientation of the device 100 (e.g., portrait or landscape).
[0066] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, as shown in Figures 1A and 3, memory 102 stores device / global internal state 157. The device / global internal state 157 includes one or more of the following: an active application state indicating which application is active, if there is an application currently active; a display state indicating which applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state including information obtained from various sensors and other input or control devices 116 of the device; and location and / or positional information relating to the location and / or orientation of the device.
[0067] An operating system (for example, an embedded operating system such as iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks) includes various software components and / or drivers for controlling and managing overall system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
[0068] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to connect to other devices directly or indirectly via a network (e.g., the Internet, Wi-Fi, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors identical or similar to and / or compatible with the 30-pin connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external ports are Lightning connectors identical or similar to and / or compatible with the Lightning connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California.
[0069] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in cooperation with the display controller 156) and contact with other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger down event), determining the intensity of contact (e.g., the force or pressure of contact, or an alternative to the force or pressure of contact), determining whether there is movement of contact and tracking movement across the touch-sensitive surface (e.g., detecting a drag event of one or more fingers), and determining whether contact has stopped (e.g., detecting a finger up event or interruption of contact). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point, represented by a series of contact data, optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions can optionally be applied to a single contact (e.g., a single finger or stylus contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.
[0070] The contact / motion module 130 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motion, timing, and / or intensity of the detected contact). Therefore, gestures are optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture involves detecting a finger down event, followed by a finger up (lift-off) event at the same position (or substantially the same position) as the finger down event (e.g., at the icon's position). In another embodiment, detecting a finger swipe gesture on the touch-sensitive surface involves detecting a finger down event, followed by a drag event of one or more fingers, and then a finger up (lift-off) event. Similarly, taps, swipes, drags, and other gestures are optionally detected with respect to the stylus by detecting a specific contact pattern with respect to the stylus.
[0071] In some embodiments, detecting a finger tap gesture depends on the length of time between detecting a finger down event and detecting a finger up event, but not on the intensity of finger contact between the two events. In some embodiments, a tap gesture is detected according to the determination that the length of time between the finger down event and the finger up event is shorter than a predetermined value (e.g., shorter than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of finger contact between taps meets a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light or deep pressure intensity threshold. Thus, a finger tap gesture can satisfy certain input criteria that do not require the characteristic intensity of contact to meet a given intensity threshold in order for certain input criteria to be met. To clarify, finger contact in a tap gesture generally needs to meet a nominal contact detection intensity threshold below which contact is not detected in order to detect a finger down event. A similar analysis applies to detecting tap gestures or other contacts by a stylus. In cases where the device can detect contact from a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold is optional and does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.
[0072] In a similar manner, the same concept applies to other types of gestures. For example, swipe gestures, pinch gestures, de-pinch gestures, and / or long press gestures are optional and are detected based on whether they meet criteria that are either unrelated to the intensity of the contacts involved in the gesture, or do not require the contact(s) performing the gesture to reach an intensity threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts. A pinch gesture is detected based on the movement of two or more contacts toward each other. A de-pinch gesture is detected based on the movement of two or more contacts toward each other. A long press gesture is detected based on the duration of contact with a touch-sensitive surface involving movement less than a threshold amount. Therefore, the statement that a particular gesture recognition criterion does not require the intensity of one or more contacts to meet individual intensity thresholds for that criterion to be satisfied means that a particular gesture recognition criterion can be satisfied when one or more contacts in a gesture do not reach individual intensity thresholds, and can also be satisfied when one or more of the contacts in a gesture reach or exceed individual intensity thresholds. In some embodiments, a tap gesture is detected based on the determination that finger-down and finger-up events were detected within a predetermined time period, regardless of whether the contacts are above or below individual intensity thresholds during a predetermined time period, and a swipe gesture is detected based on the determination that the movement of the contacts is greater than a predetermined magnitude, even if the contacts exceed individual intensity thresholds at the end of the movement of the contacts. Even in embodiments where gesture detection is influenced by the intensity of the contact performing the gesture (for example, the device detects longer presses more quickly when the contact intensity exceeds an intensity threshold, or the device is slower to detect tap inputs when the contact intensity is higher), the detection of those gestures does not require the contact to reach a specific intensity threshold, as long as the criteria for recognizing the gesture can be met in situations where the contact does not reach a specific intensity threshold (for example, even if the amount of time required to recognize the gesture changes).
[0073] Contact intensity thresholds, duration thresholds, and movement thresholds are combined in various different combinations to create heuristics for distinguishing two or more different gestures directed towards the same input element or region in certain situations, thereby enabling multiple different interactions with the same input element to provide a richer set of user interactions and responses. A description that a particular set of gesture recognition criteria does not require that the intensity of a contact(s) meets an individual intensity threshold for a particular gesture recognition criterion to be satisfied does not preclude the simultaneous evaluation of other intensity-dependent gesture recognition criteria for identifying other gestures that have criteria that are satisfied when the gesture includes contacts with an intensity exceeding an individual intensity threshold. For example, in some situations, a first gesture recognition criterion for a first gesture that does not require that the intensity of a contact(s) meet an individual intensity threshold for the first gesture recognition criterion to be satisfied is in competition with a second gesture recognition criterion for a second gesture that depends on contact(s) reaching an individual intensity threshold. In such competition, a gesture is optional and will not be recognized as satisfying the first gesture recognition criterion for the first gesture if the second gesture recognition criterion for the second gesture is first satisfied. For example, if a contact reaches a specific intensity threshold before it moves a predetermined amount, a deep press gesture will be detected instead of a swipe gesture. Conversely, if a contact moves a predetermined amount before it reaches a specific intensity threshold, a swipe gesture will be detected instead of a deep press gesture. Even in such situations, the first gesture recognition criterion for the first gesture still does not require the intensity of the contact(s) to meet a specific intensity threshold for the first gesture recognition criterion to be satisfied, because if the contact remains below the specific intensity threshold until the end of the gesture (e.g., a swipe gesture with a contact that does not increase to an intensity above the specific intensity threshold), the gesture is recognized as a swipe gesture by the first gesture recognition criterion.In this way, a particular gesture recognition criterion that does not require the intensity of a single or multiple contact to satisfy a separate intensity threshold for that particular gesture recognition criterion to be satisfied still depends on the intensity of the contact with respect to the intensity threshold, in the sense that (A) in some situations it ignores the intensity of the contact with respect to the intensity threshold (e.g., for a tap gesture), and / or (B) in some situations the particular gesture recognition criterion (e.g., for a long press gesture) does not function if a competing set of intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-dependent gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input (e.g., for a long press gesture competing with a deep press gesture for recognition).
[0074] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other display, including components for modifying the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual characteristics). As used herein, the term “graphics” includes, but is not limited to, any object that can be displayed to a user, including, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, and animations.
[0075] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like, as needed, specifying the graphics to be displayed, along with coordinate data and other graphic property data, and then generates screen image data to output to the display controller 156.
[0076] The haptic feedback module 133 includes various software components that generate commands (for example, commands used by the haptic feedback controller 161) that create haptic outputs at one or more locations on the device 100 using a haptic output generator(s) 167, in response to user interaction with the device 100.
[0077] The text input module 134 is optionally a component of the graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other applications that require text input).
[0078] The GPS module 135 determines the device's location and provides this information for use in various applications (for example, to the phone 138 for location-based phone calls, to the camera 143 as metadata for photos / videos, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map / navigation widgets).
[0079] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: ● Contact module 137 (sometimes called the address book or contact list), ● Telephone module 138, ● Video conferencing module 139, ● Email client module 140, ● Instant messaging (IM) module 141, ● Training support module 142, ● Camera module 143 for still images and / or video images, ● Image management module 144, ● Browser module 147, ● Calendar module 148, ● A widget module 149 that optionally includes one or more of the following: weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, other widgets obtained by the user, and user-created widgets 149-6. ● Widget creation module 150 for creating user-created widget 149-6, ● Search module 151, ● A video and music player module 152, which optionally consists of a video player module and a music player module. ● Memo Module 153, ● Map module 154, and / or ● Online video module 155.
[0080] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.
[0081] Together with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 includes executable instructions for managing an address book or contact list (for example, stored in the application internal state 192 of the contact module 137 in memory 102 or memory 370), which include adding names(s) to the address book, deleting names(s) from the address book, associating telephone numbers(s) to names, email addresses(s) to names, physical addresses(s) to names, or other information; associating images to names; categorizing and sorting names; providing telephone numbers and / or email addresses to initiate and / or facilitate communication by telephone 138, video conferencing 139, email 140, or IM 141, etc.
[0082] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the telephone module 138 includes executable commands for inputting a series of characters corresponding to a telephone number, accessing one or more telephone numbers in the address book 137, changing the entered telephone number, dialing each telephone number, conducting a conversation, and disconnecting or ending the call when the conversation is complete. As described above, the wireless communication may optionally use any of several communication standards, protocols, and technologies.
[0083] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, the video conferencing module 139 includes executable commands to start, conduct, and end a video conference between the user and one or more other participants, in accordance with the user's commands.
[0084] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the email client module 140 includes executable commands for creating, sending, receiving, and managing emails in response to user commands. In conjunction with the image management module 144, the email client module 140 makes it extremely easy to create and send emails containing still or video images captured by the camera module 143.
[0085] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the instant messaging module 141 includes executable instructions for inputting a series of characters corresponding to an instant message, modifying previously entered characters, sending each instant message (for example, using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant messaging, or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).
[0086] In conjunction with the RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, text input module 134, GPS module 135, map module 154, and video and music player module 152, the training support module 142 includes executable commands for creating training (e.g., having time, distance, and / or calorie consumption targets), communicating with training sensors (in sports devices and smartwatches), receiving training sensor data, calibrating sensors used to monitor training, selecting and playing music for training, and displaying, storing, and transmitting training data.
[0087] Together with the touch-sensitive display system 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, and / or deleting still images or videos from memory 102.
[0088] Together with the touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, text input module 134, and camera module 143, the image management module 144 includes executable commands for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still and / or video images.
[0089] Together with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, the browser module 147 includes executable commands for browsing the internet in accordance with user commands, including searching, linking, receiving, and displaying web pages or parts thereof, as well as attachments and other files linked to web pages.
[0090] Together with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, the calendar module 148 includes executable instructions that, in accordance with user commands, create, display, modify, and store a calendar and data associated with the calendar (e.g., calendar items, to-do lists, etc.).
[0091] Together with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget module 149 is optionally a mini-application downloaded and used by the user (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a mini-application created by the user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).
[0092] In conjunction with the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget creation module 150 includes executable commands for creating widgets (for example, converting user-specified portions of a web page into widgets).
[0093] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions to search for text, music, sounds, images, videos, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) according to user commands.
[0094] In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music player module 152 includes executable commands that enable the user to download and play recorded music or other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable commands to display, present, or otherwise play video (for example, on the touch-sensitive display system 112, or on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).
[0095] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the memo module 153 includes executable commands that create and manage notes, to-do lists, and the like, according to user commands.
[0096] In relation to the RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 includes executable commands that, in accordance with user commands, receive, display, modify, and store maps and map-related data (e.g., driving directions, data about stores and other points of interest at a specific location or nearby, and other location-based data).
[0097] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the audio circuit 110, the speaker 111, the RF circuit 108, the text input module 134, the email client module 140, and the browser module 147, the online video module 155 includes executable instructions that enable the user to access, view, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen 112, or on an external display connected wirelessly or via the external port 124), send emails containing links to specific online videos, and otherwise manage them. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos.
[0098] Each of the modules and applications identified above corresponds to one or more of the functions described above, as well as an executable instruction set for performing the methods described in this application (e.g., methods performed by a computer and other information processing methods described herein). These modules (i.e., instruction sets) do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of these modules can be optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.
[0099] In some embodiments, device 100 is a device in which the operation of a default set of functions in the device is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for device 100 to operate, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.
[0100] A default set of functions, which are performed only through the touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to the main menu, home menu, or root menu. In such embodiments, a “menu button” is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.
[0101] Figure 1B is a block diagram showing exemplary components for event processing according to several embodiments. In some embodiments, memory 102 (in Figure 1A) or 370 (in Figure 3) includes an event sorting unit 170 (e.g., within the operating system 126) and each application 136-1 (e.g., any of the applications 136, 137-155, 380-390 described above).
[0102] The event sorting unit 170 receives event information and determines the application 136-1 that distributes the event information, and the application view 191 of application 136-1. The event sorting unit 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view(s) displayed on the touch-sensitive display system 112 when the application is active or running. In some embodiments, a device / global internal state 157 is used by the event sorting unit 170 to determine which application(s) are currently active, and the application internal state 192 is used by the event sorting unit 170 to determine the application view(s) to which the event information is distributed.
[0103] In some embodiments, the application internal state 192 includes additional information such as resume information to be used when the application 136-1 resumes execution, user interface state information that indicates or is ready to display information displayed by the application 136-1, a state queue that allows the user to return to a previous state or view of the application 136-1, and one or more redo / undo queues of previous actions performed by the user.
[0104] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (for example, a user touch on the touch-sensitive display system 112 as part of a multi-touch gesture). The peripheral interface 118 transmits information received from the I / O subsystem 106, or from sensors such as the proximity sensor 166, one or more accelerometers 168, and / or the microphone 113 (via the audio circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display system 112 or the touch-sensitive surface.
[0105] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when a significant event occurs (e.g., receiving input that exceeds a predetermined noise threshold and / or for a longer period than predetermined).
[0106] In some embodiments, the event sorting unit 170 also includes a hit view determination module 172 and / or an active event recognition unit determination module 173.
[0107] The hit view determination module 172 provides software procedures for determining where in one or more views a sub-event occurred when the touch-sensitive display system 112 displays two or more views. A view consists of control devices and other elements that the user can see on the display.
[0108] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of an individual application) in which a touch is detected optionally corresponds to the program level within the application's program hierarchy or view hierarchy. For example, the lowest-level view in which a touch is detected optionally refers to a hit view, and the set of events recognized as appropriate input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch gesture.
[0109] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views arranged in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchy from which sub-events should be processed. In most situations, the hit view is the lowest-level view from which the initiating sub-event (i.e., the first sub-event in a series of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source from which it was identified as a hit view.
[0110] The active event recognition determination module 173 determines which view(s) in the view hierarchy should receive a particular set of sub-events. In some embodiments, the active event recognition determination module 173 determines that only the hit view should receive a particular set of sub-events. In other embodiments, the active event recognition determination module 173 determines that all views, including the physical location of the sub-events, are actively involved views, and therefore all actively involved views should receive a particular set of sub-events. In other embodiments, even if the touch sub-event is entirely confined to an area associated with one particular view, higher-level views in the hierarchy still remain actively involved views.
[0111] The event dispatcher module 174 dispatches event information to an event recognition unit (e.g., an event recognition unit 180). In embodiments including an active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores the event information acquired by each event receiving unit module 182 in an event queue.
[0112] In some embodiments, the operating system 126 includes an event sorting unit 170. Alternatively, application 136-1 includes an event sorting unit 170. In yet another embodiment, the event sorting unit 170 is a standalone module or part of another module stored in memory 102, such as a contact / motion module 130.
[0113] In some embodiments, application 136-1 includes a plurality of event processing units 190 and one or more application views 191, each containing instructions for processing touch events occurring within a separate view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognition units 180. Typically, a separate application view 191 includes a plurality of event recognition units 180. In other embodiments, one or more of the event recognition units 180 are part of a separate module, such as a user interface kit (not shown) or a higher-level object from which application 136-1 inherits methods and other characteristics. In some embodiments, a separate event processing unit 190 includes one or more event data 179 received from a data update unit 176, an object update unit 177, a GUI update unit 178, and / or an event sorting unit 170. The event processing unit 190 optionally utilizes or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 to update the application's internal state 192. Alternatively, one or more of the application views 191 include one or more separate event processing units 190. Also, in some embodiments, one or more of the data update unit 176, object update unit 177, and GUI update unit 178 are included in the individual application view 191.
[0114] Each individual event recognition unit 180 receives event information (e.g., event data 179) from the event sorting unit 170 and identifies an event from the event information. The event recognition unit 180 includes an event receiving unit 182 and an event comparison unit 184. In some embodiments, the event recognition unit 180 also includes at least a subset of metadata 183 and event distribution commands 188 (optionally including sub-event distribution commands).
[0115] The event receiving unit 182 receives event information from the event sorting unit 170. The event information includes information about sub-events, such as touches or the movement of touches. Depending on the sub-event, the event information also includes additional information such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, an event includes a rotation of the device from one orientation to another (e.g., from portrait to landscape, or vice versa), and the event information includes corresponding information about the current orientation of the device (also called the device's orientation).
[0116] The event comparison unit 184 compares event information with a predefined definition of an event or sub-event, and based on the comparison, determines the event or sub-event, or determines or updates the state of the event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. The event definition 186 includes definitions of events (e.g., a default set of sub-events), such as event 1 (187-1) and event 2 (187-2). In some embodiments, sub-events in event 187 include, for example, touch start, touch cancellation, touch movement, touch abortion, and multiple touches. In one embodiment, the definition for event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch (touch start) for a predetermined stage on the displayed object, a first lift-off (touch end) for a predetermined stage, a second touch (touch start) for a predetermined stage on the displayed object, and a second lift-off (touch end) for a predetermined stage. In another embodiment, event 2(187-2) is defined as a drag on a displayed object. The drag includes, for example, touching (or contacting) a predetermined number of stages on the displayed object, moving the touch across the touch-sensitive display system 112, and lifting off the touch (end of the touch). In some embodiments, the event also includes information about one or more associated event processing units 190.
[0117] In some embodiments, the event definition 187 includes event definitions for individual user interface objects. In some embodiments, the event comparison unit 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, the event comparison unit 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with an individual event processing unit 190, the event comparison unit uses the results of the hit test to determine which event processing unit 190 should be activated. For example, the event comparison unit 184 selects the sub-event and the event processing unit associated with the object that trigger the hit test.
[0118] In some embodiments, the definition of each event 187 also includes a delay action that delays the delivery of event information until it is determined whether a series of sub-events correspond to the event type of the event recognition unit.
[0119] If an individual event recognition unit 180 determines that a series of sub-events does not match any of the events in the event definition 186, the individual event recognition unit 180 enters an event impossible, event failed, or event terminated state, and thereafter ignores subsequent sub-events of the touch-based gesture. In this situation, if there are other event recognition units that remain active for the hit view, those event recognition units continue to track and process the sub-events of the ongoing touch-based gesture.
[0120] In some embodiments, an individual event recognition unit 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate to the actively involved event recognition unit how the event distribution system should perform sub-event distribution. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how the event recognition units interact with each other, or how they can interact with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are distributed to various levels in the view hierarchy or program hierarchy.
[0121] In some embodiments, an individual event recognition unit 180 activates an event processing unit 190 associated with an event when one or more specific sub-events of an event are recognized. In some embodiments, the individual event recognition unit 180 delivers event information associated with the event to the event processing unit 190. Activating the event processing unit 190 is separate from sending (and delaying the sending of) sub-events to individual hit views. In some embodiments, the event recognition unit 180 sets a flag associated with the recognized event, and the event processing unit 190 associated with that flag captures the flag and executes a default process.
[0122] In some embodiments, the event distribution command 188 includes a sub-event distribution command that distributes event information about a sub-event without activating an event processing unit. Instead, the sub-event distribution command distributes event information to an event processing unit associated with a set of sub-events, or to an actively involved view. The event processing unit associated with the set of sub-events or the actively involved view receives the event information and executes a predetermined process.
[0123] In some embodiments, the data update unit 176 creates and updates data used in application 136-1. For example, the data update unit 176 updates telephone numbers used in contact module 137 or stores video files used in video and music player module 152. In some embodiments, the object update unit 177 creates and updates objects used in application 136-1. For example, the object update unit 177 creates new user interface objects or updates the positions of user interface objects. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares display information and sends it to graphics module 132 for display on touch-sensitive display.
[0124] In some embodiments, the event processing unit(s) 190 includes or has access to a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are contained in a single module of an individual application 136-1 or application view 191. In other embodiments, they are contained in two or more software modules.
[0125] The foregoing description regarding the handling of user touch events on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 using input devices, but it should be understood that not all of this begins on the touchscreen. For example, mouse movement and mouse button presses, touch movements such as taps, drags, and scrolls on a touchpad, pen stylus input, device movement, verbal commands, detected eye movements, biometric input, and / or any combination thereof may be optionally used as inputs corresponding to sub-events that define the events to be recognized.
[0126] Figure 2 shows a portable multifunctional device 100 having a touchscreen (e.g., a touch-sensitive display system 112, Figure 1A) according to several embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In these embodiments, and in embodiments described later, the user can select one or more graphics by making gestures on the graphics using, for example, one or more fingers 202 (not shown in the figure to an exact scale) or one or more styluses 203 (not shown in the figure to an exact scale). In some embodiments, the selection of one or more graphics is performed when the user interrupts contact with that one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, right to left, upward and / or downward), and / or rolling of a finger in contact with the device 100 (from right to left, left to right, upward and / or downward). In some implementations or situations, accidental contact with a graphic does not constitute a selection of that graphic. For example, if the gesture corresponding to selection is a tap, a swipe gesture sweeping over an application icon does not arbitrarily select the corresponding application.
[0127] Device 100 optionally also includes one or more physical buttons, such as a "Home" or menu button 204. As previously mentioned, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally run on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display.
[0128] In some embodiments, device 100 includes a touchscreen display, a menu button 204 (sometimes referred to as a home button 204), a push button 206 for turning the device on / off and locking the device, one or more volume buttons 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and an external docking / charging port 124. The push button 206 is optionally used to turn the device on / off by pressing down and holding the button down for a predetermined period of time, to lock the device by pressing down and releasing the button before a predetermined period of time has elapsed, and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts verbal input through a microphone 113 to activate or deactivate certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output to the user of device 100.
[0129] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to several embodiments. Device 300 does not need to be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), game system, or control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication buses 320 optionally include circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 typically includes an input / output (I / O) interface 330, which includes a display 340, which is a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350, a touchpad 355, a touch output generator 357 for generating tactile output on a device 300 (similar to the tactile output generator(s) 167 described above with reference to Figure 1A, for example), and a sensor 359 (e.g., an optical sensor, an accelerometer, a proximity sensor, a touch sensor, and / or a touch intensity sensor similar to the touch intensity sensor(s) 165 described above with reference to Figure 1A). The memory 370 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or subsets thereof, that are stored in memory 102 of the portable multifunction device 100 (Figure 1A). Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of the portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, a word processing module 384, a website creation module 386, a disk authoring module 388, and / or a spreadsheet module 390, while memory 102 of the portable multifunction device 100 (Figure 1A) optionally does not store those modules.
[0130] Each of the elements identified in Figure 3 above is optionally stored in one or more of the previously mentioned memory devices. Each of the modules identified above corresponds to an instruction set that performs the function described above. The modules or programs (i.e., instruction sets) identified above do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of those modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.
[0131] Here, we focus on embodiments of a user interface ("UI") that are optionally implemented on the portable multi-functional device 100.
[0132] Figure 4A shows an exemplary user interface for an application menu on a portable multifunction device 100 according to several embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or subsets or supersets thereof. ● Signal strength indicators (single or multiple) for wireless communication (single or multiple) such as cellular signals and Wi-Fi signals. ● Time, ● Bluetooth (registered trademark) indicator, ● Battery status indicator, ● Tray 408 containing icons for frequently used applications, as shown below. ○ An icon 416 of the telephone module 138, labeled "Telephone," optionally including an indicator 414 for the number of missed calls or voicemail messages. ○ An icon 418 of the email client module 140, labeled "Mail," optionally including an indicator 410 of the number of unread emails. ○ Icon 420 of browser module 147, labeled "Browser", and ○ Icon 422 of the video and music player module 152 labeled "Music", and ● Icons of other applications, such as the following: ○ Icon 424 of IM module 141, labeled "Message", ○ Icon 426 of calendar module 148, labeled "Calendar" ○ Icon 428 of image management module 144, labeled "Photo" ○ Icon 430 of camera module 143, labeled "Camera" ○ Icon 432 of online video module 155, labeled "online video" ○ Icon 434 of stock price widget 149-2, labeled "Stock Price" ○ Icon 436 of map module 154, labeled "Map" ○ Icon 438 of weather widget 149-1, labeled "Weather" ○ Icon 440 of the alarm clock widget 149-4, labeled "Clock", ○ Icon 442 of training support module 142, labeled "Training Support" ○ Icon 444 of memo module 153, labeled as "Memo", and ○ An icon 446 for a configuration application or module that provides access to settings related to device 100 and its various applications 136.
[0133] Please note that the icon labels shown in Figure 4A are merely examples. For example, other labels may be used selectively for various application icons. In some embodiments, the labels for individual application icons include the name of the application to which the individual application icon corresponds. In some embodiments, the label for a particular application icon is different from the name of the application to which that particular application icon corresponds.
[0134] Figure 4B shows an exemplary user interface on a device (e.g., device 300, Figure 3) having a touch-sensitive surface 451 (e.g., tablet or touchpad 355, Figure 3) separate from the display 450. Many of the following embodiments are given by reference to input on the touchscreen display 112 (when the touch-sensitive surface and the display are combined), but in some embodiments, the device detects input on the touch-sensitive surface separate from the display, as shown in Figure 4B. In some embodiments, the touch-sensitive surface (e.g., 451 in Figure 4B) has a principal axis (e.g., 452 in Figure 4B) corresponding to a principal axis (e.g., 453 in Figure 4B) on the display (e.g., 450). According to those embodiments, the device detects contact (e.g., 460 and 462 in Figure 4B) with the touch-sensitive surface 451 at positions corresponding to each location on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in Figure 4B). Thus, when the touch-sensitive surface is separated from the display, user input detected by the device on the touch-sensitive surface (e.g., 451 in Figure 4B) (e.g., touches 460 and 462, and their movement) is used by the device to operate the user interface on the display of the multifunction device (e.g., 450 in Figure 4B). It should be understood that a similar method may be optionally used for other user interfaces described herein.
[0135] In addition, while the following embodiments are given primarily with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture, etc.), it should be understood that in some embodiments, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of touch), followed by a mouse click with cursor movement along the swipe path (e.g., instead of touch movement). As another example, a tap gesture may optionally be replaced by a mouse click (e.g., instead of touch detection and subsequent cessation of touch detection) while the cursor is located over the tap gesture location. Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice may optionally be used simultaneously, or mice and finger touch may optionally be used simultaneously.
[0136] As used herein, the term “focus selector” refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations, including a cursor or other location marker, the cursor functions as a “focus selector” such that when input (e.g., press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in Figure 3, or touch-sensitive surface 451 in Figure 4B) while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element is adjusted according to the detected input. In some implementations, including a touchscreen display (e.g., touch-sensitive display system 112 in Figure 1A, or touchscreen in Figure 4A) that enables direct interaction with user interface elements on the touchscreen display, contact detected on the touchscreen functions as a “focus selector” such that when input (e.g., press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of the user interface to another without the movement of a corresponding cursor or touch on the touchscreen display (for example, by moving focus from one button to another using the tab key or arrow keys). In these implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally a user-controlled user interface element (or touch on the touchscreen display) to communicate the user's intended interaction with the user interface (for example, by indicating to the device the user interface element that the user intends to interact with).For example, the position of a focus selector (e.g., cursor, touch, or selection box) over an individual button while pressure input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that individual button (rather than other user interface elements displayed on the device's display).
[0137] In some embodiments, the device's response to an input detected by the device depends on a criterion based on the contact intensity during the input. For example, for some "light press" inputs, a contact intensity exceeding a first intensity threshold during the input triggers a first response. In some embodiments, the device's response to an input detected by the device depends on a criterion that includes both the contact intensity during the input and a time-based criterion. For example, for some "deep press" inputs, a contact intensity exceeding a second intensity threshold during an input greater than a first intensity threshold for light presses triggers a second response only if a delay time has elapsed between satisfying the first and second intensity thresholds. This delay time is typically shorter than 200 ms (milliseconds) (e.g., 40 ms, 100 ms, or 120 ms, depending on the magnitude of the second intensity threshold, and the delay time increases as the second intensity threshold increases). This delay time helps avoid accidental recognition of deep press inputs. As another example, for some "deep press" inputs, there is a period of reduced sensitivity that occurs after the time the first intensity threshold is satisfied. During the period of reduced sensitivity, the second intensity threshold increases. This temporary increase in the second intensity threshold also helps to avoid accidental deep press inputs. For other deep press inputs, the response to the detection of deep press inputs is independent of time-based criteria.
[0138] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, the application being run, the rate at which intensity is applied, the number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), and the position of the focus selector. Illustrative factors are described in U.S. Patent Publications 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety. User interface and related processes
[0139] Here, we draw our attention to embodiments of user interfaces ("UI") and related processes that may be implemented on an electronic device such as a portable multifunction device 100 or device 300, comprising a display, a touch-sensitive surface, one or more tactile output generators for (optionally) generating tactile output, and one or more sensors for (optionally) detecting the intensity of contact with the touch-sensitive surface.
[0140] Figures 5A1 to 5D5, 5E1 to 5E8, 5F1 to 5F2, 5G1 to 5G4, 5H, 5I1 to 5I10, 6A to 6D, and 7A to 7D illustrate exemplary user interfaces for automatically providing shared content to applicable applications according to several embodiments. The user interfaces in those figures are used to illustrate the processes described below, including the processes in Figures 8A to 8D, 9A to 9F, 10A to 10E, and 11A to 11D. For convenience of explanation, some embodiments are discussed with reference to operations performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally the contact of each finger or stylus, a representative point corresponding to the finger or stylus contact (e.g., the center of each contact or a point associated with each contact), or the center of two or more contacts detected on the touch-sensitive display system 112. However, similar operations are optional and performed on devices having a display 450 and separate touch-sensing surfaces 451, in response to detection of contact on the touch-sensing surface 451 while displaying the user interface shown in the figure on the display 450, along with the focus selector.
[0141] Figures 5A1 to 5D5 show exemplary user interfaces for sharing music media content received in a messaging application with a music media application, according to several embodiments.
[0142] Figure 5A1 shows an exemplary user interface for an application menu on a portable multifunction device 100 according to several embodiments. A similar user interface is optionally implemented on device 300. In particular, Figure 5A1 shows user interface page 5050 of a multi-page home screen user interface. User interface page 5050 includes several application icons (e.g., application icons 5008a-5008p and application icons 5008q-5008t) arranged in an optionally pre-configured layout (e.g., application icons 5008a-5008p are arranged in a 5x4 grid, and application icons 5008q-5008t are arranged in a 1x4 grid within tray 509). In some embodiments, application icons 5008a-5008p are user-selected and placed in their respective positions on user interface page 5050 according to user input. In some embodiments, application icons 5008q~5008t are system-selected and automatically placed in their respective positions within the application menu appearing at the bottom of the user interface page 5050, for example, in the navigation bar, dock, or tray 409. In some embodiments, application icons 5008q~5008t may also be selected by the user. In some embodiments, if an icon appears in tray 509, the same icon is not displayed in the rest of the user interface page 5050. In some embodiments, the icon appears in both trays 509, and the same icon appears in a different part of the user interface page 5050 (for example, the icon may appear in two separate locations within the user interface). In some embodiments, the dock or tray 509 shown at the bottom of the user interface page 5050 is shown at the same location on the display 112 on other pages of the multi-page home screen user interface.
[0143] In some embodiments, tray 509 includes application icons for frequently used applications, such as the following: ○ Icon 5008q for telephone module 138 labeled "Telephone", ○ Icon 5008r of email client module 140 labeled "Mail", ○ Icon 5008s of browser module 147 labeled "Browser", and ○ Icon 5008t for contact module 137.
[0144] In some embodiments, the user interface 5050 further includes the following application icons, or subsets or supersets thereof. ○ Icon 5008a of the video and music player module 152 labeled "Music", ○ Icon 5008b of calendar module 148 labeled "Calendar", ○ Icon 5008c of image management module 144 labeled "Photo", ○ Icon 5008d of IM module 141 labeled "Message", ○ Icon 5008e of online video module 155 labeled "online video", ○ Icon 5008f of stock price widget 149-2 labeled "Stock Price", ○ Icon 5008g of map module 154 labeled "Map", ○ Icon 5008h of the book module labeled "Book", ○ An alarm clock widget 149-4 icon labeled "Clock" 5008i, ○ Icon 5008j for training support module 142 labeled "Training Support", ○ Icon 5008k for memo module 153 labeled "Memo", ○ Icon 5008l of weather widget 149-1 labeled "Weather", ○ Calculator widget 149-3 icon 5008m labeled "Calculator", ○ Icon 5008n for a podcast application or module labeled "Podcast" ○ Icon 5008o of the application store module labeled "App Store", ○ Icon 5008p for a news application or module labeled "News".
[0145] In some embodiments, more applications are available on the electronic device 100 or 300 than those shown on user interface page 5050. These additional applications can be represented by other application icons available on other user interface pages of the multi-page home screen. Some application icons may be displayed on two or more user interface pages of the multi-page home screen.
[0146] It should be noted that the icon labels associated with icons 5008a to 5008t in Figure 5A1 are merely examples. For example, other labels may be used selectively for various application icons. In some embodiments, the labels for individual application icons include the name of the application to which the individual application icon corresponds. In some embodiments, the label for a particular application icon is different from the name of the application to which that particular application icon corresponds. Application icon labels are selected to indicate the relevant function of the corresponding application.
[0147] The term “application icon” is used to describe a user interface element representing a corresponding application, and the term “application” is also used to describe a function or module that is available, stored, and / or installed on an electronic device 100 or 300 (e.g., any of the above applications 136, 137-155, 380-390). It is also understood that activating an individual application icon from among several application icons (e.g., 5008a-5008t) replaces the display of the application corresponding to the individual application icon with a display of the individual page of the multi-page home screen user interface where the individual application icon is displayed. In some embodiments, displaying an application includes displaying a view of the application or the user interface of the application.
[0148] Figures 5A1 to 5A3 show examples of navigating from a user interface page on a multi-page home screen (e.g., user interface page 5050) to a specific conversation in an instant messaging application, according to several embodiments.
[0149] Figure 5A1 shows a tap input via contact 5002. The tap input via contact 5002 is detected at a location on the touchscreen corresponding to the instant messaging application represented by the application icon 5008d. In response to the tap input via contact 5002, the instant messaging application is opened, launched, or activated, and the electronic device stops displaying a user interface page on the multi-page home screen and displays the user interface of the instant messaging application, i.e., the messaging user interface 5040 shown in Figure 5A2.
[0150] In some embodiments, communication is received in an instant messaging application represented by application icon 5008d. Incoming communication may be sent by a user's contact associated with the electronic device, or by another user who is not a contact, for example, not stored as a known contact within the user's electronic device. In some embodiments, contacts and information about contacts are stored in a contacts application (e.g., contacts module 137). Information about a contact may include one or more telephone numbers, one or more email addresses, names, addresses, and other identifying information about the contact. Contacts are typically stored in the contacts application in response to user input associated with the electronic device (e.g., a user stores contact information on electronic device 100 or 300). Contacts are users of remote electronic devices similar to device 100 or 300. Contacts optionally connect with users associated with the electronic device through communication applications such as an instant messaging application (e.g., represented by icon 5008d), a social media application, and / or an email application (e.g., represented by application icon 5008r) installed on the device.
[0151] Incoming communications may include content shared by the user's contacts associated with the electronic device. Incoming communications may also include shared content from others who are not the user's contacts. It should be noted that sharing content (e.g., providing content recommendations) does not necessarily imply a positive review of the shared content. More broadly, shared content may be content items that are shared from one device to another and can be interacted with in other applications. In some embodiments, content can be shared from one device to another by a communication application (e.g., an instant messaging application represented by 5008d or 5008r, an email application). The sharing device may send a copy of the content item, a representation of the content item, a link to the content item, or other identifiers of the content item to the receiving device, and the receiving device may determine that the content is readable by a corresponding application designed to open such content. When content is shared from a device that is not a contact of the receiving device, the receiving device may, depending on user settings, determine that the content should not be sent to a corresponding application to open the content.
[0152] In some implementations, content may be AirDropped (e.g., via Wi-Fi or Bluetooth®) or transmitted from one device to another via NFC or UWB. In other implementations, content may be shared using other applications instead of, or in addition to, communication applications.
[0153] Figures 5A2 to 5A3 show a messaging user interface 5040 including several conversations 5006a to 5006f. Conversations 5006a to 5006f are between a user associated with an electronic device and one or more contacts stored in the contact module 137. In some embodiments, a conversation may be between a user and one contact, for example, conversation 5006a is between the user and the contact "Alex" represented by avatar 5003a. Conversation 5006c is between the user and the contact "Lucy" represented by avatar 5003c. Conversation 5006d is between the user and the contact "Joe" represented by avatar 5003c, and conversation 5006e is between the user and the contact "Tony" represented by avatar 5003e. In some embodiments, a conversation may be between a user and two or more contacts. For example, the conversations "Costa Rica Travel Group" 5006b and "Family Group" 5006f are group chats in which two or more contacts are participating. Group chats are represented by multiple avatars to indicate that three or more users are participating in the conversation; for example, "Costa Rica Travel Group" 5006b is represented by a group of avatars 5003b, and "Family Group" 5006f is represented by a group of avatars 5003f.
[0154] Figure 5A3 shows a tap input by contact 5010 detected at a location on the touchscreen corresponding to conversation 5006a between the user of the electronic device and the contact "Alex". In response to the tap input by contact 5010, the electronic device opens conversation 5006a in a different user interface of the messaging application. For example, the tap input by contact 5010 causes the electronic device to stop displaying the messaging user interface 5040 and display the conversation user interface 5042.
[0155] Figure 5A4 shows at least a portion of a conversation 5006a containing several different types of shared content items shared by the contact "Alex" according to some embodiments. According to some embodiments, the conversation 5006a is displayed in a conversation user interface 5042. For example, the conversation user interface 5042 shows several messages 5001a to 5001k exchanged between the contact "Alex" and the user of the electronic device. The conversation 5006a is stored in a messaging application, and the user of the electronic device can return to continue the conversation, respond to one of the messages, review the content of the conversation, copy the content of the conversation, or delete one or more messages in the conversation. In some embodiments, the conversation 5006a includes several shared content items shared with the user of the electronic device by the contact "Alex," represented by an avatar 5041. For example, message 5001a from contact "Alex" shares a podcast with the user of the electronic device, the podcast is represented by a podcast image 5045a and a tiled "Podcast Name-A"; message 5001d from contact "Alex" shares a song with the user of the electronic device, the song is represented by a song image (e.g., representation 5045b) and is titled "Song Title-A"; message 5001g from contact "Alex" shares an article with the user of the electronic device, the article is represented by an article image 5045c and is titled "News Article Title-A". In some embodiments, sharing a content item via instant messaging may include sharing a copy of the content item, a representation of the content item, a link to the content item, or other identifiers of the content item that are readable by a corresponding application designed or capable of opening such content. For example, a music application such as video and music player module 152 is designed or capable of playing, displaying, or otherwise opening a song represented by a song image (e.g., representation 5045b).The user can tap or otherwise select the play button 5045c associated with a song represented by an image (e.g., representation 5045b). Messages 5001c, 5001f, 5001h, and 5001k from the user of the electronic device are visually distinguished from messages 5001a, 5001b, 5001d, 5001e, 5001g, and 5001j from the contact "Alex".
[0156] In some embodiments, upon receiving shared music in a message such as message 5001d in conversation 5006a, the electronic device automatically makes the music available in a dedicated section for later listening or playback, without requiring user input. Music shared with a user (e.g., in different messages, conversations, applications, or by different contacts / users) can be marked as shared for later display in a music application, or can be automatically grouped together (e.g., in a data structure). For example, if a shared content item meets certain criteria (e.g., should be sent to an application that is shared by the content and can open the content), the user does not need to open the application to which the content was shared, nor do they need to open the message or conversation to which the content was shared, nor do they need to open an application to which the content can interact and take any proactive action. When information associated with shared content is received by a receiving electronic device in one application, the receiving application makes a decision about where to display the shared content. For example, if the received content is music content, the receiving device can display the content in the application to which the content was received, as well as one or more applications specific to music. For example, device 100 includes applications for music service 1, music service 2, and music service 3, but only displays shared content in music service 1 and music service 2, without displaying shared content in music service 2.
[0157] In some embodiments, the song shared in message 5001d is automatically made available in one or more music applications (e.g., music applications represented by icon 5008a) without user input. For example, the song shared in message 5001d is automatically made available in the music application without the user selecting or requesting to open or view the shared song. Instead, the receiving device automatically determines one or more appropriate applications to display the content based on the sharing settings on the device. The sharing settings may include determining whether the content was shared by a particular user, user type, or group. The sharing settings may also include determining whether to share particular content in one or more specific applications. For example, a user device may have three music applications downloaded to the user device, but may allow content to be shared in only two of the music applications. When the user device receives music content, the user device can automatically share the music content with the two music applications for which sharing was previously enabled, rather than a third application (e.g., without sharing the content with one or more applications that do not allow content sharing). For example, as illustrated with reference to Figures 7A-7D, users can select applications for which content sharing is enabled.
[0158] Once received, the content may be automatically shared by the receiving device, and may be shared without user interaction, for example, in response to the user selecting content represented by an image (e.g., representation 5045b), for example, in response to the user selecting an image (e.g., representation 5045b), or in response to the user selecting the play button 5045c.
[0159] Figure 5A5 shows a tap input by contact 5012 detected at a location on the touchscreen corresponding to a music application represented by icon 5008a on user interface page 5050, according to one embodiment. In response to the tap input by contact 5012, the electronic device stops displaying user interface page 5050, opens the music application, and displays the default view of the music application.
[0160] Figure 5A6 shows a default view 5052 of a music application according to several embodiments, the default view including a section for shared content. According to one embodiment, the default view 5052 displays several shared songs or other music (e.g., playlists, albums of songs, etc.) in a dedicated section such as section 5054. In some embodiments, the default view 5052 of the music application includes a section 5054 dedicated to music shared with the user of the electronic device. In some embodiments, section 5054 is labeled “Shared with You”. Section 5054 groups or aggregates music shared with the user of the electronic device into a single section. For example, section 5054 displays a representation 5054a of songs shared in message 5001d (shown in Figure 5A4). In some embodiments, section 5054 displays representations 5054b and 5054c of other music shared in other conversations or in the same conversation, such as music shared in one or more of conversations 5006a to 5006f. For example, song "Song Title-B," represented by song image 5054b, is shared by "Joe" in conversation 5006d (shown in Figure 5A2), and song "Song Title-C," represented by song image 5054c, is shared by two or more contacts in one or more of conversations 5006a to 5006f (shown in Figure 5A2).
[0161] In some embodiments, the default view 5052 includes additional sections for displaying music content, such as a section 5056 that displays a representation of recently played music, or a section 604 “Highlights” (shown in Figure 5E8) that displays a representation of highlighted content. In some embodiments, the user interface of the music application includes a tray or navigation bar 5057 with options 3001a to 3001e for selecting different views of the music application, such as view 3001a for displaying currently playing content; view 3001b for browsing music in the music application; view 3001c for playing or browsing radio stations; view 3001d for displaying a library of saved music; and search 3001e for searching for artists, playlists, albums, and other music available in the music application.
[0162] In some embodiments, each music item represented within section 5054 (a section dedicated to shared music) is associated with a selectable capsule that provides information or a link to the location where the shared music item was first shared (e.g., within a messaging application). For example, a shared song "Song Title-A" represented by song image 5054a is associated with a selectable capsule 5056a labeled "From Alex." A shared song "Song Title-B" represented by song image 5054b is associated with a selectable capsule 5056b labeled "From Joe," and a shared song "Song Title-C" is associated with a selectable capsule 5056c labeled "From 3." In some embodiments, the selectable capsule 5056a is located near representation 5054a, the selectable capsule 5056b is located near representation 5054b, and the selectable capsule 5056c is located near representation 5054c.
[0163] Figure 5A6 shows a tap input by contact 5014 detected at a location on the touchscreen corresponding to a selectable capsule 5056a labeled "From Alex". In response to the tap input by contact 5014, the electronic device stops displaying the default view 5052 and opens the application (e.g., a messaging application) where the content was shared at the location where the content was shared. For example, in some embodiments, in response to the tap input by contact 5014, the electronic device displays the conversational user interface 5042 (shown in Figure 5A7). Similarly, in some embodiments, in response to a tap input at a location on the touchscreen corresponding to a selectable capsule 5056b labeled "From Joe", the electronic device stops displaying the default view 5052 and opens the messaging application at the location where the song "Song Title-B" (represented by song image 5054b) was shared. In some embodiments, in response to a tap input at a location on the touchscreen corresponding to a selectable capsule 5056c labeled "from 3 people", the electronic device stops displaying the default view 5054 and displays a deambiguation interface, such as a deambiguation user interface 572, which will be described in more detail in relation to Figures 5F1-5F2.
[0164] Figure 5A7 shows a portion 5042a of the message thread of conversation 5006a displayed in the conversational user interface 5042 in response to a tap input by contact 5014, according to one of several embodiments. The tap input by contact 5014 selects a selectable capsule 5056a, and in response to the selection of the selectable capsule 5056a, the electronic device opens conversation 5006a with shared content in the messaging application. The portion 5042a of the message thread displays both sides of conversation 5006a in chronological order on a single screen. For example, a message from contact "Alex" is displayed on one side, and a message from the user of the electronic device / computer system is displayed on the other side. The user of the electronic device can enter a text message in field 5059. Furthermore, the user of the electronic device can share content by selecting one or more of options 4001a to 4001e. For example, a user can share a photo by selecting option 4001a, an application by selecting option 4001b, music by selecting option 4001c, a game by selecting option 4001d, and contacts by selecting option 4001e. In some embodiments, a user can tap or select the selectable text "Done" 5043 to return to the default view of the music application, which includes a section 5054 dedicated to sharing music.
[0165] The portion 5042a of the message thread of conversation 5006a displays the last sent / received messages in chronological order. For example, Figure 5A7 displays conversation 5006a without scrolling to a previously received message (e.g., message 5001d) containing the shared song "Song Title-A". In some embodiments, in response to a tap input by contact 5014 (shown in Figure 5A6), the electronic device displays the portion 5042b of the message thread of conversation 5006a at the location of a previously received message (e.g., message 5001d) containing the shared song "Song Title-A" (shown in Figure 5A8 below).
[0166] Figure 5A8 shows a portion 5042b of a message thread in conversation 5006a that automatically scrolls to a previously received message 5001d, which includes a shared song “Song Title-A” represented by expression 5045b. For example, instead of displaying the last sent / received message in chronological order, portion 5042b displays the previously last received message 5001d and messages sent / received before message 5001d, such as previously shown messages 5001c, 5001b, 5001a, and messages 5001m and 5001l that have not been previously shown. Message 5001m indicates that the contact “Alex” has shared a location (e.g., an address that can be opened in a map application). In some embodiments, the user can tap or select the selectable text “Done” 5043 to return to the default view of the music application, which includes a section 5054 dedicated to shared music.
[0167] Figure 5B1 shows a portion 5062a of a conversation between a contact "Alex" and a user of an electronic device within a conversational user interface 5062, including several inline replies according to one embodiment. For example, message 5009b from the user of the electronic device is a reply to message 5009a from contact "Alex". Line 5064 indicates that message 5009b is a reply to message 5009a. Message 5009e is also a reply to message 5009a, sent after an irrelevant message 5009c was sent by contact "Alex". The irrelevant message 5009c interrupts the inline reply thread. To mitigate the cognitive burden caused by this interruption, message 5009d is automatically added to the conversation by the electronic device. Message 5009d is the same message as message 5009a (the replied message), and is shown with a dashed line to indicate that it was not actually sent by contact "Alex", but is simply repeated for visual feedback that the subsequent message is a reply. Line 5066 indicates that message 5009e is a reply to message 5009d. Message 5009f contains the text message "Yeah, we're talking about song-A! Here it is!" and the shared song "Song Title-D". Message 5009f is a reply to message 5009e, as indicated by line 5068. Messages 5009g and 5009h are not related to the inline reply thread, and therefore no lines are used to indicate that there is no relationship between the messages.
[0168] In some embodiments, as shown in Figure 5B2, after message 5009f containing the shared song "Song Title-D" is received in the messaging application, the user of the electronic device opens the music application, and the electronic device displays the default view of the music application.
[0169] Figure 5B2 shows the shared song "Song Title-D" represented by song image 5045d in section 5054 dedicated to shared music, as described above in relation to Figure 5A6. The shared song "Song Title-D" is associated with a selectable capsule 5056d labeled "From Alex".
[0170] Figure 5B2 shows a tap input by contact 5016 detected at a location on the touchscreen corresponding to a selectable capsule 5056d labeled "From Alex". In response to the tap input by contact 5016, the electronic device stops displaying the default view 5052 and opens the reply user interface of the messaging application at the location where the content was shared. For example, in some embodiments, in response to the tap input by contact 5016, the electronic device displays the inline reply user interface 5072 shown in Figure 5B3.
[0171] In some embodiments, the inline reply user interface 5072 displays only messages that are part of the same inline reply thread, such as messages 5009a, 5009b, 5009e, and 5009f. Furthermore, the inline reply user interface 5072 automatically scrolls to message 5009f, which contains the shared song "Song Title - D". The inline reply user interface 5072 further shows a keyboard 5079 and a reply input field 5077 that the user of the electronic device can use to enter a reply. According to one embodiment, the user of the electronic device can use the inline reply user interface 5072 to send a reply containing the shared song "Song Title - D" directly to message 5009f.
[0172] Figure 5C1 shows a tap input by contact 5018 on selectable text 5075 labeled "See All". Selectable text 5075 appears within the default view 5052 of the music application in a dedicated section 5054 for shared music. The default view 5052 is described with reference to Figure 5A6 above. When contact 5018 taps on the selectable text 5075 labeled "See All", the electronic device expands the dedicated section 5054 for shared music to include and display all music shared with the user of the electronic device, as shown in Figure 5C2 below.
[0173] Figure 5C2 shows the default view 5052, which has an expanded section 5054 for shared music. The expanded section 5054 is displayed in response to a tap input via contact 5018, as shown in Figure 5C1. In response to the tap input via contact 5018, the electronic device stops displaying the recently played music section 5056 and any other sections that may have been displayed simultaneously with the unexpanded section 5054.
[0174] Figure 5C2 shows the expanded section 5054 for shared music, simultaneously displaying shared songs that are shown in the unexpanded section 5054 and shared songs that are not shown in the unexpanded section 5054. For example, the shared song "Song Title-A" represented by song image 5054a, the shared song "Song Title-B" represented by song image 5054b, and the shared song "Song Title-C" represented by song image 5054c are displayed in the unexpanded section 5054 of Figure 5C1, while the shared playlist "Playlist Name" represented by playlist image 5054e and the shared song "Song Title-F" represented by song image 5054f are not displayed in the unexpanded section 5054 shown in Figure 5C1. In some embodiments, Figure 5C2 shows an indicator 5075a of the number of shared music items. For example, Figure 5C2 shows that section 5054 for shared music includes five shared music items represented by image representations 5054a, 5054b, 5054c, 5054e, and 5054f.
[0175] Figure 5C3 shows a tap input by contact 5020 detected at a location on the touchscreen corresponding to the representation 5054a of the shared song "Song Title-A". The tap input by contact 5020 selects the shared song "Song Title-A". In response to the tap input by contact 5020, the electronic device stops displaying the default view 5052, which includes section 5054 for shared music. In response to the tap input by contact 5020, the electronic device plays the shared song "Song Title-A" and displays a user interface having a detailed view of the currently playing song, such as the user interface 5032 for the currently playing song, as shown in Figure 5C4 below.
[0176] Figure 5C4 shows a user interface 5032 for the currently playing song, represented by image 5035. Image 5035 is the same as image 5045a, but displayed enlarged. User interface 5032 shows details of the currently shared song 5037, including the song title and avatar 5039, representing the contact "Alex" who shared the currently playing song, represented by image 5035. In some embodiments, the avatar 5039 is selectable and provides a link to the location in the messaging application where the currently playing song was shared. For example, in some embodiments, tapping the avatar 5038 causes the electronic device to stop displaying the user interface 5032 playing the currently playing song and display a messaging user interface as shown in Figures 5A6, 5A7, and 5B3.
[0177] Figure 5D1 shows a long press input by a contact 5022 detected at a location on the touchscreen corresponding to a selectable capsule 5056a labeled "From Alex". Figure 5D1 also shows the same default view 5052 of the music application, including a shared song "Song Title - A" represented by a song image 5054a in the section 5054 for shared music, as described in relation to Figure 5A6.
[0178] Figure 5D2 shows an overlay user interface 5082 that includes a set of controls 5080 for performing several actions related to the shared song "Song Title-A". In response to a long press input via contact 5022, the electronic device stops displaying the default view 5052 and displays the overlay user interface 5082 that includes a set of controls 5080 for performing several actions related to the shared song "Song Title-A", represented by the song image 5054a in section 5054 (shown in Figure 5D1). In some embodiments, the overlay user interface 5082 maintains the display of a selectable capsule 5056a associated with the shared song "Song Title-A". For example, the user of the electronic device can return to the conversation in which the song was shared by selecting the selectable capsule 5056a displayed in the overlay user interface 5082.
[0179] In some embodiments, the set of controls 5080 includes a control 5080a for replying to a message in which the song "Song Title-A" is shared, such as message 5001d shown in Figure 5A4 above. Depending on whether control 5080a is tapped or otherwise selected, the electronic device displays a conversation in which the song "Song Title-A" represented by a song image 5054a is shared, such as the conversation user interface 5042 described in Figures 5A7 and 5A8, or an inline reply user interface, such as the inline reply user interface 5072 shown in Figure 5B3.
[0180] In some embodiments, the set of controls 5080 includes control 5080b for viewing further details associated with the shared song. According to some embodiments, control 5080b is labeled as an "information card". In some embodiments, the set of controls 5080 includes control 5080c for initiating a shared listening session with the contact "Alex" who shared the song. In the shared listening session, the user of the electronic device and the contact "Alex" can listen to the song simultaneously in a virtual coexistence session.
[0181] In some embodiments, the set of controls 5080 includes a control 5080d for removing the shared song "Song Title-A" representation 5054a and the associated selectable capsule 5056a labeled "From Alex" from the collection of shared music. For example, in response to a tap input by contact 5084, the electronic device stops displaying the shared song in section 5054 for shared music, as shown in Figure 5D5. In some embodiments, the control 5080d allows the user of the electronic device to unassociate the shared song from the message and / or contact to whom the song was shared.
[0182] Figure 5D3 shows an overlay user interface 5082a that includes a set of controls 5080 as described in Figure 5D2 above. In some embodiments, the overlay user interface 5082 is displayed in response to a long press input via contact 5022 as shown in Figure 5D1. In some embodiments, the overlay user interface 5082a displays a fragment 5088 of a conversation in which a song is shared, such as an excerpt or portion of a conversation 5006a as shown in the conversation user interface 5042. In some embodiments, the fragment 5088 includes a message in which a song is shared, such as message 5001d (also shown in Figure 5A4 above). In some embodiments, in response to a tap input via contact 5086, the electronic device stops displaying the shared song in a section 5054 dedicated to shared music, as shown in Figure 5D5.
[0183] Figure 5D4 shows an overlay user interface 5082b that includes a set of controls 5090 for performing several actions related to the shared song "Song Title-A". The electronic device displays the overlay user interface 5082b in response to a long press input via contact 5022 (shown in Figure 5D1). In some embodiments, the overlay user interface 5082b further displays an enlarged representation 5035b of the shared song "Song Title-A" and an avatar 5039b representing the contact "Alex" who shared the song, represented by an image (e.g., representation 5035b). In some embodiments, the avatar 5039b is selectable and provides a link to the location in the messaging application where the song was shared. For example, in some embodiments, tapping the avatar 5039b causes the electronic device to stop displaying the overlay user interface 5082b and, according to some embodiments, display a messaging user interface as shown in Figures 5A6, 5A7, and 5B3.
[0184] In some embodiments, the set of controls 5090 includes 5090a for playing the shared song "Song Title-A", 5090b for downloading the song for offline playback and storing it in the memory of the electronic device, 5090c for adding the shared song to a playlist, 5090d for playing the next song in the song list (an option is displayed if the shared song "Song Title-A" is part of a playlist), and 5090d for removing the shared song from the music application library (control 5090d when the shared song is saved in the library).
[0185] In some embodiments, the electronic device includes a control 5090e for replying to a message in which the song "Song Title-A" is shared, such as message 5001d shown in Figure 5A4 above. Depending on whether the control 5090e is tapped or otherwise selected, the electronic device displays a conversation in which the song "Song Title-A" is shared, such as the conversation user interface 5042 described in Figures 5A7 and 5A8, and an inline reply user interface, such as the inline reply user interface 5072 shown in Figure 5B3.
[0186] In some embodiments, the set of controls 5090 includes a control 5080f for removing the shared song "Song Title-A" representation 5054a and the associated selectable capsule 5056a labeled "From Alex" from the collection of shared music. For example, in response to a tap input via contact 5098, the electronic device stops displaying the shared song in section 5054 for shared music, as shown in Figure 5D5. In some embodiments, the control 5090f allows the user of the electronic device to unassociate the shared song from the message and / or contact to whom the song was shared.
[0187] Figure 5D5 shows the default view 5052 of the music application, where the shared song "Song Title-A" and its associated selectable capsule 5056a (labeled "From Alex") have been removed from the section 5054 dedicated to sharing. In some embodiments, previously displayed representations 5054b and 5054c have been shifted to the top of the list of shared songs, and the shared song "Song Title-E" appears in the list of shared songs that have not previously appeared in the section 5054 dedicated to shared music.
[0188] Figures 5E1 to 5E8 show exemplary user interfaces for highlighting content representation in messaging applications according to several embodiments.
[0189] Figure 5E1 shows how user input 590 (e.g., tap input) is detected to select a user-selectable capsule 5056a. As described above with reference to Figure 5A6, when selected, the selectable capsule 5056a causes device 100 to display at least a portion of a message thread (e.g., conversation history) that shares an individual media item (e.g., song A). For example, in response to user input 590, device 100 displays the user interface shown in Figure 5E2, which includes a message (e.g., message 5001d) that shares song A (e.g., also called "song title-A"). In some embodiments, in response to user input 590, device 100 switches from a media application (e.g., a music application) to a communication application (e.g., a messaging application) that shares a media item. In some embodiments, in response to user input 590, device 100 replaces a portion of the user interface display to display a portion of the communication thread that shares a media item (e.g., the portion of the communication thread is displayed overlaid on the portion of the currently displayed media application).
[0190] Figure 5E2 shows how user input 592 (e.g., long-press user input) is received to select message 5001d.
[0191] In response to detecting user input 592, device 100 displays a tap-back menu for message 5001d, e.g., multiple user-selectable options, including multiple reaction controls 594, optionally a play control 596, a highlight control 598, and / or a reply control 599. In some embodiments, the multiple reaction controls 594, when activated, cause the device to add a “reaction” to the selected message (e.g., message 5001d). In some embodiments, the added “reaction” is added to the message thread (e.g., conversation history) so that each participant in the message thread can see the “reaction.” In some embodiments, the multiple reaction controls 594 include one or more controls for putting a heart on the message (e.g., nice), giving a thumbs up on the message (e.g., like), giving a thumbs down on the message (e.g., dislike), putting an exclamation mark on the message (e.g., emphasis), putting a question mark on the message (e.g., question), and putting a haha on the message (e.g., laugh).
[0192] In some embodiments, upon determination that the selected message 5001d contains a representation of a media item (e.g., song A), device 100 optionally provides a playback control 5045c (e.g., to start playback of the media item). In some embodiments, device 100 displays a highlighting control 598-1 that allows the user to mark (e.g., highlight) the representation of the selected media item (e.g., and / or message). After the representation of the media item has been highlighted, for example, device 100 displays a highlighting display 598-2 (as described with reference to Figure 134) to indicate that the user has highlighted the media item. In some embodiments, the highlighting display 598-2 includes a highlighting control 598-1 with modified visual features (e.g., filling in a star, changing a color, indicating content with a pin, etc.).
[0193] Figure 5E3 further illustrates how user input 597 on the reply control 599 is detected. In response to the detection of user input 597 on the reply control 599, device 100 displays a user interface for replying to the selected message (for example, within a messaging application), as shown in Figure 5E4. For example, a reply box 588 is displayed (for example, simultaneously with a virtual keyboard), allowing the user to compose and send a reply message to the selected message 5001d.
[0194] Figure 5E5 shows user input 586 (e.g., tap input) selecting a highlighting option 598-1 displayed for the representation of shared song A. In some embodiments, in response to user input 586 selecting the highlighting control 598-1, device 100 marks shared song A as highlighted (e.g., remembers) and updates the highlighting control 598-1 to be displayed in a visually distinct appearance (e.g., displayed simultaneously with the representation of the shared media item). For example, before the selection of the highlighting control 598-1, the highlighting control 598-1 contains an unfilled star. After the user selects the highlighting control 598-1, device 100 updates the highlighting control 598-1 to be displayed as a filled star (e.g., highlighting representation 598-2), as shown in Figure 5E6. It will be understood that alternative methods not shown in the illustration may also be used to visually distinguish the highlighting controls 598-1 (e.g., updating the color of the highlighting option, changing the size of the highlighting option, animating the highlighting option, adding an overlay to the highlighting option, etc.). Those skilled in the art will understand that the highlighting affordances may take any other suitable form or design to indicate two or more toggle states.
[0195] When highlighted, the representation of the content item (and the shared content item itself) is highlighted. In some embodiments, when highlighted, the content item is added to a database on the device that lists all highlighted content items. In some embodiments, after shared song A is marked as highlighted, device 100 displays a highlighting display 598-2 that optionally at least partially overlays message 5001d containing the representation of shared song A. For example, at any time thereafter, the user can open an application (e.g., a second application) that can play or view the content item (i.e., a content item having a first content type, e.g., music, video, or images), and the representation of the content item is displayed within the application. In some implementations, when a user highlights a particular content, the electronic device automatically marks the content to be highlighted in one or more corresponding applications. For example, if a song is highlighted, the electronic device marks the song as highlighted so that the highlighted content can be highlighted in any music application where the highlighted content is enabled.
[0196] Figure 5E7 shows user input 582 selecting icon 5008a corresponding to the music application. In response to user input 582, the device opens (e.g., launches, starts displaying) the music application, as shown in Figure 5E8.
[0197] Figure 5E8 shows an exemplary user interface for a music application. In some embodiments, the user interface includes a first part for displaying media items that are “highlighted” and a second part for displaying media items that are “shared with you.” For example, as illustrated with reference to Figure 5E5, shared song A is marked (e.g., remembered) as a highlighted media item depending on whether the user selects the highlighting control 598-1. Thus, a representation of shared song A is displayed in the “highlighted” part of the music application. In some embodiments, the representation of shared song A is optionally displayed together with a highlighting indicator 598-2 (e.g., a filled star).
[0198] In some embodiments, one or more media items are displayed in multiple sections of the music application. For example, song A is classified as highlighted and shared with the user of device 100. Therefore, in some embodiments, the representation of song A is displayed in both the "Highlighted" section and the "Shared with You" section of the music application's user interface. In some embodiments, upon determining that a media item is classified by two or more sections, device 100 selects one section to display the representation of the media item (for example, to avoid duplication of displays of the same media item in multiple sections of the user interface). In some embodiments, the representation of shared song A is optionally displayed in the "Shared with You" section along with the highlighting representation 598-2 (for example, highlighted shared items appearing in the "Shared with You" section optionally display the highlighting representation on the representation of the highlighted shared item).
[0199] In some embodiments, any content item (or representation of a content item) highlighted by the user includes a highlighting display 598-2 (e.g., a highlighting display in a highlighted state). In some embodiments, the user can select a highlighting display to switch it off (or to an unhighlighted state). When a representation of a content item is switched off, it is removed from the "Highlights" section of the music application's user interface. Similarly, the highlighting display is removed from the "Shared with You" section of the user interface or is shown in a toggle-off state (not highlighted).
[0200] Highlight affordances allow users to easily highlight content items they want to easily access in other applications. For example, if a user receives a favorite song from a friend via a messaging application, they can easily highlight that song in their messaging application, and the next time they open their music application, the song will be listed at the top of their music application.
[0201] Figures 5F1 to 5F2 show exemplary user interfaces for eliminating ambiguity according to several embodiments.
[0202] Figure 5F1 shows how user input 570 (e.g., tap input) is received on the selectable capsule 5056c. As described above, the selectable capsule 5056c includes a shared display that is simultaneously displayed on the media item representation 5054c, and the shared display includes information about who has shared an individual media item. In some embodiments, according to the determination that multiple separate users (e.g., and / or separate groups) have shared the same media item, the selectable capsule 5056c, once selected, allows the user of device 100 to select from a list of communication threads (e.g., within the same communication application (e.g., a messaging application) or from different communication applications (e.g., a messaging application thread and a mail application thread)).
[0203] For example, upon detecting user input 570, device 100 displays the deambiguation user interface 572 shown in Figure 5F2. In some embodiments, the user interface 572 displays a list of users (e.g., contacts), where each user in the list corresponds to the user who shared the selected media item (e.g., song C ("song title - C")). In some embodiments, each user in the list is displayed along with a user representation (e.g., avatar, contact photo, etc.) and a user identifier (e.g., text identifier) (e.g., "Alex"). In some embodiments, each user in the list is provided with an option 574 to open a communication application (e.g., a messaging application). In some embodiments, the option 574 for opening a communication application is for the communication application in which the media item was shared. For example, upon determining that the media item was shared using a mail application, a mail application icon is displayed as an option to open a communication application. In some embodiments, upon detecting user input selecting option 574, device 100 opens the message thread (e.g., within a messaging application) in which the selected media item (e.g., song C) was shared. In some embodiments, upon detecting an input selection option 574, device 100 opens an overlay user interface (e.g., a "quick reply" user interface) for sending a message to the individual users who shared the selected media item, and the overlay user interface is displayed while maintaining the display of the media application (e.g., a music application) without switching to the messaging application.
[0204] In some embodiments, the user interface 572 further includes an option 575 for initiating a shared session for each user in the user list. For example, upon detecting a user input (e.g., a tap input) that selects option 575, device 100 opens a shared session with the individual user shown in the user list (e.g., a shared session with Alex). In some embodiments, the shared session allows both the user of device C and the individual user in the user list (e.g., Alex) to play the selected media item (e.g., song C) substantially simultaneously within the shared session.
[0205] Figures 5G1 to 5G4 show exemplary user interfaces for sharing media content received in a group message thread, according to several embodiments.
[0206] Figure 5G1A shows an exemplary user interface for a messaging application that includes displaying multiple representations of message threads (e.g., each message thread corresponding to a different user (e.g., a contact) or a group of users (e.g., a group message thread)). In some embodiments, as shown in Figure 5G1B, user input 520 (e.g., tap input) is detected on a portion of the user interface corresponding to a group message (e.g., a Costa Rica travel group).
[0207] In response to detecting user input 520, device 100 displays a user interface 522 containing the message thread (e.g., conversation history) of the selected group message, as shown in Figure 5G2. In some embodiments, the selected group message includes a representation of one or more shared media items (e.g., one or more media items shared within the group message), including representations of media items shared by users of device 100 and media items shared by other members of the group who are not users of device 100. In some embodiments, the user interface 522 includes user-selectable options 523 for viewing additional information related to the message thread. For example, device 100 detects user input 524 (e.g., tap input) that selects a user-selectable option 523.
[0208] In response to detecting user input 524, device 100 updates the display to an information user interface 525. In some embodiments, the information user interface 525 includes one or more communication options 526, including option 526a for initiating a voice call with a group member, option 526b for initiating a video call with a group member, option 526c for sending a message (e.g., email) to a group member, and option 526d for initiating a sharing session with a group member. In some embodiments, the user interface 525 further includes a toggle switch 527 that allows a user of device 100 to enable and / or disable whether media items shared within a selected group message (e.g., a trip to Costa Rica) use the “Share with You” feature described herein. For example, when the toggle switch 527 is turned off, “Hide Share with You” is turned off (e.g., media items shared within a group message thus appear in a separate media application corresponding to the type of media item being shared). When toggle switch 527 is turned on, "Hide Shared With You" is turned on (for example, media items shared within a group message will not appear in the separate media application corresponding to the type of shared media item). Thus, users of device 100 can choose whether or not to enable the "Shared With You" feature for each conversation in the messaging application.
[0209] In some embodiments, the information user interface 525 further includes a “highlighting” portion of the display. In some embodiments, the “highlighting” portion includes a representation of one or more media items shared within a group message (e.g., a representation of media item 529). In some embodiments, a representation of a media item shared by another user in the group message (e.g., not a user of device 100) is displayed together with a sharing indicator 528. In some embodiments, the sharing indicator 528 includes a representation of the user who shared the media item (e.g., an avatar or contact photo corresponding to the user who shared the media item). In some embodiments, a representation of a media item shared by a user of device 100 is not displayed together with a representation of the user who shared the media item (e.g., because it was shared by a user of device 100). In some embodiments, the representation of one or more media items within the “highlighting” portion includes only the representation of media items (e.g., within a group message) that have been highlighted by the user (e.g., as described above with reference to Figures 5E5-5E6), and does not include representations of media items that have not been highlighted by the user. In some embodiments, the “highlighted” portion includes representations of one or more media items that have been highlighted by the user, in addition to representations of one or more media items that have not been highlighted by the user. For example, device 100 determines one or more media items that are likely to be relevant to the user to include in the “highlighted” portion (whether, for example, the user has chosen to highlight the media items).
[0210] Figure 5G3 shows how a user input 530 (e.g., a swipe input) corresponding to a scroll request (e.g., an upward swipe input to scroll down the user interface) is detected. In response to the user input 530, device 100 scrolls the user interface 525 to display user interface 525b, as shown in Figure 5G4. In some embodiments, user interface 525b includes a "Photos" section 531, as shown in Figure 5G4. In some embodiments, the "Photos" section 531 includes one or more photos shared in a group message. In some embodiments, photos shared in a group message are optionally displayed with a sharing indicator (not shown) (e.g., a sharing indicator 528 as described with reference to Figure 5G3).
[0211] Figure 5H shows an exemplary user interface 51000 that is displayed in response to a user request to search for media items. For example, in some embodiments, device 100 detects user input (e.g., a swipe input) that causes device 100 to display the search user interface 51000. In some embodiments, the search user interface 51000 includes a search box (where the user can enter a search query using keywords). In some embodiments, before receiving user input requesting a search (e.g., before receiving a search query), device 100 automatically displays representations of one or more suggested media items. For example, device 100 determines one or more media items that are likely to be relevant to the user (e.g., based on media items the user has highlighted (e.g., added as favorites), based on the user's search usage history, based on media items in the user's media library, etc.).
[0212] In some embodiments, the search user interface 51000 includes representations of multiple contacts (e.g., representation 51002). For example, device 100 displays representations of multiple contacts that have been marked as favorite contacts by the user. In some embodiments, each representation of one of the multiple contacts includes a photo of the contact, an animated contact head, or text (e.g., the first initial of the contact's name). For example, representation 51002 includes an animated contact head corresponding to a contact that has been marked as a favorite by the user of device 100.
[0213] In some embodiments, the search user interface 51000 includes multiple sections, such as a “Highlights” section for displaying representations of one or more media items that have been “highlighted” (e.g., marked) by the user, and a “Photos” section for displaying one or more images (e.g., also displayed in a photo application).
[0214] In some embodiments, the search user interface 51000 includes displaying representations of one or more shared media items not stored in the user's media library (for example, instead of, or in addition to, displaying representations of media items stored in the user's media library). For example, device 100 includes a media library for audio items (e.g., songs, podcasts, etc.). In some embodiments, song A image 51005 represents a song (e.g., "Song A") that is a shared media item shared with device 100 via a communication application (e.g., as indicated by a shared display 51006). In some embodiments, "Song A" (e.g., corresponding to "Song A image" 51005) is not stored in the media library of audio items on device 100. In some embodiments, "Podcast A image" 51003 represents a podcast (e.g., "Podcast A") that is stored in the media library of audio items. In some embodiments, the song A image and the podcast A image are overlaid with the "song A image" 51005 and the "podcast A image" 51003 and highlighted (e.g., flagged as a favorite, pinned, starred, marked, or otherwise tagged) as indicated by the highlighting indicator 51004 that appears in the "highlights" section of the search user interface 51000.
[0215] Figures 5I1 to 5I10 show exemplary user interfaces for displaying the representation of shared media items in different media applications, based on the individual types of shared media items.
[0216] For example, Figure 5I1 shows a device 100 that detects user input 540 (e.g., tap input) to select a news application icon 5008p. In response to detecting user input to select a news application icon 5008p, device 100 opens the news application (e.g., launches, starts execution, or otherwise displays its user interface) as shown in Figure 5I2. Figure 5I2 shows an exemplary user interface of the news application, including a “Shared with You” section 542. In some embodiments, the “Shared with You” section 542 includes a list of one or more news articles that have been shared with users of device 100 (e.g., via a communication application such as a messaging application). For example, the “Shared with You” section 542 includes a representation of article A, including an image 543 representing article A, and other information about the article (e.g., text information) (e.g., title, author, summary, etc.). In some embodiments, the representation of article A further includes a sharing indicator 544 that provides information about who shared the individual article. For example, Alex shared article A with users of device 100. In some embodiments, the shared display 544 is selectable, and depending on the selection of the shared display 544, the user is provided with the option to send a message to the user who shared the article and / or is provided with additional shared information.
[0217] In some embodiments, in response to detecting user input on the article's presentation, device 100 displays (e.g., opens) the selected article (e.g., for the user to view).
[0218] Figure 5I3 shows device 100 detecting user input 545 (e.g., tap input) to select the application icon 5008h. In response to detecting user input to select the application icon 5008h, device 100 opens the news application (e.g., launches, starts execution, or otherwise displays its user interface) as shown in Figure 5I4. Figure 5I4 shows an exemplary user interface of the application, including a “Shared with You” section 546. In some embodiments, the “Shared with You” section 546 includes a list of one or more books shared with users of device 100 (e.g., via a communication application such as a messaging application). For example, the “Shared with You” section 546 includes a representation of book title A, which includes an image representing book title A, and other information about the book (e.g., text information) (e.g., title, author, summary, etc.). In some embodiments, the representation of book title A further includes a sharing indicator 547 that provides information about who shared the individual book. For example, Lucy shared book title A with users of device 100. In some embodiments, the shared display 547 is selectable, and depending on the selection of the shared display 547, the user is provided with the option to send a message to the user who shared the book (e.g., Lucy) and / or is provided with additional shared information.
[0219] In some embodiments, upon detecting user input on the book's representation, device 100 displays (e.g., opens) a preview of the selected book (e.g., for the user to view) and / or opens a user interface for purchasing the selected book.
[0220] Figure 5I5 shows device 100 detecting user input 548 (e.g., tap input) to select an App Store application icon 5008o. In response to detecting user input to select an App Store application icon 5008o, device 100 opens the App Store application (e.g., launches, starts execution, or otherwise displays its user interface) as shown in Figure 5I6. Figure 5I6 shows an exemplary user interface of the App Store application, including a “Shared with You” section 549. In some embodiments, the “Shared with You” section 549 includes a list of one or more applications that have been shared with users of device 100 (e.g., via a communication application such as a messaging application). For example, the “Shared with You” section 549 includes a representation of App A, including an image representing App A, and other information about the App (e.g., text information) (e.g., title, publisher, rating, etc.). In some embodiments, the representation of App A further includes a sharing display 550 that provides information about who shared the individual app. For example, Joe shared App A with users of device 100. In some embodiments, the shared display 550 is selectable, and depending on the selection of the shared display 550, the user is provided with the option to send a message to the user who shared the app (e.g., Joe) and / or is provided with additional shared information.
[0221] In some embodiments, upon detecting user input on the app's representation, device 100 displays a user interface for downloading the selected app (e.g., Open) and / or launches the individual app (e.g., if the app is already installed on device 100).
[0222] FIG. 5I7 shows a device 100 that detects a user input 551 (e.g., a tap input) to select a podcast application icon 5008n. In response to detecting a user input to select the podcast application icon 5008n, the device 100 opens (e.g., launches, starts execution of, or otherwise displays a user interface therefor) the podcast application, as shown in FIG. 5I8. FIG. 5I8 shows an exemplary user interface of the podcast application that includes a “Shared with You” section 552. In some embodiments, the “Shared with You” section 552 includes a list of one or more podcasts that are shared with the user of the device 100 (e.g., via a communication application such as a messaging application). For example, the “Shared with You ” section 552 includes a representation of Podcast A that includes an image representing Podcast A and other information about the podcast (e.g., text information) (e.g., title, author name, episode number, etc.). In some embodiments, the representation of Podcast A further includes a share display 553 that provides information about who shared the individual podcast. For example, Joe shared Podcast A with the user of device 100. In some embodiments, the share display 553 is selectable, and in response to selecting the share display 553, the user is provided with an option to send a message to the user who shared the podcast (e.g., Joe), and / or additional share information is provided.
[0223] In some embodiments, in response to detecting a user input on a representation of a podcast, the device 100 displays (e.g., opens) the user interface of the podcast (e.g., together with additional information about the podcast), and / or starts playback of the podcast.
[0224] FIG. 5I9 shows a device 100 that detects a user input 554 (e.g., a tap input) to select a TV (“TV”) application icon 5008e. In response to detecting a user input to select the TV application icon 5008e, the device 100 opens (e.g., launches, starts execution of, or otherwise displays a user interface therefor) the TV application, as shown in FIG. 5I10. FIG. 5I10 shows an exemplary user interface of the TV application that includes a “Shared with You” section 555. In some embodiments, the “Shared with You” section 555 includes a list of one or more TV programs, movies, or other media (e.g., video) items (e.g., sports events, etc.) that are shared with the user of the device 100 (e.g., via a communication application such as a messaging application). For example, the “Shared with You” section 555 includes a representation of a TV program that includes an image representing the TV program and other information related to the TV program (e.g., text information) (e.g., title, names of actors, episode number, etc.). In some embodiments, the representation of the TV program further includes a shared display 556 that provides information about who shared the individual TV program. For example, Alex shared a TV program with the user of the device 100 (e.g., Alex also shared movies and sports events). In some embodiments, the shared display 556 is selectable, and in response to selecting the shared display 556, the user is provided with an option to send a message to the user (e.g., Alex) who shared the TV program and / or additional shared information is provided.
[0225] In some embodiments, in response to detecting a user input on the representation of the TV program, the device 100 displays (e.g., opens) the user interface of the TV program (e.g., together with additional information about the TV program) and / or starts playback of the TV program.
[0226] Figures 6A to 6D show exemplary user interfaces for initializing the sharing function. In some embodiments, the exemplary user interfaces shown in Figures 6A to 6D are displayed automatically (e.g., without user input) according to the determination that the user has not turned on the sharing function (e.g., in "Settings"). In some embodiments, the exemplary user interfaces shown in Figures 6A to 6D are displayed by device 100 according to the determination that device 100 has become capable of using the function (e.g., the first time the device becomes capable of using the sharing function). For example, Figures 6A to 6D include an introductory user interface for guiding the user through the various capabilities and functions available to the shared media item.
[0227] Figure 6A shows the “Share with You” user interface 6000. In some embodiments, the “Share with You” user interface 6000 includes text information 6002 that provides information about how shared media items will appear in various applications. For example, in some embodiments, shared media items received by device 100 within one or more communication applications (e.g., messaging applications, email applications, etc.) will automatically appear in one or more media applications (e.g., separate from the one or more communication applications) without requiring any input. For example, text information 6002 provides an overview of how device 100 will handle shared media items. In some embodiments, the type of shared media item determines which media application(s) the individual media item will appear in. For example, shared photos and / or videos will appear in the photos application, shared songs will appear in the music application, and shared TV shows and / or movies will appear in the TV application, and so on.
[0228] In some embodiments, representations of one or more media applications enabling the integration of shared media are displayed on the “Shared with You” user interface 6000. For example, application icons for each media application are displayed, such as the photo application 6004a, the application store 6004b, the news application 6004c, this application 6004d, the TV application 6004e, the internet browser application 6004f, and the music application 6004g. It will be understood that the examples of media applications shown in Figure 6A are not an exhaustive list. For example, the media applications enabling the integration of shared media items may include additional media applications, or a subset of the media applications shown in Figure 6A, but not all of them.
[0229] In some embodiments, the “Share with You” user interface 6000 includes a navigation bar 6006 that updates based on the currently displayed user interface. For example, each dot represents another user interface in a sequence of user interfaces (e.g., a user interface onboarding sequence). For example, in Figure 6A, the leftmost dot (of the three dots) is filled in to indicate that the currently displayed user interface is the first user interface in the sequence of user interfaces. In some embodiments, a swipe input (e.g., left and / or right) detected on device 100 causes device 100 to navigate between user interfaces in a set of user interfaces. For example, a swipe to the left causes device 100 to switch the display of user interface 6000 (shown in Figure 6A) to user interface 6012 (shown in Figure 6B).
[0230] In some embodiments, the “Share with you” user interface 6000 includes a button 6008 (e.g., “OK”) which, when selected, causes the device to display the next user interface in the sequence of user interfaces (e.g., replacing the display of user interface 6000 in Figure 6A with the display of user interface 6012 in Figure 6B). In some embodiments, selecting button 6008 causes the device 100 to end the sequence of user interfaces (e.g., returning to the home screen, such as interface 400 shown in Figure 4A).
[0231] In some embodiments, the “Share with you” user interface 6000 includes a user-selectable option 6010 (e.g., “Edit settings”) that allows the user to update the functionality of shared media items for different applications. In some embodiments, in response to device 100 detecting a user selection of the “Edit in settings” option 6010 (e.g., user input selection), device 100 opens the settings user interface, as described in more detail with reference to Figures 7A–7D.
[0232] In some embodiments, in response to detecting user input (e.g., a swipe to the left) for navigating to the next user interface in a sequence of user interfaces, device 100 displays the “Share with You in Photos” user interface 6012. In some embodiments, the “Share with You in Photos” user interface 6012 provides text information 6014 about how a shared media item (e.g., a shared photo and / or shared video) is integrated with a media application (e.g., a Photos application). In some embodiments, the “Share with You in Photos” user interface 6012 includes one or more examples of shared media items (e.g., and a sharing indicator displayed with the shared media item). For example, a shared media item 6016a includes a sharing indicator 6018a that shows information about how the media item was shared (e.g., “From Lucy”), and a shared media item 6016b includes a sharing indicator 6018b. In some embodiments, the sharing indicators 6018a and 6018b include a representation of the user who shared the media item (e.g., a saved contact photo, an emoticon representing the user, etc.).
[0233] In some embodiments, the “Share with you in photos” user interface 6012 includes a navigation bar 6006 that updates based on the currently displayed user interface in a sequence of user interfaces. For example, in Figure 6B, the middle dot (of three dots) is filled to indicate that the currently displayed user interface 6012 is the middle user interface in a sequence of user interfaces (for example, one user interface in the sequence (e.g., user interface 6000 in Figure 6A) precedes the current user interface, and one user interface (e.g., user interface 6020 in Figure 6C) follows the current user interface). In some embodiments, a swipe input (e.g., left and / or right) detected on device 100 causes device 100 to navigate between user interfaces in a sequence of user interfaces. For example, when a right swipe input is detected, device 100 replaces the display of user interface 6012 (shown in Figure 6B) with user interface 6000 (shown in Figure 6A), and when a left swipe input is detected, device 100 replaces the display of user interface 6012 (shown in Figure 6B) with user interface 6020 (shown in Figure 6C).
[0234] In some embodiments, the user interface 6012 also includes a button 6008 and an "Edit Settings" option 6010, as described above with reference to Figure 6A.
[0235] It will be understood that additional user interfaces for specific applications may be included in the sequence of user interfaces. For example, the “Share with you in photos” user interface 6012 shown in Figure 6B is an example of an information user interface for a first media application (e.g., a photos application). In some embodiments, an information user interface is displayed for different media applications (e.g., news, web browser, music, etc.) instead of (e.g., in addition to) user interface 6012.
[0236] In response to detecting user input (e.g., a detected left swipe input) for navigating to the next user interface in a sequence of user interfaces, device 100 displays a “Highlight” user interface 6020. In some embodiments, the “Highlight” user interface 6020 provides text information 6022 describing a function that allows the user to mark a particular media item (e.g., star, pin, highlight, or otherwise flag it). In some embodiments, the marked media item is displayed simultaneously with an indicator 6024 that shows the media item is a marked media item. In some embodiments, device 100 displays one or more marked media items in a dedicated portion of the user interface (e.g., to make the marked media items easier for the user to see). For example, part of the user interface of a separate media application includes a section for displaying marked media items.
[0237] In some embodiments, the “Highlight” user interface 6020 also includes a navigation bar 6006, a button 6008, and an “Edit Settings” option 6010, as described above with reference to Figures 6A and 6B.
[0238] Figure 6D shows an exemplary user interface for providing information about shared media item integration on device 6026. In some embodiments, device 6026 comprises a computer system communicatively coupled to a display 6028 (e.g., a television display). In some embodiments, device 6026 and display 6028 comprise the same device (e.g., device 6026 includes display 6028). In some embodiments, device 6026 is an example of a portable multifunction device 100. For example, device 6026 includes multiple applications, including a television application (e.g., corresponding to application icon 6004e shown in Figure 6A). In some embodiments, the television application is capable of displaying (e.g., streaming and presenting) television programs and / or movies on display 6028.
[0239] In some embodiments, device 100 displays a “Shared with You” user interface 6030. In some embodiments, the “Shared with You” user interface 6030 includes text information 6032 that provides information on how shared media items will be displayed in applications on device 6026 (e.g., in a TV application). For example, shared media items (e.g., TV shows and / or movies) received by device 100 within one or more communication applications (e.g., messaging applications, email applications, etc.) will be automatically displayed in one or more media applications (e.g., separate from the one or more communication applications) without requiring any input. For example, text information 6032 provides an overview of how device 100 will handle shared media items.
[0240] In some embodiments, the “Share with You” user interface 6030 includes a button 6034 that allows the user to enable the shared media integration feature on device 6026 (e.g., and / or display 6028), and an option 6036 that allows the user to opt out of enabling the shared media integration feature on device 6026. For example, depending on whether the user selects button 6034, device 6030 becomes capable of synchronizing shared content (e.g., received via a communications application) displayed within a media application (e.g., a TV application). In some embodiments, the user can update their selection (e.g., later) to enable the shared media integration functionality within the settings interface, as described with reference to Figures 7A–7D.
[0241] Figures 7A to 7D show exemplary user interfaces for enabling and / or disabling shared media integration features for multiple applications. Figure 7A shows an exemplary user interface 7002. In some embodiments, the user can navigate to the user interface 7002 using a settings application (for example, by selecting icon 446 shown in Figure 4A) (for example, through a series of user inputs).
[0242] In some embodiments, the user selects an application to enable and / or disable the highlighting feature (for example, as described with reference to Figures 5E5–5E8). For example, two or more applications enable the shared media integration feature (for example, based on controls described with reference to Figures 7A–7D), and the first of the two or more applications allows the highlighted content to be displayed in the first application, but the second of the two or more applications does not allow the highlighted content to be displayed in the second application (for example, the second application does not have a “Highlighting” section, and / or the content items are not displayed with the highlighting display (for example, as described with reference to Figure 5E8)). For example, a user interface is provided (for example, accessible using a settings application) that allows the user to enable and / or disable the highlighting feature of each application (for example, using toggle switches provided to each application in the list of applications, such as the toggle switches described with reference to Figures 7A–7D). Figure 7A shows multiple applications, including one or more media applications (e.g., applications 7006a to 7006l) and one or more communication applications (e.g., social media application 7008a and messaging application 7008b). In some embodiments, one or more media applications are displayed in different parts of the user interface than one or more communication applications (e.g., communication applications are grouped in one section, and media applications are grouped in a different section). In some embodiments, one or more media applications and one or more communication applications are not displayed in separate parts of the user interface 7002 (e.g., both types of applications are displayed in the same section, as shown in Figure 7A).The one or more media applications 7006a to 7006l shown in Figure 7A are examples of media applications, and it should be understood that additional or alternative media applications may be displayed in the user interface 7002.
[0243] In some embodiments, the user interface 7002 includes a toggle switch 7004 for controlling whether the shared media integration feature is enabled or disabled for all applications displayed on the user interface 7002 (e.g., applications 7006a-7006l and 7008a-7008b). For example, when the toggle switch 7004 is enabled, one or more applications can use the shared media integration feature (e.g., based on the current setting of toggle switches 7010a-7010n). In some embodiments, toggle switches 7010a-7010n are provided to each application. For example, each toggle switch 7010 can be turned on or off to enable or disable the shared media integration feature for individual applications. In some embodiments, as will be described in more detail below with respect to Figure 7C, when the toggle switch 7004 is disabled (e.g., set to off), no application uses the shared media integration feature (e.g., also referred to herein as the “Share with You” feature). For example, if toggle switch 7004 is set to "off," all toggle switches 7010a through 7010n will also be set to "off" (for example, the "Share with you" feature will be disabled for all applications).
[0244] In some embodiments, as shown in Figure 7A, device 100 automatically enables the “Share with you” feature (for example, by default) (for example, toggle switch 7004 is set to ON). In some embodiments, device 100 automatically enables the “Share with you” feature for all applications in the list shown in Figure 7A.
[0245] Figure 7B shows an exemplary user interface for a user of device 100 that requests the user to disable the "Share with you" feature for a first set of applications (for example, without disabling the "Share with you" feature for all the applications listed in Figure 7A). For example, Figure 7B shows that toggle switches 7010a (corresponding to the photo application 7006a), 7010e (corresponding to, for example, the TV application 7006e), 7010h (corresponding to, for example, the movie application 7006h), and 7010j (corresponding to, for example, the social media application 7008a) are disabled (as indicated, for example, by each of the unblocked toggle switches), while toggle switches 7010b (corresponding to, for example, the music application 1 7006b), 7010c (corresponding to, for example, the Safari (e.g., web browser) application 7006c), 7010d (corresponding to, for example, the news application 7006d), 7010f (corresponding to, for example, this application 7006f), 7010g (corresponding to, for example, the App Store application 7006g), and 7010i (corresponding to, for example, the music application 2 The toggle switches 7010k (for example, for the podcast application 7006j), 7010l (for example, for the messaging application 7008b), 7010m (for example, for the map application 7006k), and 7010n (for example, for the iTunes Store application 7006l) indicate that the feature is enabled (for example, as indicated by a blocked toggle switch). Thus, the user can choose which applications have the "Share with You" feature enabled and which do not.
[0246] For example, while disabling the "Shared with You" feature of the social media application 7008a, by continuing to enable the "Shared with You" feature of the messaging application 7008b, the device 100 synchronizes media items shared via the messaging application 7008b with individual media applications for individual media items, and does not synchronize media items shared via the social media application 7008a. Similarly, shared media items are not synchronized in media applications where the "Shared with You" feature is disabled (e.g., the toggle switch is set to off). For example, shared photos and / or videos (e.g., shared via the messaging application 7008b) are not displayed in the photo application 7006a because the toggle switch 7010a is set to off. Thus, in some embodiments, only shared media items that are shared via a communication application that enables use of the "Shared with You" feature and are to be displayed in a media application that enables use of the "Shared with You" feature are displayed to the user in an individual media application (e.g., the shared media item must meet both criteria). For example, if a shared media item is shared via a communication that does not enable use of the "Shared with You" feature, or if a shared media item is of a type of media item that is displayed in a media application that does not enable use of the "Shared with You" feature, the shared media item is not displayed to the user in an individual media application.
[0247] Figure 7C shows an exemplary user interface where toggle switch 7004 is set to off and the “Share with You” feature is disabled for all applications in the list of applications displayed in Figure 7C. In some embodiments, device 100 automatically sets toggle switch 7004 to off (e.g., without user input) (e.g., the user of device 100 can select which applications, if any, are enabled to use the “Share with You” feature). For example, when toggle switch 7004 is set to off, all of toggle switches 7010a to 7010n are also set to off (e.g., because the “Share with You” feature is disabled for all applications).
[0248] Figure 7D shows an exemplary user interface that appears when a user enables the “Share with You” feature for one or more applications. In some embodiments, the user selects one or more applications for which to enable the “Share with You” feature (without selecting one or more other applications for which the “Share with You” feature remains disabled). In some embodiments, the user must select at least one communications application (for example, one in which media items are shared) and at least one media application (for example, one in which individual media items are displayed (based on the type of media item)).
[0249] In some embodiments, the toggle switch 7004 is automatically switched from "off" to "on" (e.g., without user input) depending on whether the user has selected one or more applications for which to enable the "Share with you" feature (e.g., by toggling a toggle switch selected from toggle switches 7010a to 7010n). For example, the "Share with you" feature (as represented by toggle switch 7004) is also enabled depending on whether the user has selected to enable the "Share with you" feature for one application.
[0250] Figures 8A to 8D are flowcharts illustrating Method 800, which automatically makes content shared by one or more receiving applications available within a corresponding media application based on the content type. Method 800 is performed in a computer system (e.g., device 300 in Figure 3, or portable multifunction device 100 in Figure 1A) that communicates with (e.g., or includes) a display generating component (e.g., optionally a touch-sensitive display device, projector, head-mounted display, head-up display, etc.) and one or more input devices (e.g., including one or more sensors for detecting the intensity of contact with a touch-sensitive surface). In some embodiments, the display generating component is a touchscreen display, and the touch-sensitive surface is on or incorporated into the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations of Method 800 are optionally combined, and / or the order of some operations is optionally changed.
[0251] As described herein, Method 800 automatically makes content shared in a receiving application available within a related application that can open the shared content (e.g., the application in which the content is typically served, viewed, played, or listened to). Making the shared content automatically available within a related application improves the discoverability of the shared content without requiring additional user input and reduces the number of inputs required to perform actions. Furthermore, Method 800 aggregates shared content of the same content type into a separate application section dedicated to the shared content, reducing the number of inputs and cognitive burden on the user attempting to view or interact with the content. The reduced number of inputs required to interact with shared content items improves device usability and makes the user-device interface more efficient (e.g., by helping the user make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power consumption and improves battery life by allowing the user to use the device more quickly and efficiently.
[0252] The computer system receives information identifying a first shared content item (e.g., media content item such as music, photos, videos, books, articles or news, podcasts, television programs, or map locations or web links) through a first messaging conversation in a first receiving application among several applications (e.g., a first messaging application (e.g., represented by icon 5008d), a mail application (e.g., represented by icon 5008r), or a social media application) (802). In some embodiments, the information identifying the first shared content item includes a representation of the content item, a link to the content item, a preview of the content item that links to the content item, and a pointer or identifier of the content item that can be read by the corresponding host application. In some embodiments, instead of identifying information or links, a copy of the content item is shared.
[0253] In response to receiving information identifying a first shared content item (804) and determining that the first shared content item is of a first type (806), the computer system automatically makes the first shared content item available in a first application of a group of applications (e.g., without additional user input), and the first application is associated with content of the first type. For example, after information identifying a content item has been received by the receiving application, when a user opens the first application (e.g., by contact 5012 shown in Figure 5A), the item, a copy of the item, or a representation of the item (e.g., representation 5045a shown in Figure 5A6) automatically appears in a dedicated area (section 5054 shown in Figure 5A6) within the first application without requiring any action from the user. In some embodiments, a first application (e.g., a music application represented by icon 5008a) is designated as the default for processing a first type of content (e.g., music), and the first application processes the first type of content such that the music application processes music media content, the web browser processes web links, the podcast application processes podcasts, and the book application opens and displays digital books.
[0254] In response to receiving information identifying a first shared content item and determining that the first shared content item is of a second type (808), the computer system automatically makes the first shared content item available in a second application among multiple applications, and the second application is associated with the second type of content. In some embodiments, the second type is different from the first type (810). In some embodiments, the first application is not associated with the second type of content, and the second application is not associated with the first type of content (e.g., it cannot open, play, or display the content). In some embodiments, the first receiving application is a messaging application (811). For example, content received by a receiving application (e.g., a messaging application) (e.g., recommendations received from contacts about movies, books, songs, videos, photos, podcasts, articles, places to visit, etc.) is made public or provided to various applications in which the user typically consumes such content (e.g., see or hear). In some embodiments, content items are stored within the messaging application but not within the first or second application (for example, the first and second applications refer to the messaging application to display shared media items).
[0255] Making different types of content shared in a receiving application automatically available within different applications that can open individual content types (e.g., applications that typically serve, view, play, or listen to that type of content) improves the discoverability of shared content without requiring additional user input and reduces the number of inputs needed to perform actions. Reducing the number of inputs required to interact with shared content items improves device usability and makes the user-device interface more efficient (e.g., by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, it reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0256] In some embodiments, the computer system receives information identifying a second shared content item from a second receiving application among multiple applications (e.g., an instant messaging application, a mail application, or a social media application) (812), the second receiving application being different from the first receiving application. In some embodiments, upon receiving information identifying a second shared content item and determining that the second shared content item is of a first type (e.g., music content), the computer system automatically makes the second shared content item available within the first application among multiple applications (e.g., representation 5045b of song B (also called "song title-B") shown in Figure 5A6). In some embodiments, upon receiving information identifying a second shared content item and determining that the second shared content item is of a second type (e.g., a book), the computer system automatically makes the second shared content item available within the second application among multiple applications (e.g., representation 546a in section 546 shown in Figure 5I4).
[0257] Making content shared across different receiving applications automatically available within different applications that can open individual content types (e.g., applications where that type of content is typically delivered, viewed, played, or listened to) improves the discoverability of shared content without requiring additional user input and reduces the number of inputs required to perform actions. Furthermore, Method 800 aggregates the same type of content shared across different receiving applications into dedicated sections for each application, reducing the number of inputs and cognitive burden on users attempting to view or interact with the content. Reducing the number of inputs required to interact with shared content items improves device usability and makes the user-device interface more efficient (e.g., by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0258] In some embodiments, information identifying a first shared content item is received at a first location of a first receiving application, and the computer system receives information identifying a third shared content item at a second location of the first receiving application (814). In response to receiving information identifying a third shared content item and determining that the third shared content item is of a first type, the computer system automatically makes the third shared content item available in a first application of a plurality of applications. In response to receiving information identifying a third shared content item and determining that the third shared content item is of a second type, the computer system automatically makes the third shared content item available in a second application of a plurality of applications. For example, a second type of content (e.g., podcast name A) is shared in a messaging conversation with Alex, as shown in Figure 5A4 (for example, a shared podcast is a different content type from songs and news articles shared in a messaging conversation).
[0259] Making content shared across different messaging conversations automatically available within various applications that can open individual content types (e.g., applications where that type of content is typically delivered, viewed, played, or listened to) improves the discoverability of shared content without requiring additional user input and reduces the number of inputs needed to perform actions. Furthermore, aggregating the same type of shared content into specific sections of individual applications can reduce the number of inputs and cognitive burden on users attempting to view or interact with the content. Reducing the number of inputs required to interact with shared content items improves device usability and makes the user-device interface more efficient (e.g., by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces device power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0260] In some embodiments, the first shared content item is of a first type (816), and the computer system receives user input to add the first shared content item to the library of the first application (e.g., adding a song to the Apple Music library or another music application library, bookmarking a web link in Safari or another browser, adding a video to the Apple TV library or another video / TV library). In some embodiments, in response to receiving user input to add the first shared content item to the library of the first application, the computer system adds the first shared content item to the library of the first application (e.g., saving or storing the shared content in a library associated with a user of an electronic device that can be stored in the cloud or accessed from multiple electronic devices). In some embodiments, the first shared content item is not automatically added to the library of the first application in response to receiving information identifying the first shared content item (818).
[0261] In related applications where individual content types can be opened without automatically adding shared content to the related application's content library, automatically making different types of shared content available in the receiving application improves the discoverability of shared content, gives the user control over the sharing process, and thus reduces the number of inputs required to perform the action. Reducing the number of inputs required to interact with shared content items improves device usability and makes the user-device interface more efficient (for example, by helping the user make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power usage and improves battery life by allowing the user to use the device more quickly and efficiently.
[0262] In some embodiments, upon receiving information identifying a first shared content item and determining that the first shared content item is of a second type (e.g., a photograph or video), the computer system automatically adds the first shared content item to the library in a second application among multiple applications (820). For example, a copy of the content item is not only automatically made available in the second application but is also stored (e.g., added) to the library of the second application. In some embodiments, the copy of the content item remains in the library unless the user takes action to remove it (e.g., by using control 5090d shown in Figure 5D4). For example, a copy of a content item stored in the library is not dynamically updated by the computer system. In some embodiments, the second type of content item is not available in any of the multiple applications unless it is user-generated and shared in the receiving application, or manually added to the libraries of each of the multiple applications, or generated and stored by the user of the electronic device.
[0263] Automatically adding shared content to the content library of related applications, while simultaneously making different types of content shared by the receiving application available in related applications that can open individual content types, improves the discoverability of shared content and reduces the number of inputs required to perform actions. Reducing the number of inputs required to interact with shared content enhances device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces device power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0264] In some embodiments, the computer system aggregates (e.g., groups or places into sets) multiple pieces of information that identify each of the first types of shared content items for display in a first application (822). In some embodiments, the computer system aggregates (e.g., groups or places into sets) multiple pieces of information that identify each of the second types of shared content items for display in a second application. For example, one type of shared content items may be grouped into one set, and another type of content items may be grouped into another set. In some embodiments, the multiple pieces of information that identify each shared content item may include information shared by different users, information shared at different times and / or in different conversations.
[0265] Automatically marking different types of content for sharing, and / or automatically grouping similar types of shared content for display in dedicated application sections for shared content, improves the discoverability of shared content and reduces the number of inputs and cognitive burden on the user when trying to view or interact with shared content. Reducing the number of inputs required to interact with shared content improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0266] In some embodiments, the computer system displays multiple pieces of information identifying each of a first type of shared content item in a dedicated section of a first application (e.g., section 5054 “Shared with You” shown in Figure 5A6) and multiple pieces of information identifying each of a second type of shared content item in a dedicated section of a second application (e.g., section 546 “Shared with You” shown in Figure 5I4) (824). For example, the first application includes a section that displays first type content items shared in the receiving application, and the second application includes a section that displays second type content items shared in the receiving application. Thus, content items of the same type are grouped so that they are displayed in the “Shared with You” section of each application. In some embodiments, not all content items of the same type identified as shared are grouped for display; for example, only a subset of shared content items are displayed at such a time (e.g., only the most relevant content items are displayed, and relevance may be determined based on various criteria).
[0267] Automatically displaying the same type of shared content in dedicated application sections improves the discoverability of shared content and reduces the number of inputs and cognitive burden on the user when viewing or interacting with shared content. Reducing the number of inputs required to interact with shared content improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0268] In some embodiments, the computer system receives information identifying a first set of shared content items in a first receiving application (e.g., a messaging application) (826), wherein the first set of shared content items is of a first type. After receiving the information identifying the first set of shared content items, the computer system receives a request to open the first application via one or more input devices (e.g., a user can tap an icon representing the first application). In response to receiving the request to open the first application, the computer system simultaneously displays the information identifying the first set of shared content items in a dedicated section of the first application (section 5054 in Figure 5A6) and a second set of content items of a first type in a part of the first application other than the dedicated section (e.g., section 5056 of recently played music in Figure 5A6). For example, when a user of an electronic device opens a first application, the default user interface (e.g., default view 5052 in Figure 5A6) is displayed, which includes content items that the user has added, saved, played, or favorited in one section, as well as content items recommended to the user, or content items shared with the user via a messaging application in another separate section (e.g., section 5054).
[0269] Displaying shared content (within a dedicated section) and non-shared content (in other sections) simultaneously within each application improves the discoverability of shared and non-shared content and reduces the number of inputs and cognitive burden on users attempting to view or interact with shared content. Reducing the number of inputs required to interact with shared and non-shared content enhances device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces device power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0270] In some embodiments, the first group of shared content items are of a first type (828), and according to a determination that certain criteria are met for a content item within the first group of shared content items, the computer system removes the information identifying the content item from display in a dedicated section of the first application. For example, content items included in the first group for display are dynamically updated based on one or more criteria, but not limited to, when the information identifying the content was received, whether the shared content was consumed after the information identifying the content was received, whether the shared content was manually highlighted or otherwise marked as more important than other unmarked content, whether the information identifying the content was received in a messaging conversation that was marked as more important than other unmarked messaging conversations, or whether the information identifying the content was sent by a particular contact, such as a contact that was marked as more important or a contact that the user of the electronic device frequently connects with.
[0271] Removing content from groups of content items intended for display in a section dedicated to shared content, based on various predetermined criteria, improves the relevance of displayed content and reduces the number of inputs required to discover shared and relevant content. Reducing the number of inputs required to perform actions improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0272] In some embodiments, the predetermined criterion is a time-based criterion (830), and according to the determination that a predetermined amount of time (e.g., 30 days) has elapsed since the content item was shared, the computer system removes the information identifying the content item from the display in a dedicated section of the first application.
[0273] Attenuating shared content within a group within a given time frame improves the relevance of displayed content and reduces the number of inputs required to discover shared and relevant content. Reducing the number of inputs required to perform actions enhances device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0274] In some embodiments, the dedicated sections of the first application and the dedicated sections of the second application are visually similar (832). In some embodiments, the dedicated sections of the first application and the dedicated sections of the second application are associated with the same attribute (e.g., the same tab, label, or name associated with the dedicated sections) indicating that the content displayed in each dedicated section is shared. For example, the attribute is a text label such as “Shared with you” or “From your friends.” The attribute is also the same color and size in both the first and second applications. For example, sections 5054 of the music application (shown in Figure 5A6), section 542 of the news application (shown in Figure 5I2), section 546 of the main application (shown in Figure 5I4), section 549 of the App Store application (shown in Figure 5I6), section 552 (shown in Figure 5I8), and section 555 of the TV application are visually similar and all are labeled “Shared with you.”
[0275] Having dedicated sections for visually similar shared content in different applications improves the discoverability of shared content across different applications and provides users with visual feedback indicating that the content in each dedicated section is shared content, even if the content in different applications is not visually consistent or similar. Providing users with improved visual feedback enhances device usability, increases the efficiency of the user-device interface (for example, by helping users provide appropriate input when operating / interacting with the device and reducing user errors), and further reduces device power consumption and improves battery life by enabling users to use the device more quickly and efficiently.
[0276] In some embodiments, the first shared content item is of a first type (834), and making the first shared content item available automatically (e.g., without additional user input) within the first application includes providing within the first application a mechanism (e.g., capsule 5056a shown in Figure 5A6) for navigating to the location within the first receiving application where information identifying the first shared content item was first received (e.g., as described in relation to Figures 5A7, 5A8, and 5B3 above). For example, by making the shared content available in the corresponding application, the user can open the content from the corresponding application (e.g., by selecting representation 5054a) and also refer to the location in the receiving application where the content item was first shared (e.g., by selecting capsule 5056a shown in Figure 5A6).
[0277] Providing a mechanism within a media application displaying shared content to navigate to the location in the receiving application where the shared content was first shared improves the discoverability of shared content and reduces the number of inputs required to find shared and relevant content, while also improving communication with the contact who shared the content. Reducing the number of inputs required to perform an action enhances the usability of the device, making the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, it reduces power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0278] In some embodiments, providing a mechanism within the first application for navigating to the location within the first receiving application where information identifying the first shared content item was first received includes the computer system automatically scrolling to the location within the first receiving application where the first shared content item was first received (e.g., a message) (e.g., a first messaging conversation within the first messaging application) (836). In some embodiments, the receiving application is a messaging application with a messaging conversation, and the messaging conversation includes multiple messages (e.g., the computer system displays message 5001d last, as described in Figure 5A8 above). For example, by making the shared content available in the corresponding application, a user can open the content from the corresponding application and also refer to the message in the messaging application where the content item was first shared.
[0279] Providing a mechanism within a media application that displays shared content to directly navigate to the message in the messaging conversation where the shared content was first shared improves the discoverability of shared content and reduces the number of inputs required to find shared and relevant content, while also improving communication with the contact who shared the content. Reducing the number of inputs required to perform an action enhances the usability of the device and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0280] In some embodiments, the request to share the first shared content item is initiated from a remote messaging application on a remote electronic device (838).
[0281] In some embodiments, multiple contacts are stored in a contacts application among multiple applications (840), and before the computer system automatically makes a first shared content item available in a first or second application, it determines that information identifying the first shared content item has been received from one or more of the multiple contacts.
[0282] Determining whether shared content was shared by a contact (rather than a non-contact user) before automatically making it available in related applications improves the relevance of shared content displayed in those applications and reduces the number of inputs required to perform actions. Reducing the number of inputs required to interact with shared content improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0283] In some embodiments, before automatically making a first shared content item available within a first or second application, the computer system determines that information identifying the first shared content item as shared content has been received from one or more contacts that meet communication criteria (e.g., recently communicated contacts, regularly communicated contacts, favorites list, contacts VIP list, recently added friends, etc.) (842).
[0284] Determining whether shared content was shared by a contact that meets certain communication criteria (rather than any contact or user) before automatically making it available in related applications improves the relevance of shared content displayed in related applications and reduces the number of inputs required to perform actions. Reducing the number of inputs required to interact with shared content improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors). In addition, it reduces power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0285] In some embodiments, before automatically making a first shared content item available within a first or second application, the computer system determines that information identifying the first shared content item has been received from one or more contacts that meet a communication threshold level (e.g., recently communicated with the contact, regularly communicated with the contact) (844). In some embodiments, the communication threshold level may be predetermined or may change dynamically, for example, based on the behavior of the computer system's user.
[0286] Determining whether shared content was shared by a contact that meets a communication threshold level (rather than any contact or user) before automatically making shared content available in related applications improves the relevance of shared content displayed in related applications and reduces the number of inputs required to perform actions. Reducing the number of inputs required to interact with shared content improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0287] In some embodiments, multiple contacts are stored in a contacts application among multiple applications (846), and before automatically making a first shared content item available in a first or second application, the computer system determines that information identifying the first shared content item has been received by a group conversation that meets a communication threshold level, and two or more of the multiple contacts are participants in the group conversation.
[0288] Determining whether shared content was shared in a group conversation that meets a communication threshold level (rather than just any group conversation) before automatically making shared content available in related applications improves the relevance of shared content displayed in related applications and reduces the number of inputs required to perform actions. Reducing the number of inputs required to interact with shared content improves device usability and makes the user-device interface more efficient (for example, by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0289] Figures 9A to 9F are flowcharts illustrating a method 900 for automatically displaying shared content within a media application based on whether a sharing function is enabled, according to several embodiments. The method 900 is performed in a computer system (e.g., device 300 in Figure 3, or portable multifunction device 100 in Figure 1A) that communicates with a display generating component (e.g., optionally a touch-sensitive display device, projector, head-mounted display, head-up display, etc.) and one or more input devices (e.g., including one or more sensors for detecting the intensity of contact with a touch-sensitive surface). In some embodiments, the display generating component is a touchscreen display, and the touch-sensitive surface is on or incorporated into the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations of the method 900 are optionally combined, and / or the order of some operations is optionally changed.
[0290] As described herein, Method 900 determines when an individual content item has been identified and / or shared with a user using a receiving application and automatically displays the individual content item in the corresponding application without requiring the user to provide input to select or add the item within the corresponding application. The Method automatically displays the individual item if the user of the electronic device has permitted this function (for example, if the sharing setting of the corresponding application is enabled). By performing an operation (e.g., automatically) when a set of conditions are met without requiring further user input, the usability of the system is improved and the user-device interface becomes more efficient (e.g., by assisting the user in achieving the intended result and by reducing user failures during system operation / interaction with the system), which in turn allows the user to use the device more quickly and efficiently, reducing the device's power usage and improving battery life.
[0291] The computer system receives information identifying a first shared content item as shared content from a receiving application among several applications (e.g., an instant messaging application, a mail application, or a social media application) (902). The first shared content item is of a first type. In some embodiments, the information identifying the first shared content item is a representation of the content item and / or a link to the content item (e.g., not the content item itself). In some embodiments, the information identifying the first shared content item is a pointer or identifier of the content item that can be read by a corresponding application designed to open such content (e.g., a media application). In some embodiments, a copy of the content item itself, rather than identification information or a link, is shared. For example, as illustrated with reference to Figure 5A4, information identifying the first shared content item (e.g., representation 5045b) is received by a messaging application.
[0292] In response to receiving information identifying a first shared content item as shared content (904), and in accordance with the determination that a first application among multiple applications can open a first type of content (e.g., the application can open and / or display the content), and in accordance with the determination that it is permitted to provide the first application with information identifying the first shared content item as shared content (906), the computer system automatically makes the first shared content item available within the first application (e.g., without additional user input); and in accordance with the determination that it is not permitted to provide the first application with information identifying the first shared content item as shared content (908), the computer system does not automatically make the first shared content item available within the first application. For example, in response to a user opening the first application (e.g., after information identifying the content item has been received by the messaging application), the content item, a copy of the content item, and / or a representation of the content item automatically appear in a dedicated area within the first application without requiring any action from the user. For example, the user does not need to open the messaging application and select the content item to display in the first application. For example, whether information about content identified as being shared in a messaging application is provided to the corresponding application (e.g., an application designed to read and open such content) can be enabled or disabled for specific applications and / or content types. For instance, as illustrated with reference to Figures 5A4–5A6, a representation of song A shared in a messaging conversation with Alex will appear in the music application shown in Figure 5A6.
[0293] The ability to enable or disable the sharing of information that identifies content items as shared content improves the discoverability of that content within the relevant application without requiring further user input, while providing control over the sharing process. Performing an operation (e.g., automatically) when a set of conditions are met, without requiring further user input, improves system usability and makes the user-device interface more efficient (e.g., by assisting the user in achieving their intended results and by reducing user failures during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0294] In some embodiments, the computer system receives information from a messaging application identifying a second shared content item as shared content (910), wherein the second shared content item is of a second type, different from the first content type. In some embodiments, upon receiving information identifying the second shared content item as shared content, and in accordance with a determination that a second application, different from the first application among a plurality of applications, can open the second type of content and is permitted to provide the second application with information identifying the second shared content item as shared content, the computer system automatically makes the second shared content item available within the second application (e.g., without additional user input). For example, if a user opens the second application after the information identifying the content item has been received by the messaging application, the content item, a copy of the content item, or a representation of the content item automatically appears in a dedicated area within the first application without requiring any action from the user (e.g., the user does not need to open the messaging application and select the content item to display it in the second application). In some embodiments, the computer system does not automatically make the second shared content item available within the first application, based on a determination that it is not permitted to provide the second application with information identifying the second shared content item as shared content. For example, whether information about content identified as shared in a messaging application is provided to the corresponding application (e.g., an application designed to read and open such content) can be enabled or disabled for specific applications and / or content types. For example, a second type of content (e.g., podcast name A) is shared in a messaging conversation with Alex, as shown in Figure 5A4 (e.g., a shared podcast is a different content type from songs and news articles shared in a messaging conversation).
[0295] The ability to enable or disable the sharing of information that identifies content items as shared content for various applications and / or content types improves the discoverability of content items within the relevant applications without requiring further user input, while providing control over the sharing process. Performing operations (e.g., automatically) when a set of conditions are met without requiring further user input improves system usability and makes the user-device interface more efficient (e.g., by assisting the user in achieving intended results and by reducing user failures during system operation / interaction with the system). This further enables users to use the device more quickly and efficiently, reducing device power consumption and improving battery life.
[0296] In some embodiments, the default setting of the computer system is to enable automatic content sharing from a messaging application to multiple applications (912). For example, as shown in Figure 7A, initially all listed applications enable automatic contact sharing. In some embodiments, the computer system receives user input requesting to include a new application in the multiple applications. In some embodiments, in response to user input requesting to include a new application in the multiple applications, the computer system includes the new application in the multiple applications (for example, the new application may be downloaded or installed on the computer system). In some embodiments, the computer system receives user input requesting to turn off the default setting for the new application (for example, the user requests to disable automatic sharing for the new application). For example, as shown in Figure 7B, the user uses a toggle switch to "turn off" automatic content sharing for one or more applications. In some embodiments, in response to user input requesting to turn off the default setting for the new application, the computer system includes the new application in a set of applications that are not allowed to automatically share content from the messaging application to the set of applications.
[0297] The ability to disable automatic content sharing from messaging applications to various applications and / or content types by blacklisting applications improves the discoverability of content items within the relevant applications without requiring further user input, while providing control over the sharing process. Performing an operation (e.g., automatically) when a set of conditions are met, without requiring further user input, improves system usability and makes the user-device interface more efficient (e.g., by assisting the user in achieving their intended results and by reducing user failures during system operation / interaction with the system). This further enables users to use the device more quickly and efficiently, reducing device power consumption and improving battery life.
[0298] In some embodiments, the computer system receives information from the messaging application identifying a second shared content item as shared content (914), wherein the second shared content item is of a second type (e.g., different from the first type). In some embodiments, in response to receiving information identifying a second shared content item as shared content, and according to the determination that (i) a second application can open content of the second type, and (ii) the second application is included in a set of applications that are not permitted to automatically share content from the messaging application to a set of applications, the computer system does not make the second shared content item automatically available within the second application.
[0299] The ability to disable automatic content sharing from messaging applications to various applications and / or content types, so that shared content does not automatically appear in blacklisted applications, provides users with improved visual feedback by reducing the amount of clutter displayed in accessed applications when users access blacklisted applications. Providing users with improved visual feedback enhances system usability and makes the user-device interface more efficient (for example, by helping users achieve their intended results and by reducing user failures during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0300] In some embodiments, the default setting of the computer system is to not allow automatic content sharing from the messaging application to multiple applications (916). For example, as shown in Figure 7C, in some embodiments, “Share with you” is set to off for all applications (for example, as shown by the toggle switch 7004 set to off). In some embodiments, the computer system receives user input requesting to include a new application in multiple applications. In some embodiments, in response to user input requesting to include a new application in multiple applications, the computer system includes the new application in multiple applications (for example, the new application may be downloaded or installed on the computer system). In some embodiments, the computer system receives user input requesting to turn off the default setting for the new application (for example, the user requests to enable automatic sharing for the new application). In some embodiments, in response to user input requesting to turn off the default setting for the new application, the computer system includes the new application in the set of applications that are allowed to automatically share content from the messaging application to the set of applications. For example, as shown in Figure 7D, the user can turn on the “Share with you” setting for one or more applications.
[0301] The ability to enable automated content sharing for various applications and / or content types by whitelisting applications improves the discoverability of shared content items within the relevant applications without requiring further user input, while providing control over the sharing process. By performing actions (e.g., automatically) when a set of conditions are met, without requiring further user input, system usability is improved, and the user-device interface becomes more efficient (e.g., by assisting the user in achieving their intended results and by reducing user errors during system operation / interaction with the system). This further enables users to use the device more quickly and efficiently, reducing device power consumption and improving battery life.
[0302] In some embodiments, the computer system receives information from the messaging application identifying a second shared content item as shared content (918), wherein the second shared content item is of a second type. In some embodiments, in response to receiving information identifying a second shared content item as shared content, and according to the determination that (i) a second application can open content of the second type, and (ii) the second application is included in a set of applications that are permitted to automatically share content from the messaging application to a set of applications, the computer system automatically makes the second shared content item available within the second application.
[0303] The ability to enable automatic content sharing for various applications and / or content types, so that shared content automatically appears in whitelisted applications, provides users with improved visual feedback by displaying content without requiring them to navigate complex menu hierarchies to manually request the display of shared content within whitelisted applications. By providing users with improved visual feedback and performing operations (e.g., automatically) without requiring further user input when a set of conditions are met, the system's usability is improved, the user and device interface becomes more efficient (e.g., by assisting users in achieving their intended results and by reducing user failures when operating / interacting with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0304] In some embodiments, the receiving application is a messaging application, and information identifying a first shared content item as shared content is received in a first conversation of a plurality of conversations in the messaging application (920), and the default setting of the computer system is to enable automatic content sharing from a plurality of conversations in the messaging application to a plurality of applications. For example, information identifying a first shared content item is received in a communication thread in the messaging application. In some embodiments, the communication thread may include the exchange of messages between a user of an electronic device and one or more contacts stored on the electronic device (e.g., stored in a contacts application). In some embodiments, the communication thread may include a single message containing information identifying content as shared. In some embodiments, the message may include text. In some embodiments, the message may also include and / or identify other types of content, such as media content (e.g., photos, videos, GIFs, podcasts, applications, songs, etc.). In some embodiments, the computer system receives a new conversation in the messaging application and, in response to receiving a new conversation in the messaging application, includes the new application in the plurality of conversations. For example, a new conversation may be initiated by a user of a computer system by sending a message from a messaging application, or a new conversation may be a response to a user of a computer system receiving a message from a contact in a messaging application. In some embodiments, the computer system receives user input requesting that the default setting for new conversations be turned off (for example, the user requests that automatic sharing of new conversations be disabled).In some embodiments, in response to user input requesting to turn off the default setting for new conversations, the computer system includes the new conversation in a set of conversations that are not allowed to automatically share content from that set to multiple applications (e.g., a blacklist of messaging conversations for which automatic sharing is disabled).
[0305] The ability to disable automatic content sharing from specific message threads within a messaging application by blacklisting those threads without restricting automatic content sharing across all message threads within the messaging application provides improved visual feedback to the user by automatically displaying shared content only from approved message threads, without requiring user input to manually sort the shared content the user wants to see, and without displaying shared content received from blacklisted message threads. By performing an operation (e.g., automatically) without requiring further user input when a set of conditions are met, and providing the user with improved visual feedback, the system's usability is improved, the user-device interface becomes more efficient (e.g., by helping the user achieve their intended results and by reducing user failures when operating / interacting with the system), which in turn allows the user to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0306] In some embodiments, the computer system receives information in the second conversation identifying a second shared content item as shared content (922), wherein the second shared content item is of type first. In some embodiments, in response to receiving information identifying the second shared content item as shared content, and according to the determination that (i) the first application can open content of type first, and (ii) the second conversation is included in a set of conversations from which automatic content sharing to multiple applications is not permitted, the computer system does not make the second shared content item automatically available within the first application. For example, as shown in Figure 5G3, the user can use a toggle switch 527 to enable “Hide Sharing with You” to control whether the selected conversation is enabled to use the automatic content sharing feature.
[0307] The ability to disable automatic content sharing for specific messaging conversations (regardless of content type) improves the discoverability of shared content items within the relevant application without requiring further user input, while providing control over the sharing process. Performing actions (e.g., automatically) when a set of conditions are met, without requiring further user input, improves system usability and makes the user-device interface more efficient (e.g., by assisting users in achieving their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0308] In some embodiments, the receiving application is a messaging application, and information identifying a first shared content item as shared content is received in a first conversation of a plurality of conversations in the messaging application (924), and the default setting of the computer system is not to allow automatic content sharing from a plurality of conversations in the messaging application to a plurality of applications. For example, information identifying a first shared content item is received in a communication thread within the messaging application. In some embodiments, the communication thread may include the exchange of messages between a user of an electronic device and one or more contacts stored on the electronic device (e.g., stored in a contacts application). In some embodiments, the communication thread may include a single message containing information identifying content as shared. In some embodiments, the message may include text. In some embodiments, the message may also include and / or identify other types of content, such as media content (e.g., photos, videos, GIFs, podcasts, applications, songs, etc.). In some embodiments, the computer system receives a new conversation in a messaging application, and in response to receiving a new conversation in a messaging application, including a new application for multiple conversations, the computer system receives user input requesting that a default setting for the new conversation be turned off (for example, the user requests that automatic sharing for the new conversation be disabled). In some embodiments, in response to user input requesting that a default setting for the new conversation be turned off, the computer system adds the new conversation to a set of conversations that are allowed to automatically share content from that set to multiple applications (for example, a blacklist of messaging conversations for which automatic sharing is disabled).
[0309] The ability to enable automatic content sharing from specific message threads within a messaging application by whitelisting those specific message threads, without enabling automatic content sharing across all message threads within the messaging application, provides the user with improved visual feedback by automatically displaying shared content only from approved (e.g., whitelisted) message threads, without requiring user input to manually sort the shared content the user wants to see, and without displaying shared content received from unauthorized message threads. By performing an operation (e.g., automatically) without requiring further user input when a set of conditions are met, and providing the user with improved visual feedback, the system's usability is improved, the user-device interface becomes more efficient (e.g., by helping the user achieve their intended results and by reducing user failures when operating / interacting with the system), which in turn allows the user to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0310] In some embodiments, the computer system receives information in the second conversation identifying a second shared content item as shared content (926), wherein the second shared content item is of type first. In some embodiments, in response to receiving information identifying the second shared item as shared content in the second conversation, and according to the determination that (i) the first application can open content of type first, and (ii) the second conversation is included in a set of conversations for which automatic content sharing from a set of conversations to multiple applications is permitted, the computer system makes the second shared content item available from within the first application automatically (e.g., without user input). For example, automatic content sharing is turned "on" for a message conversation with Alex, as described with reference to Figures 5A4-5A6, and therefore, music content shared by Alex in the message conversation (e.g., song A) is automatically displayed in the music application, as shown in Figure 5A6.
[0311] The ability to enable the sharing of information that identifies content items as shared content for a specific messaging conversation (regardless of content type) improves the discoverability of content items within the relevant application without requiring further user input, while providing control over the sharing process. Performing an operation (e.g., automatically) when a set of conditions are met without requiring further user input improves system usability and makes the user-device interface more efficient (e.g., by assisting the user in achieving their intended results and by reducing user failures during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0312] In some embodiments, the multiple applications include a configuration application (corresponding to the configuration application icon 446 in Figure 4A, which provides the user with options to adjust settings related to electronic devices (e.g., system settings, network and connectivity settings, battery settings, display settings) and settings related to applications stored in the computer system) (928). In some embodiments, the computer system receives an input requesting to modify the content sharing settings, and in response to receiving such an input, the computer system displays a user interface for the content sharing settings, which includes a number of controls for authorizing the provision of information identifying content as shared to multiple applications. For example, a user of the computer system can use the user interface for the sharing-specific settings to authorize (or remove) the provision of information identifying content as shared. In some embodiments, the user interface for the content sharing settings is a user interface for a configuration application for adjusting content sharing and other settings of the computer system. For example, as described above with reference to Figures 7A-7D, a user can change the content sharing settings of device 100 by updating toggle switches 7010a-7010n.
[0313] The ability to enable or disable the sharing of information that identifies content items as various types of shared content, using a user interface for sharing-specific settings, improves the discoverability of content items within the relevant application without requiring further user input, while providing control over the sharing process. Performing an operation (e.g., automatically) when a set of conditions are met, without requiring further user input, improves system usability and makes the user-device interface more efficient (e.g., by assisting the user in achieving their intended results and by reducing user failures during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power consumption and improving battery life.
[0314] In some embodiments, the multiple controls include controls for activating and deactivating the provision of information that identifies content as shared content for all applications of the multiple applications (930). For example, as described above with reference to Figures 7A-7D, a toggle switch 7004 is provided to control the sharing settings for all related applications.
[0315] Providing a control that allows users to choose whether or not to allow all applications to provide information identifying content as shared content using a single user input provides additional control options that do not require navigating through complex menu hierarchies, improving the user experience and providing additional visual feedback. By providing users with improved visual feedback and additional control options without requiring further user input (e.g., automatically), the system becomes more usable, and the user-device interface becomes more efficient (e.g., by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0316] In some embodiments, the controls include controls for activating and deactivating the provision of information identifying the content as shared content to each application of a plurality of applications that can open content that has received information identifying the content as shared content (for example, toggle switches 7010a to 7010n as described with reference to Figures 7A to 7D) (932).
[0317] Providing a control that allows users to choose whether or not to allow each application to provide information identifying content as shared content, using a single user input, improves the user experience and provides additional visual feedback by offering additional control options that do not require navigating through complex menu hierarchies. By providing users with improved visual feedback and additional control options without requiring further user input (e.g., automatically), the system becomes more user-friendly and the user-device interface becomes more efficient (e.g., by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0318] In some embodiments, providing information identifying content as shared content to the first application is not permitted (934), and the content item is not automatically made available within the first application upon receiving information identifying the content item as shared content. In some embodiments, the computer system receives a request that permits the provision of information identifying the content item as shared content to one or more relevant applications, including the first application. For example, the user highlights the information identifying the content item as shared content. In some embodiments, in response to a request that permits the provision of information identifying the content item as shared content to one or more relevant applications, the computer system automatically makes the content item available from within the first application (for example, as described with reference to Figures 5E5 to 5E8). In some embodiments, upon receiving input to select information identifying the content item as shared content, controls for marking the content item as a favorite (e.g., star, pin, heart) are displayed, superimposed on or near the location where the information identifying the content item was received. In some embodiments, upon receiving input to select the controls, the content item is marked as a favorite. In some embodiments, the same user input can be used to unmark a content item as a favorite, and in response to unmarking the content as a favorite, the computer system stops displaying information that identifies the content item as being shared in the first application.
[0319] Providing controls that allow users to mark individual items as favorites enhances the user experience and provides visual feedback by offering additional control options for the shared process without requiring users to navigate through complex menu hierarchies. By providing users with improved visual feedback and offering additional control options without requiring further user input (e.g., automatically) when a set of conditions are met, the system's usability is improved, and the user-device interface becomes more efficient (e.g., by assisting users in achieving their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0320] In some embodiments, the computer system receives a request in a messaging application to open a conversation among several conversations (936), the conversation containing a message with information identifying a content item. In some embodiments, in response to the request to open a conversation, the computer system displays a user interface that includes a first messaging conversation, at least partially, and a first set of user-selectable affordances that, when selected, display a first set of controls relating to the first messaging conversation. For example, as described with reference to Figures 5G2-5G3, a tap input detected on an information icon in a messaging conversation (e.g., a user-selectable option 523) (e.g., at the top of the messaging conversation user interface) causes the user interface shown in Figure 5G3 to be displayed. In some embodiments, tapping an image associated with a messaging conversation or tapping a top description associated with a messaging application opens a detailed view of the messaging conversation. In some embodiments, in response to the selection of a user-selectable affordance, the computer system simultaneously displays a first set of controls that include controls for activating and deactivating the provision of information identifying content as shared content in a conversation (e.g., the “Hide Shared With You” toggle switch 527 in Figure 5G3). For example, the first set of controls includes the names of the members associated with the messaging conversation, and the images or avatars of the contacts who are participants in the messaging conversation.
[0321] Providing control within the messaging conversation user interface that allows users to choose whether or not to allow the provision of information identifying content as shared content for individual messaging conversations provides additional control options that do not require navigating through complex menu hierarchies, improving the user experience and providing additional visual feedback. By providing users with improved visual feedback and additional control options without requiring further user input (e.g., automatically), the system becomes more usable, and the user-device interface becomes more efficient (e.g., by helping users achieve their intended results and by reducing user failures when operating / interacting with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0322] In some embodiments, the computer system simultaneously displays a first set of conversation-specific controls and information, as described with reference to Figure 5G3 (938).
[0323] Automatically displaying additional information about a selected conversation, along with the set of controls for that conversation, improves the user experience and provides additional visual feedback by offering the user additional control options without requiring them to navigate a complex menu hierarchy. By providing the user with improved visual feedback and additional control options without requiring further user input (e.g., automatically), the system's usability is enhanced, and the user-device interface becomes more efficient (e.g., by assisting the user in achieving their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows the user to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0324] In some embodiments, information specific to the first conversation includes, as shown in Figure 5G3, the names of the contacts associated with the conversation (e.g., a list of names of participants in a messaging conversation), the avatars of the contacts associated with the conversation (e.g., avatars of the participants in the conversation), and / or the number of contacts associated with the conversation (e.g., the number of contacts that are part of the conversation or group chat) (940).
[0325] Automatically displaying additional information, such as identifying information about individuals participating in a selected conversation, provides users with real-time visual feedback on which users are included in the conversation without requiring them to navigate a complex menu hierarchy to find the information. By providing users with improved visual feedback and additional control options (e.g., automatically) without requiring further user input, the system's usability is enhanced, and the user-device interface becomes more efficient (e.g., by assisting users in achieving their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0326] In some embodiments, a first set of controls includes one or more options for communicating with one or more users participating in the conversation (e.g., an option for a voice conversation with a participant in the messaging conversation, an option for a video conversation / chat with a participant in the messaging conversation, an option for sending an email to a participant in the messaging conversation, and an option for a coexistence experience) (942) (e.g., controls 526a-526d described with reference to Figure 5G3). In some embodiments, the messaging conversation is a group chat with three or more participants. In some embodiments, some of the participants are contacts stored in the memory of an electronic device. In some embodiments, one or more options for communicating with one or more users participating in the first messaging conversation include an option for sharing a location with one or more users participating in the conversation, an option for leaving the conversation, and / or an option for editing a name or photo associated with the conversation.
[0327] Providing users with the option to reply to or otherwise contact individuals participating in a selected conversation provides users with improved visual feedback and the option to easily communicate with individuals participating in a selected conversation, without requiring them to navigate through a complex menu hierarchy to communicate with individuals. By providing users with improved visual feedback and additional control options without requiring further user input (e.g., automatically), the system's usability is improved, and the user-device interface becomes more efficient (e.g., by assisting users in achieving their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0328] In some embodiments, displaying a first set of controls and conversation-specific information simultaneously further includes displaying a plurality of shared content items and a plurality of indicators for each displayed content item of the plurality of content items (944), each of which indicators indicates that information identifying the corresponding content item among the plurality of content items has been received in the conversation. For example, as shown in Figure 5G3, shared content is displayed together with a shared indicator 528.
[0329] Displaying information to distinguish shared content items within a selected conversation, along with additional controls for interacting with the selected conversation, provides users with improved visual feedback by making it easier for them to view information about shared content items, without requiring them to navigate complex menu hierarchies to manually retrieve the information. Providing users with improved visual feedback and additional control options without requiring further user input (e.g., automatically) improves system usability and makes the user-device interface more efficient (e.g., by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0330] In some embodiments, a request to share a content item is initiated from a remote messaging application on a remote electronic device (946). For example, a content item (information identifying a first shared content item) is shared via a messaging interface that includes a message transcript between at least two users. In some embodiments, the messaging interface includes a messaging conversation that has the ability to return to the messaging conversation in response to requests from different applications (as described with reference to Figures 5E1-5E2 and 5F2, for example).
[0331] Enabling content items to be shared from a remote device via a messaging application, which is also used by the sharing and receiving devices, improves the user experience by providing users with improved visual feedback to make it easier to view shared content items within the messaging thread of the messaging application, and by enabling users of the device to share and receive shared content items using the messaging application. Providing users with improved visual feedback improves system usability and makes the user-device interface more efficient (for example, by helping users achieve their intended results and by reducing user failures when operating the system / interacting with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0332] In some embodiments, a request to share a content item is initiated by a remote user of a remote messaging application (948), and providing the first application with information identifying the content item as shared content is permitted (for example, according to the configuration interface described above). In some embodiments, upon receiving information identifying the content item as shared content, the computer system does not make the content item available within the first application, according to the determination that the remote user is not a contact of any of the multiple contacts stored in memory. In some embodiments, the computer system does not make the content item available within the first application, according to the determination that the remote user is a contact of any of the multiple contacts stored in memory.
[0333] Allowing content items to be shared from remote devices associated with individuals recognized by the receiving device as saved contacts, without automatically enabling shared content from individuals not recognized by the device as saved contacts, improves the user experience by providing users with improved visual feedback, while only allowing shared content from saved content to be automatically displayed. Providing users with improved visual feedback improves system usability and makes the user-device interface more efficient (for example, by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0334] In some embodiments, making a first shared content item available automatically (e.g., without additional user input) within a first application does not require the messaging application to respond to a user request to open the first shared content item (950). For example, a user can view and open a first shared content item within a messaging application regardless of the content items available within the first application. For example, a first shared content item is made available within the first application even if the user has not opened the first shared content (e.g., a conversation thread) in the messaging application.
[0335] Providing users with the option to view shared content items within a messaging application without affecting how shared content items are automatically displayed in the primary application provides users with improved visual feedback to ensure that shared content items are automatically available within the primary application, regardless of whether the user viewed and / or opened them in the messaging application to which they were shared. Providing users with improved visual feedback enhances system usability and makes the user-device interface more efficient (for example, by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system), which in turn allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0336] In some embodiments, the first shared content item is automatically made available for later viewing within the first application, in accordance with a determination that it is permitted to provide the first application with information identifying the first shared content item as shared content (952). For example, a representation of the shared content item is displayed in a dedicated section for shared items within the first application, and the user may, at a later time, select the shared content item to view, play, or otherwise consume the shared content item. For example, Figures 5I1 to 5I10 provide several examples of shared content items displayed within different media applications (for example, in accordance with a determination that sharing functionality is enabled for the applications shown in Figures 5I1 to 5I10).
[0337] Automatically displaying shared content within the relevant application based on its type, after it has been received by a separate messaging application, provides users with improved visual feedback by making shared content items accessible within the relevant application without requiring user input to discover or view the shared content items from within the relevant application. Performing an operation (e.g., automatically) when a set of conditions are met, without requiring further user input, improves system usability and makes the user-device interface more efficient (e.g., by assisting the user in achieving their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power consumption and improving battery life.
[0338] In some embodiments, the receiving application is a messaging application (954).
[0339] It should be understood that the specific order of operations described in Figures 9A to 9E is merely an example, and is not intended to indicate that the described order is the only order in which the operations can be performed. Those skilled in the art will recognize various methods for reordering the operations described herein. Furthermore, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., methods 800, 1000, and 1100) are also applicable to methods similar to method 900 described above with respect to Figures 9A to 9E. For example, the user input described above with reference to method 900 optionally possesses one or more characteristics of the user inputs described herein with reference to other methods described herein (e.g., methods 800, 1000, and 1100). For brevity, those details will not be repeated here.
[0340] Figures 10A to 10E are flowcharts illustrating Method 1000 for displaying information about how content items have been shared, according to several embodiments. Method 1000 is performed in an electronic device (e.g., device 300 in Figure 3, or portable multifunction device 100 in Figure 1A) that communicates with a display generating component (e.g., optionally a touch-sensitive display device, projector, head-mounted display, head-up display, etc.) and one or more input devices (e.g., including one or more sensors for detecting the intensity of contact with a touch-sensitive surface). In some embodiments, the display generating component is a touchscreen display, and the touch-sensitive surface is on or incorporated into the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations of Method 1000 are optionally combined, and / or the order of some operations is optionally changed.
[0341] As described herein, Method 1000 provides a selectable attribute affordance in a first (e.g., media) application that provides a link to a location in a second (e.g., messaging) application where information identifying a content item was first shared. Selecting the selectable attribute affordance displays the second (e.g., messaging) application at the location where information identifying a content item was first shared, improving the discoverability of the shared content and the message in which the content was shared, improving communication with the contact who shared the content, reducing the number of inputs required to discover the shared content and the message in which the content was shared, and reducing the number of inputs required to reply to the contact who shared the content. Furthermore, the selectable attribute affordance provides visual feedback that the content has been shared, who shared the content, and how many contacts (if more than two) shared the content. By providing the user with improved visual feedback and reducing the number of inputs required to perform an action, the usability of the device is enhanced, and the user-device interface becomes more efficient (e.g., by helping the user provide appropriate input when operating / interacting with the device and reducing user errors). This also reduces power consumption and improves the device's battery life by enabling users to use the device more quickly and efficiently.
[0342] While displaying a view of a first application from among multiple applications stored in memory (e.g., a TV app, a music app, a book app, a podcast app, a news app, etc.) (e.g., the user interface of an application including a set of tabs such as a library user interface, or a single tab within a set of tabs (e.g., "Listen Now" may be one tab in the main user interface of the "Apple Music" application, and "Radio" may be another tab)) (1001), the electronic device displays a representation of a first shared content item (1002) (e.g., a representation or identifying information of a content item is added to a conversation in a messaging app) and a selectable attribute affordance (1004) (e.g., a user interface object) that provides a link to the location in a second application where the information identifying the first shared content item was first shared. For example, as illustrated with reference to Figures 5A6-5A7, while the music application is displayed, a representation of song title-A is displayed, and a selectable capsule 5056a (e.g., a selectable attribute affordance), when selected, causes the device to display the message thread in which song title-A was shared. For example, when a selectable attribute affordance is selected, a process is initiated to display the user interface of a second application that has received information identifying the content item. In some embodiments, the first shared content item is a single media file. In some embodiments, the first shared content item is a collection of media items (e.g., a photo album, a song album, a series of episodes, a podcast channel with a series of podcasts, etc.).
[0343] In some embodiments, before displaying the first shared content item in the first application, the electronic device acquires information identifying the first shared content item in the first application (1006), based on the determination that information identifying the first shared content item has been first received from one or more contacts stored in the electronic device's memory. For example, before displaying song title-A in a music application, device 100 determines that song title-A has been shared by a contact stored in device 100 (for example, "Alex" is a contact, as described with reference to Figures 5A6-5A7).
[0344] Determining whether shared content was shared by a contact (rather than by a non-contact user) before automatically retrieving information identifying the shared content within the first (e.g., media) application improves the relevance of the shared content displayed within the relevant application (e.g., the application from which the content can be opened) and reduces the number of inputs required to perform the action. Reducing the number of inputs required to interact with shared content improves the usability of the device, makes the user-device interface more efficient (e.g., by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and, in addition, reduces power consumption and improves battery life by allowing users to use the device more quickly and efficiently.
[0345] In some embodiments, the selectable attribute affordance further indicates the name of the contact who sent the information identifying the first shared content item (1008). For example, as shown in Figure 5A6, the selectable capsule 5056a displays "Alex," indicating that the contact who shared the first shared content item is "Alex."
[0346] Displaying the names of contacts who have shared content items provides additional visual feedback. By providing users with improved visual feedback, the system becomes more user-friendly, and the user-device interface becomes more efficient (for example, by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0347] In some embodiments, the selectable attribute affordance further indicates an avatar representing the contact who sent the information identifying the first shared content item (1010). For example, as shown in Figure 5A6, the selectable capsule 5056a includes an avatar of Alex. In some embodiments, device 100 simultaneously displays text identifying the contact (e.g., "Alex") and / or an avatar representing Alex. In some embodiments, it will be understood that device 100 displays an alternative and / or additional representation of the contact who shared the first content item.
[0348] Displaying avatars of contacts who have shared content items provides additional visual feedback. By providing users with improved visual feedback, the system becomes more user-friendly, and the user-device interface becomes more efficient (for example, by helping users achieve their intended results and by reducing user errors during system operation / interaction with the system). This, in turn, allows users to use the device more quickly and efficiently, reducing device power usage and improving battery life.
[0349] In some embodiments, the electronic device receives input in a first application to select a selectable attribute affordance (e.g., tapping or long-pressing the “from” capsule) (1012), and in response to the input, displays information identifying the first shared content item at the location in a second application where the information identifying the first shared content item was first shared (e.g., the location in a second application where a request to share the content item was received). For example, as described with reference to Figures 5A6 and 5A7, in some embodiments, in response to the user selecting a selectable capsule 5056a, the electronic device displays the message thread in which song title-A was shared (e.g., the message thread with Alex). In some embodiments, the device displays the most recent portion of the message thread (e.g., instead of displaying a portion of the message thread in which the media item was shared).
[0350] Selecting an available attribute affordance displays information identifying the shared content item in the location within the second place where the information was first shared (e.g., a messaging application), improving the discoverability of shared content, improving the discoverability of the conversation in which the content was shared, improving communication with the contact who shared the content, reducing the number of inputs required to discover shared content and the message / conversation in which the content was shared, and reducing the number of inputs required to reply to the contact who shared the content. Reducing the number of inputs required to perform actions improves device usability and makes the user-device interface more efficient (e.g., by helping users make appropriate inputs when operating / interacting with the device and reducing user errors), and in addition, reduces device power usage and improves battery life by allowing users to use the device more quickly and efficiently.
[0351] In some embodiments, the second application is a messaging application that includes multiple messaging conversations, and the information identifying the content item was received in the first messaging conversation among the multiple messaging conversations. In some embodiments, the electronic device, upon receiving input to select a selectable attribute affordance, displays a portion of the message thread of the first messaging conversation, including previously received messages in the first messaging conversation in which the information identifying the first shared content item was first shared (1014) (for example, the message thread displays both sides of the messaging conversation in chronological order on one screen, with messages from contacts displayed on one side and messages from users of the electronic device / computer system displayed on the other side). For example, when a selectable attribute affordance is tapped, the method displays the messaging conversation and automatically scrolls to the message containing the information identifying the content item as shared content. For example, as described with reference to Figures 5A6 and 5A8, in some embodiments, depending on the user's selection of a selectable capsule 5056a, the electronic device displays message 5001d (e.g., a message sharing song title-A) as the bottom message in the message thread of the messaging application shown in Figure 5A8.
[0352] Selecting an available attribute affordance to display previously received messages containing information identifying the shared content item improves the discoverability of the shared content, improves the discoverability of previously received messages on which the content was shared, improves communication with the contact who shared the content, reduces the number of inputs required to discover the shared content and the messages / conversations on which the content was shared, and reduces the number of inputs required to reply to the contact who shared the content. Reducing the number of inputs required to perform an action improves the usa...
Claims
1. It is a method, In an electronic device including a display generation component, one or more input devices, and memory, While displaying the view of the media application among the multiple applications stored in the memory, within the view of the media application, A selectable representation of the first shared media content item, which is triggered by the media application when selected, Information identifying the first shared media content item provides a selectable attribute affordance that provides a link to a message in a communication application among the multiple applications that was previously shared, The second set of selectable expressions for shared media content items, The information identifying the second shared media content item is displayed simultaneously with a second selectable attribute affordance that provides a link to a message in the communication application that was previously shared, Receiving input within the media application to select the aforementioned selectable attribute affordances, In response to the input that selects the aforementioned selectable attribute affordance, A method comprising displaying the message in which the information identifying the first shared media content item was previously shared.
2. The method according to claim 1, comprising obtaining the information identifying the first shared media content item for display in the media application, based on a determination that the information identifying the first shared media content item has been received from one or more contacts stored in the memory of the electronic device, before displaying the first shared media content item in the view of the media application.
3. The communication application includes a plurality of messaging conversations, and the information identifying the first shared media content item is received in the first messaging conversation among the plurality of messaging conversations, and the method is In response to the input that selects the aforementioned selectable attribute affordance, The method according to claim 1, comprising displaying a portion of a message thread of a first messaging conversation, which includes a plurality of previously received messages, the information identifying the first shared media content item includes the previously shared message.
4. The method according to claim 1, wherein the message is displayed simultaneously with a field for replying to the message.
5. The message is displayed according to the determination that the input for selecting the selectable attribute affordance is an input of a first type, and the method is The method according to claim 1, comprising displaying a set of controls for performing an action related to the first shared media content item, based on the determination that the input for selecting the selectable attribute affordance is a second type of input different from the first type of input.
6. The method according to claim 5, wherein the set of controls includes a control for responding to the message.
7. The method of claim 5, wherein the set of controls includes controls for removing the selectable representation and the selectable attribute affordance of the first shared media content item from display in the view of the media application.
8. The method according to claim 5, wherein the set of controls includes controls for listening to or viewing the first shared media content item together with one or more other users.
9. The message is received in a first messaging conversation among a plurality of messaging conversations, and the method is In response to the input that selects the aforementioned selectable attribute affordance, The method according to claim 1, comprising displaying a user interface for the communication application including the first messaging conversation, which includes the message and message input fields.
10. The view in the media application is a default view that includes a distinguishable section for displaying shared content items, and the method is The method according to claim 1, comprising displaying the information identifying the first shared media content item in the communication application, and receiving a single input to return to the default view of the media application which includes a distinguishable section for the shared content item.
11. Within the view of the media application, Multiple representations of multiple shared media content items, including the first shared media content item, The method according to claim 1, comprising displaying a plurality of selectable attribute affordances that provide links to individual messages in the communication application that have been previously shared, each of which individual information identifies the plurality of shared media content items includes the selectable attribute affordances that provide links to individual messages in the communication application that have been previously shared.
12. Receiving an input for selecting the second selectable attribute affordance, The method according to claim 1, comprising displaying the message that the information identifying the second shared media content item has been previously shared, in response to the input selecting the second selectable attribute affordance.
13. The information identifying the second shared media content item is received in an individual messaging conversation among a plurality of messaging conversations, and the method is: The method of claim 12, comprising displaying the individual messaging conversation, which includes the message previously shared, in response to the input selecting the second selectable attribute affordance, the information identifying the second shared media content item.
14. The view in the media application is a first view, and the method is While displaying the second view of the aforementioned media application, In accordance with the determination that the first shared media content item is a single media file, a first attribute affordance is displayed near the representation of the single media file, wherein the first attribute affordance indicates that the single media file was shared by a first set of one or more contacts. The method according to claim 1, further comprising: displaying a second attribute affordance near a representation of the collection of media, in accordance with the determination that the first shared media content item is a collection of media, wherein the second attribute affordance indicates that the collection of media was shared by a second set of one or more contacts.
15. The first attribute affordance includes a first set of one or more avatars corresponding to the first set of one or more contacts, The method according to claim 14, wherein the second attribute affordance includes a second set of one or more avatars corresponding to the second set of one or more contacts.
16. The first attribute affordance is selectable and provides a link to one or more messages in the communication application that receive information identifying the single media file as shared content. The method according to claim 14, wherein the second attribute affordance is selectable and provides a link to one or more messages in the communication application that receive information identifying the set of media as shared.
17. The aforementioned selectable attribute affordances are capsule user interface objects, The method according to claim 1, wherein the selectable attribute affordance is displayed near the selectable representation of the first shared media content item.
18. The memory stores multiple contacts of the user of the electronic device. The electronic device has received a plurality of messages in which the information identifying the first shared media content item is shared. The method according to claim 1, wherein the plurality of messages are sent from a first set of contacts among the plurality of contacts.
19. The method of claim 18, wherein the selectable attribute affordance further indicates a distinct number of contacts in the first set of contacts among the plurality of contacts that sent the plurality of messages to the electronic device.
20. In response to receiving the input for selecting the selectable attribute affordance, The method according to claim 18, comprising displaying a plurality of selectable contact affordances, each selectable contact affordance being associated with one of the first set of contacts among the plurality of contacts that sent the plurality of messages to the electronic device.
21. Receiving a second input for selecting a contact affordance from among the plurality of selectable contact affordances, wherein the selected contact affordance is associated with a specific contact. The method of claim 20, comprising: displaying a view of the communication application including a message which includes an individual message from among a plurality of messages sent by the specific contact, in response to the second input, wherein the individual message includes the information which identifies the first shared media content item.
22. The method according to claim 21, wherein the contact affordance includes an option to send a message to the specific contact and an option to view or listen to the first shared media content item with the specific contact.
23. Among the plurality of contacts stored in the memory of the electronic device, the contact that sent the message in which the information identifying the first shared media content item was previously shared, The method according to claim 1, wherein the selectable attribute affordance further indicates the name of the contact.
24. Among the plurality of contacts stored in the memory of the electronic device, the contact that sent the message previously shared the information identifying the first shared media content item, The method according to claim 1, wherein the selectable attribute affordance further indicates an avatar representing the contact.
25. The method according to claim 1, comprising displaying an indicator of the number of shared media content items while displaying the view of the media application.
26. A computer system, Display generation components and, One or more input devices, One or more processors, A memory storing one or more programs is provided, and the one or more programs are configured to be executed by the one or more processors, While displaying the view of the media application among the multiple applications stored in the memory, within the view of the media application, When selected, the representation of the first shared media content item triggered by the media application, Information identifying the first shared media content item provides a selectable attribute affordance that provides a link to a message in a communication application among the multiple applications that was previously shared, The second set of selectable expressions for shared media content items, The information identifying the second shared media content item is displayed simultaneously with a second selectable attribute affordance that provides a link to a message in the communication application that was previously shared, Receiving input within the media application to select the aforementioned selectable attribute affordances, In response to the input that selects the aforementioned selectable attribute affordance, A computer system including instructions for displaying the message that the information identifying the first shared media content item has been previously shared.
27. The system according to claim 26, wherein one or more programs include instructions for performing the method described in any one of claims 2 to 25.
28. When executed by a computer system having a display generation component and one or more input devices, the computer system, While displaying a view of the media application among multiple applications stored in the memory of the computer system, within the view of the media application, When selected, the representation of the first shared media content item triggered by the media application, Information identifying the first shared media content item provides a selectable attribute affordance that provides a link to a message in a communication application among the multiple applications that was previously shared, The second set of selectable expressions for shared media content items, The information identifying the second shared media content item is displayed simultaneously with a second selectable attribute affordance that provides a link to a message in the communication application that was previously shared, Receiving input within the media application to select the aforementioned selectable attribute affordances, In response to the input that selects the aforementioned selectable attribute affordance, A program including an instruction to cause the program to display the message that the information identifying the first shared media content item has been previously shared.
29. The program according to claim 28, further comprising, when executed by the computer system, an instruction causing the computer system to perform the method according to any one of claims 2 to 25.
Citation Information
Patent Citations
Technologies for sharing and remixing media via messaging systems
JP2018514025A
Private messaging via content provider web sites and applications
US20150172250A1
Device, Method, and Graphical User Interface for Sharing Content from a Respective Application
US20190073102A1
Preloading of application on a user device based on content received by the user device
WO2020096776A1