Integration of avatars with multiple applications

Abstract

The invention relates to integration of an avatar with multiple applications. The present disclosure generally relates to user interfaces for displaying and using avatars. In some embodiments, the avatar is used to generate a sticker for transmission in a content creation user interface. In some implementations, an avatar is used to generate a representation of a connectable user in a connectable user editing user interface. In some embodiments, a user interface may be used to create and edit an avatar. In some embodiments, a user interface may be used to display an avatar in response to a detected change in a user's facial gesture. In some embodiments, contact information is transmitted or received.

Description

[0001] This application is a divisional application of the invention patent application filed on March 31, 2020, with application number 202080001137.2 and title "Integration of Avatar with Multiple Applications".

[0002] Cross-references to related applications

[0003] This application claims priority to the following patent applications: U.S. Patent Application No. 62 / 843,967, filed May 6, 2019, entitled "AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS"; U.S. Patent Application No. 62 / 855,891, filed May 31, 2019, entitled "AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS"; Danish Patent Application No. PA 2019 70530, filed August 27, 2019, entitled "AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS"; Danish Patent Application No. PA 201970531, filed August 27, 2019, entitled "AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS"; and [other patent applications filed September 25, 2019, entitled "AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS"]. The contents of the following U.S. patent applications are incorporated herein by reference in their entirety: U.S. Patent Application No. 16 / 582,500 entitled “MULTIPLE APPLICATIONS”; U.S. Patent Application No. 16 / 582,570 entitled “AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS”, filed on September 25, 2019; and U.S. Patent Application No. 16 / 583,706 entitled “AVATAR INTEGRATION WITH MULTIPLE APPLICATIONS”, filed on September 26, 2019. Technical Field

[0004] This disclosure relates generally to computer user interfaces, and more specifically to techniques for displaying avatars in various application user interfaces. Background Technology

[0005] Multimedia content such as emojis, stickers, and avatars are sometimes used in various application user interfaces. Emojis, stickers, and avatars represent a wide variety of people, objects, actions, and / or other things. Personal contact information such as their name and photo is used in messaging applications. Summary of the Invention

[0006] However, some technologies used to display and use avatars in various application user interfaces on electronic devices are often cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may involve multiple keystrokes or button presses. Existing technologies require more time than necessary, resulting in wasted user time and device power. This latter consideration is particularly important in battery-powered devices.

[0007] Therefore, this technology provides electronic devices with a faster and more efficient method and interface for displaying avatars in various application user interfaces. Such methods and interfaces optionally complement or replace other methods for displaying avatars in various application user interfaces. These methods and interfaces reduce the cognitive burden on users and result in a more efficient human-computer interface. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charging.

[0008] This document describes an example method. One example method includes, at an electronic device having a display device and an input device: receiving a request to display a sticker user interface via one or more input devices; and, in response to receiving the request to display the sticker user interface, displaying the sticker user interface via the display device, the sticker user interface including representations of multiple sets of stickers based on user-created avatars, including: displaying a first multiple set of sticker representations based on determining that the user has created a first set of two or more user-created avatars including a first avatar and a second avatar, wherein the first multiple set of sticker representations includes representations of a set of stickers based on the first avatar and representations of a set of stickers based on the second avatar; and displaying a second multiple set of sticker representations different from the first multiple set of sticker representations based on determining that the user has created a second set of two or more user-created avatars including a third avatar not included in the first set of two or more user-created avatars, wherein the second multiple set of sticker representations includes representations of a set of stickers based on the third avatar not included in the first multiple set of sticker representations.

[0009] This document describes an example non-transitory computer-readable storage medium. An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and an input device, the one or more programs including instructions for: receiving a request to display a sticker user interface via one or more input devices; and, in response to receiving the request to display the sticker user interface, displaying the sticker user interface via the display device, the sticker user interface including representations of multiple sets of stickers based on user-created avatars, including: displaying a first multiple set of sticker representations based on determining that the user has created a first group of two or more user-created avatars including a first avatar and a second avatar, wherein the first multiple set of sticker representations includes representations of a set of stickers based on the first avatar and a set of stickers based on the second avatar; and displaying a second multiple set of sticker representations different from the first multiple set of sticker representations based on determining that the user has created a second group of two or more user-created avatars including a third avatar not included in the first group of two or more user-created avatars, wherein the second multiple set of sticker representations includes representations of a set of stickers based on the third avatar not included in the first multiple set of sticker representations.

[0010] This document describes an example transient computer-readable storage medium. An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and an input device, the one or more programs including instructions for: receiving a request to display a sticker user interface via one or more input devices; and, in response to receiving the request to display the sticker user interface, displaying the sticker user interface via the display device, the sticker user interface including representations of multiple sets of stickers based on user-created avatars, including: displaying a first multiple set of sticker representations based on determining that the user has created a first group of two or more user-created avatars including a first avatar and a second avatar, wherein the first multiple set of sticker representations includes representations of a set of stickers based on the first avatar and a set of stickers based on the second avatar; and displaying a second multiple set of sticker representations different from the first multiple set of sticker representations based on determining that the user has created a second group of two or more user-created avatars including a third avatar not included in the first group of two or more user-created avatars, wherein the second multiple set of sticker representations includes representations of a set of stickers based on the third avatar not included in the first multiple set of sticker representations.

[0011] This document describes an example electronic device. An example electronic device includes a display device; an input device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: receiving a request to display a sticker user interface via the one or more input devices; and, in response to receiving the request to display the sticker user interface, displaying the sticker user interface via the display device, the sticker user interface including representations of multiple sets of stickers based on user-created avatars, including: displaying a first multiple set of sticker representations based on determining that the user has created a first group of two or more user-created avatars including a first avatar and a second avatar, wherein the first multiple set of sticker representations includes a representation of a set of stickers based on the first avatar and a representation of a set of stickers based on the second avatar; and displaying a second multiple set of sticker representations different from the first multiple set of sticker representations based on determining that the user has created a second group of two or more user-created avatars including a third avatar not included in the first group of two or more user-created avatars, wherein the second multiple set of sticker representations includes a representation of a set of stickers based on the third avatar not included in the first multiple set of sticker representations.

[0012] This document describes an example electronic device. An example electronic device includes a display device; an input device; means for receiving a request to display a sticker user interface via one or more input devices; and means for displaying a sticker user interface via the display device in response to receiving the request to display the sticker user interface, the sticker user interface including representations of multiple sets of stickers based on user-created avatars, including: displaying a first multiple set of sticker representations based on determining that the user has created a first group of two or more user-created avatars including a first avatar and a second avatar, wherein the first multiple set of sticker representations includes representations of a set of stickers based on the first avatar and representations of a set of stickers based on the second avatar; and displaying a second multiple set of sticker representations different from the first multiple set of sticker representations based on determining that the user has created a second group of two or more user-created avatars including a third avatar not included in the first group of two or more user-created avatars, wherein the second multiple set of sticker representations includes representations of a set of stickers based on the third avatar not included in the first multiple set of sticker representations.

[0013] This document describes an example method. An example method includes, at an electronic device having a display device and one or more input devices: displaying a contactable user editable user interface via the display device, the contactable user editable user interface including one or more representation options for a contactable user, including an avatar representation option; detecting a selection of the avatar representation option via the one or more input devices; in response to detecting a selection of the avatar representation option, initiating a process for selecting an avatar to be used as a representation of the contactable user in the contactable user interface; as part of the process for selecting the avatar to be used as a representation of the contactable user in the contactable user interface, receiving a sequence of one or more inputs via the one or more input devices corresponding to the selection of a simulated 3D avatar; and in response to the selection of the simulated 3D avatar, displaying a posing user interface via the display device, the posing user interface including one or more controls for selecting a pose of the simulated 3D avatar from a plurality of different poses.

[0014] This document describes an example non-transitory computer-readable storage medium. An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more input devices, the one or more programs including instructions for: displaying a contactable user-editable user interface via the display device, the contactable user-editable user interface including one or more representation options for a contactable user, including an avatar representation option; detecting a selection of the avatar representation option via the one or more input devices; in response to detecting a selection of the avatar representation option, initiating a process for selecting an avatar to be used as a representation of the contactable user in the contactable user interface; as part of the process for selecting the avatar to be used as a representation of the contactable user in the contactable user interface, receiving a sequence of one or more inputs via the one or more input devices corresponding to the selection of a simulated 3D avatar; and in response to the selection of the simulated 3D avatar, displaying a posing user interface via the display device, the posing user interface including one or more controls for selecting a pose of the simulated 3D avatar from a plurality of different poses.

[0015] This document describes an example transient computer-readable storage medium. An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more input devices, the one or more programs including instructions for: displaying a contactable user-editable user interface via the display device, the contactable user-editable user interface including one or more representation options for a contactable user, including an avatar representation option; detecting a selection of the avatar representation option via the one or more input devices; in response to detecting a selection of the avatar representation option, initiating a process for selecting an avatar to be used as a representation of the contactable user in the contactable user interface; as part of the process for selecting the avatar to be used as a representation of the contactable user in the contactable user interface, receiving a sequence of one or more inputs via the one or more input devices corresponding to the selection of a simulated 3D avatar; and in response to the selection of the simulated 3D avatar, displaying a posing user interface via the display device, the posing user interface including one or more controls for selecting a pose of the simulated 3D avatar from a plurality of different poses.

[0016] This document describes an example electronic device. An example electronic device includes a display device; one or more input devices; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a contactable user-editable interface via the display device, the contactable user-editable interface including one or more representation options for a contactable user, including an avatar representation option; detecting a selection of the avatar representation option via the one or more input devices; in response to detecting a selection of the avatar representation option, initiating a process for selecting an avatar to be used as a representation of the contactable user in the contactable user interface; as part of the process for selecting the avatar to be used as a representation of the contactable user in the contactable user interface, receiving a sequence of one or more inputs via the one or more input devices corresponding to the selection of a simulated 3D avatar; and in response to the selection of the simulated 3D avatar, displaying a posing user interface via the display device, the posing user interface including one or more controls for selecting a pose of the simulated 3D avatar from a plurality of different poses.

[0017] This document describes an example electronic device. An example electronic device includes a display device; one or more input devices; means for displaying a user-editable interface via the display device, the user-editable interface including one or more representation options for the user, including an avatar representation option; means for detecting a selection of the avatar representation option via the one or more input devices; means for initiating a process for selecting an avatar to be used as a representation of the user-editable interface in response to detecting a selection of the avatar representation option; means for receiving, via the one or more input devices, a sequence of one or more inputs corresponding to the selection of a simulated 3D avatar as part of the process for selecting an avatar to be used as a representation of the user-editable interface; and means for displaying, via the display device, a posing user interface including one or more controls for selecting a pose of the simulated 3D avatar from a plurality of different poses in response to the selection of the simulated 3D avatar.

[0018] This document describes an example method. An example method includes, at an electronic device having a display device and an input device, displaying an avatar editing user interface via the display device, the avatar editing user interface comprising: an avatar including a first feature having a first color pattern generated using a first set of colors including a first color in a first region of the first color pattern; a set of color options for the first feature; and a plurality of color pattern options for the first feature including second color pattern options different from the first color pattern; and, when the first feature is displayed having the first color pattern generated using the first set of colors including the first color in the first region of the first color pattern, detecting via the input device a color corresponding to the second color in the set of color options. The selection of a color option; in response to detecting the selection: changing the appearance of one or more color pattern options having a first portion corresponding to the group of color options, wherein changing the appearance includes changing a portion of a second color pattern option from the corresponding color to the second color; and maintaining the display of an avatar including a first feature having the first color pattern; detecting the selection of the corresponding color pattern option having the changed appearance; and in response to detecting the selection of the corresponding color pattern option and when a second color is selected for the group of color options, changing the appearance of the first feature of the avatar to have an appearance generated based on the corresponding color pattern option, wherein the second color is applied to a portion of the corresponding color pattern option.

[0019] This document describes an example non-transitory computer-readable storage medium. An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and an input device, the one or more programs including instructions for: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar including a first feature having a first color pattern generated using a first set of colors including a first color in a first region of the first color pattern; a set of color options for the first feature; and a plurality of color pattern options for the first feature including a second color pattern option different from the first color pattern; when the first feature is displayed having the first color pattern generated using the first set of colors including the first color in the first region of the first color pattern, via... The input device detects the selection of a color option corresponding to a second color in the set of color options; in response to detecting the selection: changes the appearance of one or more color pattern options having a first portion corresponding to the set of color options, wherein changing the appearance includes changing a portion of the second color pattern option from the corresponding color to the second color; and maintains the display of an avatar including a first feature having the first color pattern; detects the selection of a corresponding color pattern option having the changed appearance in the set of color pattern options; and in response to detecting the selection of the corresponding color pattern option and when a second color is selected for the set of color options, changes the appearance of the first feature of the avatar to have an appearance generated based on the corresponding color pattern option, wherein the second color is applied to a portion of the corresponding color pattern option.

[0020] This document describes an example transient computer-readable storage medium. An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and an input device, the one or more programs including instructions for: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar including a first feature having a first color pattern generated using a first set of colors including a first color in a first region of the first color pattern; a set of color options for the first feature; and a plurality of color pattern options for the first feature including a second color pattern option different from the first color pattern; and, when the first feature is displayed having the first color pattern generated using the first set of colors including the first color in the first region of the first color pattern, via... The input device detects the selection of a color option corresponding to a second color in the set of color options; in response to detecting the selection: changes the appearance of one or more color pattern options having a first portion corresponding to the set of color options, wherein changing the appearance includes changing a portion of the second color pattern option from the corresponding color to the second color; and maintains the display of an avatar including a first feature having the first color pattern; detects the selection of a corresponding color pattern option having the changed appearance in the set of color pattern options; and in response to detecting the selection of the corresponding color pattern option and when a second color is selected for the set of color options, changes the appearance of the first feature of the avatar to have an appearance generated based on the corresponding color pattern option, wherein the second color is applied to a portion of the corresponding color pattern option.

[0021] This document describes an example electronic device. An example electronic device includes a display device; an input device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar including a first feature having a first color pattern generated using a first set of colors including a first color in a first region of the first color pattern; a set of color options for the first feature; and a plurality of color pattern options for the first feature including a second color pattern option different from the first color pattern; and the first feature being displayed having the first color pattern generated using the first set of colors including the first color in the first region of the first color pattern. When the input device detects the selection of a color option corresponding to a second color in the set of color options; in response to detecting the selection: the appearance of one or more color pattern options having a first portion corresponding to the set of color options is changed, wherein changing the appearance includes changing a portion of the second color pattern option from the corresponding color to the second color; and the display of an avatar including a first feature having the first color pattern is maintained; the selection of a corresponding color pattern option having the changed appearance in the set of color pattern options is detected; and in response to detecting the selection of a corresponding color pattern option and when a second color is selected for the set of color options, the appearance of the first feature of the avatar is changed to have an appearance generated based on the corresponding color pattern option, wherein the second color is applied to a portion of the corresponding color pattern option.

[0022] This document describes an example electronic device. An example electronic device includes a display device; an input device; and means for displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar including a first feature, the first feature having a first color pattern generated using a first set of colors including a first color in a first region of the first color pattern; a set of color options for the first feature; and a plurality of color pattern options for the first feature including second color pattern options different from the first color pattern; and means for detecting, via the input device, a selection of a color option corresponding to a second color in the set of color options when the first feature is displayed having the first color pattern generated using a first set of colors including a first color in a first region of the first color pattern; and using... The apparatus for: in response to detecting the selection, changing the appearance of one or more color pattern options having a first portion corresponding to the group of color options, wherein changing the appearance includes changing a portion of a second color pattern option from a corresponding color to the second color; and maintaining the display of an avatar including a first feature having the first color pattern; means for detecting the selection of a corresponding color pattern option having a changed appearance; and means for: in response to detecting the selection of a corresponding color pattern option and when a second color is selected for the group of color options, changing the appearance of the first feature of the avatar to have an appearance generated based on the corresponding color pattern option, wherein the second color is applied to a portion of the corresponding color pattern option.

[0023] This document describes an example method. An example method includes, at an electronic device having a display device and an input device: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar, a corresponding avatar feature having a first pose; and an avatar option selection area including a plurality of avatar feature options corresponding to a set of candidate values ​​for characteristics of the avatar feature and including a plurality of avatar feature options having an appearance based on the avatar; detecting via the input device a request to display options for editing the corresponding avatar feature; and, in response to detecting the request, updating the avatar option selection area to display avatar feature options corresponding to a set of candidate values ​​for characteristics of the corresponding avatar feature, including simultaneously displaying: a representation of the first option for the corresponding avatar feature, wherein the corresponding avatar feature has a second pose; and a representation of the second option for the corresponding avatar feature, wherein the corresponding avatar feature has a third pose different from the second pose.

[0024] This document describes an example non-transitory computer-readable storage medium. An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and an input device, the one or more programs including instructions for: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar, a corresponding avatar feature having a first pose; and an avatar option selection area including a plurality of avatar feature options corresponding to a set of candidate values ​​for characteristics of the avatar feature and including a plurality of avatar feature options having an appearance based on the avatar; detecting a request via the input device to display options for editing the corresponding avatar feature; and in response to detecting the request, updating the avatar option selection area to display avatar feature options corresponding to a set of candidate values ​​for characteristics of the corresponding avatar feature, including simultaneously displaying: a representation of the first option for the corresponding avatar feature, wherein the corresponding avatar feature has a second pose; and a representation of the second option for the corresponding avatar feature, wherein the corresponding avatar feature has a third pose different from the second pose.

[0025] This document describes an example transient computer-readable storage medium. An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and an input device, the one or more programs including instructions for: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar, a corresponding avatar feature having a first pose; and an avatar option selection area including a plurality of avatar feature options corresponding to a set of candidate values ​​for characteristics of the avatar feature and including a plurality of avatar feature options having an appearance based on the avatar; detecting a request via the input device to display options for editing the corresponding avatar feature; and in response to detecting the request, updating the avatar option selection area to display avatar feature options corresponding to a set of candidate values ​​for characteristics of the corresponding avatar feature, including simultaneously displaying: a representation of the first option for the corresponding avatar feature, wherein the corresponding avatar feature has a second pose; and a representation of the second option for the corresponding avatar feature, wherein the corresponding avatar feature has a third pose different from the second pose.

[0026] This document describes an example electronic device. An example electronic device includes a display device; an input device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar, a corresponding avatar feature having a first pose; and an avatar option selection area including a plurality of avatar feature options corresponding to a set of candidate values ​​for characteristics of the avatar feature and including a plurality of avatar feature options having an appearance based on the avatar; detecting a request via the input device to display options for editing the corresponding avatar feature; and in response to detecting the request, updating the avatar option selection area to display avatar feature options corresponding to a set of candidate values ​​for characteristics of the corresponding avatar feature, including simultaneously displaying: a representation of the first option for the corresponding avatar feature, wherein the corresponding avatar feature has a second pose; and a representation of the second option for the corresponding avatar feature, wherein the corresponding avatar feature has a third pose different from the second pose.

[0027] This document describes an example electronic device. An example electronic device includes a display device; an input device; means for displaying an avatar editing user interface via the display device, the avatar editing user interface including: an avatar, a corresponding avatar feature having a first pose; and an avatar option selection area including a plurality of avatar feature options corresponding to a set of candidate values ​​for characteristics of the avatar feature and including a plurality of avatar feature options having an appearance based on the avatar; means for detecting a request via the input device to display options for editing the corresponding avatar feature; and means for: in response to detecting the request, updating the avatar option selection area to display avatar feature options corresponding to a set of candidate values ​​for characteristics of the corresponding avatar feature, including simultaneously displaying: a representation of the first option for the corresponding avatar feature, wherein the corresponding avatar feature has a second pose; and a representation of the second option for the corresponding avatar feature, wherein the corresponding avatar feature has a third pose different from the second pose.

[0028] This document describes an example method. An example method includes, at an electronic device having a display device and one or more cameras: displaying via the display device a virtual avatar having one or more avatar features, the one or more avatar features changing appearance in response to detected changes in facial pose in the field of view of the one or more cameras, including a first avatar feature having a first appearance, the first appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; when a face including one or more detected facial features is detected in the field of view of the one or more cameras, detecting movement of one or more facial features of the face; in response to detecting movement of the one or more facial features: modifying the virtual avatar to display an appearance different from the first appearance based on determining that the detected movement of the one or more facial features satisfies a first pose criterion. The virtual avatar is modified to display a first avatar feature having a third avatar feature having a different appearance from the first and second appearances, based on a determination that the detected movement of the one or more facial features satisfies a second pose criterion different from the first pose criterion; and the virtual avatar is modified to display a first avatar feature by modifying the first appearance of the first avatar feature in response to the detected change in facial pose in the field of view of the one or more cameras, based on a determination that the detected movement of the one or more facial features satisfies a criterion for maintaining the display of the first avatar feature having the first appearance.

[0029] This document describes an example non-transitory computer-readable storage medium. An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more cameras, the one or more programs including instructions for: displaying a virtual avatar having one or more avatar features via the display device, the one or more avatar features changing appearance in response to detected changes in facial pose in the field of view of the one or more cameras, including a first avatar feature having a first appearance, the first appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; detecting movement of one or more facial features of the face when a face including one or more detected facial features is detected in the field of view of the one or more cameras; in response to detecting movement of the one or more facial features: determining that the detected movement of the one or more facial features causes a load to be satisfied. The virtual avatar is modified to display a first avatar feature having a second appearance different from the first appearance, based on a first pose criterion, wherein the second appearance is modified in response to detected changes in facial pose in the field of view of the one or more cameras; the virtual avatar is modified to display a first avatar feature having a third appearance different from the first and second appearances, based on a determination that detected movement of the one or more facial features satisfies a second pose criterion different from the first pose criterion, wherein the third appearance is modified in response to detected changes in facial pose in the field of view of the one or more cameras; and the virtual avatar is modified to display a first avatar feature by modifying the first appearance of the first avatar feature in response to detected changes in facial pose in the field of view of the one or more cameras, based on a determination that detected movement of the one or more facial features satisfies a criterion for maintaining the display of the first avatar feature having the first appearance.

[0030] This document describes an example transient computer-readable storage medium. An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more cameras, the one or more programs including instructions for: displaying a virtual avatar having one or more avatar features via the display device, the one or more avatar features changing appearance in response to detected changes in facial pose in the field of view of the one or more cameras, including a first avatar feature having a first appearance, the first appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; detecting movement of one or more facial features of the face when a face including one or more detected facial features is detected in the field of view of the one or more cameras; in response to detecting movement of the one or more facial features: determining that the detected movement of the one or more facial features causes a condition to be met. A first pose criterion is used to modify the virtual avatar to display a first avatar feature having a second appearance different from the first appearance, the second appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; based on determining that the detected movement of the one or more facial features satisfies a second pose criterion different from the first pose criterion, the virtual avatar is modified to display a first avatar feature having a third appearance different from the first and second appearances, the third appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; and based on determining that the detected movement of the one or more facial features satisfies a criterion for maintaining the display of the first avatar feature having the first appearance, the virtual avatar is modified to display the first avatar feature by modifying the first appearance of the first avatar feature in response to detected changes in facial pose in the field of view of the one or more cameras.

[0031] This document describes an example electronic device. An example electronic device includes a display device; one or more cameras; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a virtual avatar having one or more avatar features via the display device, the one or more avatar features changing appearance in response to detected changes in facial pose in the field of view of the one or more cameras, including a first avatar feature having a first appearance, the first appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; detecting movement of one or more facial features of the face when a face including one or more detected facial features is detected in the field of view of the one or more cameras; and in response to detecting movement of the one or more facial features: determining the detected movement of the one or more facial features... The virtual avatar is modified to display a first avatar feature having a second appearance different from the first appearance, which is modified in response to detected changes in facial pose in the field of view of the one or more cameras, in order to satisfy a first pose criterion; the virtual avatar is modified to display a first avatar feature having a third appearance different from the first and second appearances, which is modified in response to detected changes in facial pose in the field of view of the one or more cameras, in order to satisfy a second pose criterion different from the first pose criterion, which is determined to satisfy a second pose criterion different from the first pose criterion; and the virtual avatar is modified to display a first avatar feature by modifying the first appearance of the first avatar feature in response to detected changes in facial pose in the field of view of the one or more cameras, in order to satisfy a first pose criterion for maintaining the display of the first avatar feature having the first appearance.

[0032] This document describes an example electronic device. An example electronic device includes a display device; one or more cameras; means for displaying a virtual avatar having one or more avatar features via the display device, the one or more avatar features changing appearance in response to detected changes in facial pose in the field of view of the one or more cameras, including a first avatar feature having a first appearance, the first appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; means for operating to: when a face including one or more detected facial features is detected in the field of view of the one or more cameras, detect movement of one or more facial features of the face; means for operating to: in response to detecting movement of the one or more facial features: modify the virtual avatar to display a virtual avatar based on determining that the detected movement of the one or more facial features satisfies a first pose criterion. The virtual avatar displays a first avatar feature having a second appearance different from the first appearance, the second appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; based on determining that the detected movement of the one or more facial features satisfies a second pose criterion different from the first pose criterion, the virtual avatar is modified to display a first avatar feature having a third appearance different from the first and second appearances, the third appearance being modified in response to detected changes in facial pose in the field of view of the one or more cameras; and based on determining that the detected movement of the one or more facial features satisfies a criterion for maintaining the display of the first avatar feature having the first appearance, the virtual avatar is modified to display the first avatar feature by modifying the first appearance of the first avatar feature in response to detected changes in facial pose in the field of view of the one or more cameras.

[0033] This document describes an example method. An example method includes, at an electronic device having a display device and one or more input devices: displaying a content creation user interface via the display device; while displaying the content creation user interface, receiving a request via the one or more input devices to display a first display area, the first display area including a plurality of graphical objects corresponding to predefined content for insertion into the content creation user interface, wherein displaying the first display area includes: in response to receiving the request, displaying via the display device a first display area comprising a first subgroup of graphical objects having an appearance based on a set of avatars available at the electronic device, including: based on determining that the set of avatars includes avatars of a first type, displaying one graphical object in the first subgroup having an appearance based on avatars of the first type; and based on determining that the set of avatars does not include any avatars of the first type, displaying graphical objects in the first subgroup having an appearance based on avatars of a second type different from the first type, without displaying one graphical object in the first subgroup having an appearance based on avatars of the first type.

[0034] This document describes an example non-transitory computer-readable storage medium. An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more input devices, the one or more programs including instructions for: displaying content creation user interface via the display device; and, while displaying the content creation user interface, receiving a request via the one or more input devices to display a first display area, the first display area including a plurality of graphical objects corresponding to predefined content for insertion into the content creation user interface, wherein displaying the first display area includes: in response to receiving the request, displaying via the display device a first display area comprising a first subgroup of graphical objects having an appearance based on a set of avatars available at the electronic device, including: based on determining that the set of avatars includes avatars of a first type, displaying one graphical object in the first subgroup having an appearance based on avatars of the first type; and based on determining that the set of avatars does not include any avatars of the first type, displaying graphical objects in the first subgroup having an appearance based on avatars of a second type different from the first type, without displaying one graphical object in the first subgroup having an appearance based on avatars of the first type.

[0035] This document describes an example transient computer-readable storage medium. An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more input devices, the one or more programs including instructions for: displaying content creation user interface via the display device; and, while displaying the content creation user interface, receiving a request via the one or more input devices to display a first display area, the first display area including a plurality of graphical objects corresponding to predefined content for insertion into the content creation user interface, wherein displaying the first display area includes: in response to receiving the request, displaying via the display device a first display area comprising a first subgroup of graphical objects having an appearance based on a set of avatars available at the electronic device, including: displaying one graphical object in the first subgroup having an appearance based on an avatar of the first type, depending on whether the set of avatars includes an avatar of the first type; and displaying graphical objects in the first subgroup having an appearance based on an avatar of the second type, different from the first type, while not displaying one graphical object in the first subgroup having an appearance based on an avatar of the first type, depending on whether the set of avatars does not include any avatar of the first type.

[0036] This document describes an example electronic device. An example electronic device includes a display device; one or more input devices; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying content creation user interface via the display device; and, while displaying the content creation user interface, receiving a request via the one or more input devices to display a first display area, the first display area including a plurality of graphical objects corresponding to predefined content for insertion into the content creation user interface, wherein displaying the first display area includes: in response to receiving the request, displaying via the display device a first display area comprising a first subgroup of graphical objects having an appearance based on a set of avatars available at the electronic device, including: based on determining that the set of avatars includes avatars of a first type, displaying one graphical object in the first subgroup having an appearance based on avatars of the first type; and based on determining that the set of avatars does not include any avatars of the first type, displaying graphical objects in the first subgroup having an appearance based on avatars of a second type different from the first type, without displaying one graphical object in the first subgroup having an appearance based on avatars of the first type.

[0037] This document describes an example electronic device. An example electronic device includes a display device; one or more input devices; means for creating a user interface by displaying content via the display device; means for: receiving, while displaying the content-creating user interface, a request to display a first display area via the one or more input devices, the first display area including a plurality of graphical objects corresponding to predefined content for insertion into the content-creating user interface, wherein displaying the first display area includes: in response to receiving the request, displaying, via the display device, a first display area comprising a first subgroup of graphical objects having an appearance based on a set of avatars available at the electronic device, including: based on determining that the set of avatars includes avatars of a first type, displaying one graphical object in the first subgroup having an appearance based on avatars of the first type; and means for: based on determining that the set of avatars does not include any avatars of the first type, displaying graphical objects in the first subgroup having an appearance based on avatars of a second type different from the first type, without displaying one graphical object in the first subgroup having an appearance based on avatars of the first type.

[0038] This document discloses exemplary methods. One example method includes, at an electronic device having one or more communication devices, wherein a user is associated with the electronic device, receiving a request to transmit a first message to a set of contactable users, the set of contactable users including a first contactable user; and in response to receiving the request to transmit the first message: based on determining that the first contactable user meets a set of shared criteria, the set of shared criteria including a first shared criterion satisfied when the first contactable user corresponds to an approved recipient: transmitting the first message and contact information of the user associated with the electronic device to the first contactable user via the one or more communication devices; and based on determining that the first contactable user does not meet the set of shared criteria: transmitting the first message to the first contactable user via the one or more communication devices without transmitting the contact information of the user associated with the electronic device.

[0039] This document describes an exemplary non-transitory computer-readable storage medium. One example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having one or more communication devices, wherein a user is associated with the electronic device and the one or more programs include instructions for: receiving a request to transmit a first message to a group of contactable users, the group of contactable users including a first contactable user; and in response to receiving the request to transmit the first message: based on determining that the first contactable user meets a set of shared criteria, the set of shared criteria including a first shared criterion satisfied when the first contactable user corresponds to an approved recipient: transmitting the first message and contact information of the user associated with the electronic device to the first contactable user via the one or more communication devices; and based on determining that the first contactable user does not meet the set of shared criteria: transmitting the first message to the first contactable user via the one or more communication devices without transmitting the contact information of the user associated with the electronic device.

[0040] This document describes an exemplary transient computer-readable storage medium. One example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having one or more communication devices, wherein a user is associated with the electronic device and the one or more programs include instructions for: receiving a request to transmit a first message to a group of contactable users, the group of contactable users including a first contactable user; and in response to receiving the request to transmit the first message: based on determining that the first contactable user meets a set of shared criteria, the set of shared criteria including a first shared criterion satisfied when the first contactable user corresponds to an approved recipient: transmitting the first message and contact information of the user associated with the electronic device to the first contactable user via the one or more communication devices; and based on determining that the first contactable user does not meet the set of shared criteria: transmitting the first message to the first contactable user via the one or more communication devices without transmitting the contact information of the user associated with the electronic device.

[0041] This document describes an exemplary electronic device. One example electronic device includes: one or more communication devices; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, wherein a user is associated with the electronic device and the one or more programs include instructions for: receiving a request to transmit a first message to a group of contactable users, the group of contactable users including a first contactable user; and in response to receiving the request to transmit the first message: based on determining that the first contactable user meets a set of shared criteria, the set of shared criteria including a first shared criterion satisfied when the first contactable user corresponds to an approved recipient: transmitting the first message and contact information of the user associated with the electronic device to the first contactable user via the one or more communication devices; and based on determining that the first contactable user does not meet the set of shared criteria: transmitting the first message to the first contactable user via the one or more communication devices without transmitting the contact information of the user associated with the electronic device.

[0042] This document describes an exemplary electronic device. An example electronic device includes one or more communication devices, wherein a user is associated with the electronic device; means for receiving a request to transmit a first message to a group of contactable users, the group of contactable users including a first contactable user; and means for: in response to receiving the request to transmit the first message: based on determining that the first contactable user meets a set of shared criteria, the set of shared criteria including a first shared criterion satisfied when the first contactable user corresponds to an approved recipient: transmitting the first message and contact information of the user associated with the electronic device to the first contactable user via the one or more communication devices; and based on determining that the first contactable user does not meet the set of shared criteria: transmitting the first message to the first contactable user via the one or more communication devices without transmitting the contact information of the user associated with the electronic device.

[0043] This document discloses exemplary methods. One example method includes, at an electronic device having a display device and one or more communication devices: receiving a first message via the one or more communication devices; after receiving the first message, receiving a request to display the first message; and in response to receiving the request to display the first message: simultaneously displaying on the display device: the first message and a visual indication that the updated contact information is available for the first contactable user, based on determining that a first contactable user meets a set of prompting criteria, wherein the set of prompting criteria includes a first prompting criterion that is met when updated contact information corresponding to the first contactable user has been received; and displaying on the display device, without displaying a visual indication that the updated contact information is available for the first contactable user, based on determining that the first contactable user does not meet the set of prompting criteria.

[0044] This document describes an exemplary non-transitory computer-readable storage medium. One example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more communication devices, the one or more programs including instructions for: receiving a first message via the one or more communication devices; receiving a request to display the first message after receiving the first message; and in response to receiving the request to display the first message: simultaneously displaying on the display device: the first message and a visual indication regarding the updated contact information available to the first contactable user, based on determining that a first contactable user meets a set of prompting criteria, wherein the set of prompting criteria includes a first prompting criterion that is met when updated contact information corresponding to the first contactable user has been received; and displaying on the display device, without displaying the visual indication regarding the updated contact information available to the first contactable user, based on determining that the first contactable user does not meet the set of prompting criteria.

[0045] This document describes an exemplary transient computer-readable storage medium. One example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more communication devices, the one or more programs including instructions for: receiving a first message via the one or more communication devices; receiving a request to display the first message after receiving the first message; and in response to receiving the request to display the first message: simultaneously displaying on the display device: the first message and a visual indication regarding the updated contact information available to the first contactable user, based on determining that a first contactable user meets a set of prompting criteria, wherein the set of prompting criteria includes a first prompting criterion that is met when updated contact information corresponding to the first contactable user has been received; and displaying on the display device, without displaying the visual indication regarding the updated contact information available to the first contactable user, based on determining that the first contactable user does not meet the set of prompting criteria.

[0046] This document describes an exemplary electronic device. One example device includes a display device; one or more communication devices; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: receiving a first message via the one or more communication devices; receiving a request to display the first message after receiving the first message; and in response to receiving the request to display the first message: simultaneously displaying on the display device: the first message and a visual indication regarding the updated contact information available to the first contactable user, based on determining that a first contactable user meets a set of prompting criteria, wherein the set of prompting criteria includes a first prompting criterion that is met when updated contact information corresponding to the first contactable user has been received; and displaying on the display device, without displaying a visual indication regarding the updated contact information available to the first contactable user, based on determining that the first contactable user does not meet the set of prompting criteria.

[0047] This document describes an exemplary electronic device. One example device includes a display device; one or more communication devices; means for receiving a first message via the one or more communication devices; means for receiving a request to display the first message after receiving the first message; and means for: in response to receiving the request to display the first message; simultaneously displaying on the display device: the first message and a visual indication that the updated contact information is available for the first contactable user, based on a determination that a first contactable user meets a set of prompting criteria, wherein the set of prompting criteria includes a first prompting criterion that is met when updated contact information corresponding to the first contactable user has been received; and displaying on the display device, without displaying a visual indication that the updated contact information is available for the first contactable user, based on a determination that the first contactable user does not meet the set of prompting criteria.

[0048] Executable instructions for performing these functions may optionally be included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0049] Therefore, this provides devices with faster and more efficient methods and interfaces for displaying avatars in various application user interfaces, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods used for displaying avatars in various application user interfaces. Attached Figure Description

[0050] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0051] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.

[0052] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0053] Figure 2 A portable multi-functional device with a touchscreen is shown according to some embodiments.

[0054] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0055] Figure 4AAn exemplary user interface for a menu on an application on a portable multifunction device, according to some implementation schemes, is shown.

[0056] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0057] Figure 5A A personal electronic device according to some implementation schemes is shown.

[0058] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0059] Figures 6A to 6V Exemplary user interfaces for displaying avatars in sticker application user interfaces and avatar keyboard application user interfaces, according to some implementations, are shown.

[0060] Figure 7 This is a flowchart illustrating a method for displaying an avatar in a sticker application user interface according to some implementation schemes.

[0061] Figure 8 This is a flowchart illustrating a method for displaying an avatar in an avatar keyboard application user interface according to some embodiments.

[0062] Figures 9A to 9AG An exemplary user interface for displaying an avatar in a contacts application user interface is shown according to some implementations.

[0063] Figure 10 This is a flowchart illustrating a method for displaying an avatar in a contact application user interface according to some implementations.

[0064] Figures 11A to 11AD An exemplary user interface for displaying an avatar in an avatar editing application user interface, according to some implementations, is shown.

[0065] Figure 12 This is a flowchart illustrating a method for displaying an avatar in the user interface of an avatar editing application, according to some implementation schemes.

[0066] Figure 13 This is a flowchart illustrating a method for displaying an avatar in the user interface of an avatar editing application, according to some implementation schemes.

[0067] Figures 14A to 14E An exemplary user interface for displaying virtual avatars is shown according to some implementation schemes.

[0068] Figure 15This is a flowchart illustrating a method for displaying a virtual avatar according to some implementation schemes.

[0069] Figures 16A to 16X Exemplary devices and user interfaces for sharing contact information are shown according to some implementation schemes.

[0070] Figure 17 This is a flowchart illustrating a method for providing contact information using an electronic device according to some implementation schemes.

[0071] Figure 18 This is a flowchart illustrating a method for receiving contact information using an electronic device according to some implementation schemes. Detailed Implementation

[0072] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0073] Electronic devices require efficient methods and interfaces for displaying avatars in various application user interfaces. For example, existing applications display avatars, but the processes used for doing so are often cumbersome and inefficient. Furthermore, such processes do not provide seamless integration of avatars with other user interfaces. This paper discloses techniques for displaying avatars in various application user interfaces. These techniques can reduce the cognitive burden on users using avatars in various application user interfaces, thereby improving productivity. In addition, these techniques can reduce processor power and battery power that would otherwise be wasted on redundant user input.

[0074] under, Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5B Descriptions of exemplary devices for displaying avatars in various application user interfaces are provided. Figures 6A to 6V Exemplary user interfaces for displaying avatars in sticker application user interfaces and avatar keyboard application user interfaces, according to some implementations, are shown. Figure 7 This is a flowchart illustrating a method for displaying an avatar in a sticker application user interface according to some implementation schemes. Figure 8 This is a flowchart illustrating a method for displaying an avatar in an avatar keyboard application user interface according to some embodiments. Figures 6A to 6V The user interface in the document is used to display including Figure 7 and Figure 8 The process described below is the process in the middle. Figures 9A to 9AGAn exemplary user interface for displaying an avatar in a contacts application user interface is shown according to some implementations. Figure 10 This is a flowchart illustrating a method for displaying an avatar in a contact application user interface according to some implementations. Figures 9A to 9AG The user interface in the document is used to display including Figure 10 The process described below is the process in the middle. Figures 11A to 11AD An exemplary user interface for displaying an avatar in an avatar editing application user interface, according to some implementations, is shown. Figure 12 This is a flowchart illustrating a method for displaying an avatar in the user interface of an avatar editing application, according to some implementation schemes. Figure 13 This is a flowchart illustrating a method for displaying an avatar in the user interface of an avatar editing application, according to some implementation schemes. Figures 11A to 11AD The user interface in the document is used to display including Figure 12 and Figure 13 The process described below is the process in the middle. Figures 14A to 14E An exemplary user interface for displaying virtual avatars is shown according to some implementation schemes. Figure 15 This is a flowchart illustrating a method for displaying a virtual avatar according to some implementation schemes. Figures 14A to 14E The user interface in the document is used to display including Figure 15 The process in the middle is the process in the following text. Figures 16A to 16X Exemplary devices and user interfaces for sharing contact information are shown according to some implementation schemes. Figure 17 This is a flowchart illustrating a method for providing contact information using an electronic device according to some implementation schemes. Figure 18 This is a flowchart illustrating a method for receiving contact information using an electronic device according to some implementation schemes. Figures 16A to 16X The exemplary device and user interface in the example are used to illustrate including Figure 17 and Figure 18 The process in the middle is the process in the following text.

[0075] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first touch may be named a second touch and similarly, a second touch may be named a first touch, without departing from the scope of the various described embodiments. Both the first touch and the second touch are touches, but they are not the same touch.

[0076] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “includes”, “including”, “comprises”, and / or “comprising”, when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0077] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "at," or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if determination..." or "if detection [the stated condition or event]" may optionally be interpreted as meaning "in response to determination..." or "in response to detection [the stated condition or event]."

[0078] This document describes implementations of electronic devices, user interfaces for such devices, and related processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, those from Apple Inc. (Cupertino, California). Devices, iPod Equipment, and Device. Optionally, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0079] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0080] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0081] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0082] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0083] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values ​​that includes at least four different values ​​and more typically hundreds of different values ​​(e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or variation of the contact area detected on the touch-sensitive surface, the capacitance and / or variation of the touch-sensitive surface near the contact, and / or the resistance and / or variation of the touch-sensitive surface near the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold (e.g., the intensity threshold is described in units corresponding to the substitute measurement) has been exceeded. In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold (e.g., the intensity threshold is a pressure threshold measured in units of pressure) has been exceeded. Using the intensity of the contact as an attribute of user input allows the user to access additional device functions that would otherwise be inaccessible to the user on smaller devices with limited physical space, such smaller devices being used (e.g., on a touch-sensitive display) to display power indications and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0084] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement does not move. For example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or sensed by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user.

[0085] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0086] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls other components of device 100 to access memory 102.

[0087] Peripheral interface 118 can be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0088] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals and vice versa, and communicates with communication networks and other communication devices via these electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates with networks and other devices wirelessly, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as through near-field communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0089] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset with both output (e.g., a mono or binaural headphone) and input (e.g., a microphone).

[0090] I / O subsystem 106 couples input / output peripherals on device 100, such as touchscreen 112 and other input control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive / send electrical signals from / to other input control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dials, slide switches, joysticks, click dials, etc. In some alternative embodiments, input controllers 160 are optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing devices such as a mouse. One or more buttons (e.g., Figure 2 Optionally, 208) includes an increase / decrease button for volume control of speaker 111 and / or microphone 113. One or more buttons optionally include a push-button (e.g., Figure 2 (206 in the middle).

[0091] A quick press of the push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of the push button (e.g., 206) optionally powers the device 100 on or off. The function of one or more buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0092] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0093] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0094] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that from Apple Inc. (Cupertino, California). and iPod The technology used.

[0095] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles the multi-touch-sensitive touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0096] In some embodiments, the touchscreen 112 has a touch-sensitive display as described in the following patent applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, entitled “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, entitled “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, entitled “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, entitled “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 18, 2005, entitled “Mode-Based Graphical User Interfaces For Touch Sensitive”. (6) U.S. Patent Application No. 11 / 038,590 entitled “Input Devices”; (7) U.S. Patent Application No. 11 / 228,758 entitled “Virtual Input Device Placement On A Touch Screen User Interface”, filed September 16, 2005; (8) U.S. Patent Application No. 11 / 228,700 entitled “Operation Of A Computer With A Touch Screen Interface”, filed September 16, 2005; (9) U.S. Patent Application No. 11 / 228,737 entitled “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard”, filed September 16, 2005; and (10) U.S. Patent Application No. 11 / 367,749 entitled “Multi-Functional Hand-Held Device”, filed March 3, 2006. The full text of all these applications is incorporated herein by reference.

[0097] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally interact with touchscreen 112 using any suitable object or accessory such as a stylus, finger, etc. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.

[0098] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. Optionally, the touchpad is a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0099] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0100] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 is shown coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensor within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0101] The device 100 optionally also includes one or more depth camera sensors 175. Figure 1A A depth camera sensor 175 is shown coupled to a depth camera controller 169 in I / O subsystem 106. The depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, the depth camera sensor 175 is optionally used, in conjunction with an imaging module 143 (also referred to as a camera module), to determine depth maps of different portions of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information are optionally acquired for video conferencing while the user views other video conferencing participants on a touchscreen display, and selfies with depth map data are captured. In some embodiments, the depth camera sensor 175 is located at the rear of the device, or both the rear and front of device 100. In some embodiments, the position of the depth camera sensor 175 can be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that the depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.

[0102] In some implementations, the depth map (e.g., a depth map image) contains information (e.g., values) relating to the distance of objects in the scene from the viewpoint (e.g., a camera, optical sensor, depth camera sensor). In one implementation of the depth map, each depth pixel defines the location of its corresponding two-dimensional pixel on the Z-axis of the viewpoint. In some implementations, the depth map consists of pixels, where each pixel is defined by a value (e.g., 0 to 255). For example, a "0" value represents the pixel furthest from the viewpoint (e.g., a camera, optical sensor, depth camera sensor) in the "3D" scene, and a "255" value represents the pixel closest to the viewpoint in the "3D" scene. In other implementations, the depth map represents the distance between objects in the scene and the plane of the viewpoint. In some implementations, the depth map includes information about the relative depth of various features of the object of interest within the field of view of the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of a user's face). In some implementations, the depth map includes information that enables the device to determine the contour of the object of interest in the z-direction.

[0103] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor 165 is shown coupled to a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0104] The device 100 optionally also includes one or more proximity sensors 166. Figure 1AA proximity sensor 166 is shown coupled to a peripheral device interface 118. Alternatively, the proximity sensor 166 may be optionally coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally perform as described in the following U.S. patent applications: No. 11 / 241,839, entitled "Proximity Detector In Handheld Device"; No. 11 / 240,788, entitled "Proximity Detector In Handheld Device"; No. 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; No. 11 / 586,862, entitled "Automated Response To And Sensing Of U.S. patent applications No. 11 / 638,251, entitled “User Activity In Portable Devices”, are incorporated herein by reference in their entirety. In some embodiments, the proximity sensor is turned off and the touchscreen 112 is disabled when the multifunction device is placed near the user’s ear (e.g., when the user is making a phone call).

[0105] The device 100 may optionally also include one or more tactile output generators 167. Figure 1AA haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components; and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymerizers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., a haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0106] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally perform as described in the following U.S. patent publications: U.S. Patent Publication No. 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and U.S. Patent Publication No. 20060017692, entitled "Methods and Apparatuses For Operating A Portable Device Based On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.

[0107] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application program (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the diagram. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, which indicates what applications, views or other information occupy various areas of the touchscreen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and positional information about the device's position and / or orientation.

[0108] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0109] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is connected to… (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.

[0110] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0111] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0112] The touch / motion module 130 optionally detects the user's gesture input. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0113] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other display, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual characteristics). As used herein, the term "graphics" includes any object that can be displayed to a user, and non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0114] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like to specify the graphic to be displayed, and, if necessary, also receives coordinate data and other graphic attribute data, and then generates screen image data for output to the display controller 156.

[0115] The haptic feedback module 133 includes various software components for generating instructions that are used by the haptic output generator 167 to produce haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0116] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).

[0117] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to phone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather desktop apps, local yellow pages desktop apps, and map / navigation desktop apps).

[0118] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0119] • Contacts module 137 (sometimes called address book or contact list);

[0120] • Telephone module 138;

[0121] • Video conferencing module 139;

[0122] • Email client module 140;

[0123] • Instant Messaging (IM) module 141;

[0124] Fitness support module 142;

[0125] • Camera module 143 for still images and / or video images;

[0126] • Image management module 144;

[0127] • Video player module;

[0128] Music player module;

[0129] • Browser module 147;

[0130] • Calendar module 148;

[0131] • Desktop mini-program module 149, which optionally includes one or more of the following: weather desktop mini-program 149-1, stock market desktop mini-program 149-2, calculator desktop mini-program 149-3, alarm clock desktop mini-program 149-4, dictionary desktop mini-program 149-5, and other desktop mini-programs obtained by the user, and desktop mini-programs created by the user 149-6;

[0132] • Desktop app creator module 150 for creating user-created desktop apps 149-6;

[0133] • Search module 151;

[0134] • Video and music player module 152, which combines a video player module and a music player module;

[0135] • Notepad module 153;

[0136] • Map module 154; and / or

[0137] • Online video module 155.

[0138] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0139] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0140] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify entered telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0141] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0142] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0143] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing a received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0144] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (executive devices); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0145] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0146] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0147] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.

[0148] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0149] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, desktop applet module 149 is optionally a micro-application downloaded and used by a user (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) or a user-created micro-application (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop applet includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop applet).

[0150] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, the desktop applet creator module 150 can optionally be used by the user to create desktop applets (e.g., converting user-specified portions of a webpage into desktop applets).

[0151] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0152] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0153] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0154] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is optionally used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0155] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following actions: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Further descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0156] Each of the above modules and applications corresponds to an executable instruction set for performing one or more of the functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods as described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 optionally stores a subset of the above-described modules and data structures. Additionally, memory 102 optionally stores other modules and data structures not described above.

[0157] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on device 100 can be optionally reduced.

[0158] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to a main menu, home menu, or root menu. In such embodiments, a touchpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0159] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 137 to 151, 155, 380 to 390).

[0160] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0161] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0162] Event monitor 171 receives event information from peripheral device interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripheral device interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral device interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0163] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other implementations, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0164] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0165] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0166] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0167] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0168] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0169] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0170] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0171] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0172] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies events from the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0173] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device pose).

[0174] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event (187) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on the displayed object for a predetermined duration, movement of the touch on the touch-sensitive display 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0175] In some implementations, event definition 187 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0176] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0177] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0178] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0179] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some implementations, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with that tag receives the tag and executes a predefined procedure.

[0180] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined procedure.

[0181] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on touch-sensitive display.

[0182] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0183] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0184] Figure 2A portable multifunction device 100 with a touchscreen 112 is shown according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0185] Device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0186] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0187] Figure 3This is a block diagram of an exemplary multifunctional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 167 and sensors 359 (e.g., optical sensors, accelerometers, proximity sensors, touch sensors, and / or contact intensity sensors similar to those mentioned above) are described. Figure 1A The one or more contact strength sensors 165 mentioned above. Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores data with portable multifunction device 100 (…). Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0188] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each of the modules described above corresponds to an instruction set for performing the functions described above. The modules or programs (e.g., instruction sets) described above need not be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures described above. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0189] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.

[0190] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is shown. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof:

[0191] • One or more signal strength indicators 402 for wireless communications such as cellular signals and Wi-Fi signals;

[0192] • Time 404;

[0193] Bluetooth indicator 405;

[0194] • Battery status indicator 406;

[0195] • Tray tray 408 with icons for commonly used applications, such as:

[0196] ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemails;

[0197] ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails;

[0198] ○ The icon 420 labeled "Browser" in browser module 147; and

[0199] ○ The icon 422 labeled "iPod" for the video and music player module 152 (also known as the iPod (a trademark of Apple Inc.) module 152); and

[0200] • Icons of other applications, such as:

[0201] ○The icon 424 of the IM module 141 marked as "Message";

[0202] ○The calendar module 148 has an icon 426 labeled "Calendar";

[0203] ○ The icon 428 of the image management module 144 is labeled "Photo".

[0204] ○ The icon 430 of camera module 143, which is labeled "camera";

[0205] ○ The icon 432 of the online video module 155, which is labeled "Online Video";

[0206] ○ The icon 434 in the Stock Market Desktop Mini Program 149-2 that is marked as "Stock Market";

[0207] ○The icon 436 of the map module 154 that is labeled "map";

[0208] ○ The icon 438 in the Weather desktop mini-program 149-1 that is marked as "Weather";

[0209] ○ The icon 440 in the alarm clock desktop mini-program 149-4 that is marked as "clock";

[0210] ○ The icon 442 of the fitness support module 142 is labeled "fitness support";

[0211] ○ The icon 444 labeled "Notepad" in Notepad module 153; and

[0212] ○ The icon 446, labeled "Settings," is used to set the settings of the device 100 and its various applications 136.

[0213] It should be pointed out that, Figure 4A The icon labels shown are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0214] Figure 4B A touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)) is shown. Figure 3 Devices (e.g., tablets or touchpads 355) Figure 3An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.

[0215] While some examples of input on a reference touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations the device detects input on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450)). Figure 4B The spindle corresponding to 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to the corresponding position on the display (e.g., in the example). Figure 4B In the middle, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, on touch-sensitive surfaces (e.g., Figure 4B 451) and the display of the multi-functional device ( Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on the touch-sensitive surface (e.g., touches on 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0216] Additionally, while the examples below are primarily given with reference to finger input (e.g., finger touch, single-finger tap, finger swipe), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the swipe path (e.g., instead of movement of the touch). Similarly, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Likewise, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0217] Figure 5AAn exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include components relative to devices 100 and 300 (e.g., Figures 1A to 4B Some or all of the features described herein. In some embodiments, device 500 has a touch-sensitive display 504, referred to below as touchscreen 504. As an alternative to or complement to touchscreen 504, device 500 has a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., touch). The one or more intensity sensors of touchscreen 504 (or touch-sensitive surface) can provide output data representing the intensity of the touch. The user interface of device 500 can respond to touches based on the intensity of the touch, meaning that touches of different intensities can invoke different user interface operations on device 500.

[0218] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related patent applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application,” published as WIPO Patent Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships,” published as WIPO Patent Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0219] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) may be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0220] Figure 5B An exemplary personal electronic device 500 is illustrated. In some embodiments, device 500 may include a reference. Figure 1A , Figure 1B and Figure 3 Some or all of the components described herein. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive components 522 and optionally have an intensity sensor 524 (e.g., a contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 may optionally be a rotatable input device or a pressable input device and a rotatable input device. In some examples, input mechanism 508 may optionally be a button.

[0221] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.

[0222] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause the computer processors to perform techniques including processes 700, 800, 1000, 1200, 1300, 1500, 1700, and 1800. Figure 7 , Figure 8 , Figure 10 , Figure 12 , Figure 13 , Figure 15 , Figure 17 and Figure 18 A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.

[0223] As used herein, the term "power indication" refers optionally to the power indication in devices 100, 300, and / or 500 ( Figure 1A , Figure 3 and Figures 5A to 5B A user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.

[0224] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of enabling direct interaction with user interface elements on the touchscreen display) Figure 1A The touch-sensitive display system 112 or Figure 4AIn some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0225] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted away, before or after contact begins to move, before contact ends, before or after contact intensity is detected to increase and / or before or after contact intensity decreases). The characteristic intensity of the contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half maximum value of the contact intensity, the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some implementations, the feature intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a feature intensity not exceeding the first threshold results in a first action, contact with a feature intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a feature intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the feature intensity and one or more thresholds is used to determine whether to perform one or more actions (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0226] In some implementations, a portion of the gesture is identified to determine the characteristic intensity. For example, a touch-sensitive surface optionally receives a series of swipe contacts that transition from a starting position to an ending position, where the contact intensity increases. In this example, the characteristic intensity of the contact at the ending position is optionally based only on a portion of the series of swipe contacts, rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some implementations, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: unweighted moving average smoothing algorithm, triangular smoothing algorithm, median filter smoothing algorithm, and / or exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contact to achieve the purpose of determining the characteristic intensity.

[0227] Optionally, the contact intensity on a touch-sensitive surface is characterized relative to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device performs an operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device performs an operation different from the operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, when a contact with a characteristic intensity lower than the light press intensity threshold (e.g., and higher than the nominal contact detection intensity threshold, where contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector based on the movement of the contact on the touch-sensitive surface without performing the operation associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent across different groups of user interface figures.

[0228] An increase in contact intensity from below a light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in contact intensity from below a deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in contact intensity from below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in contact intensity from above a contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a contact being lifted off the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0229] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the intensity of the contact (or multiple contacts) increases to above a press input intensity threshold. In some embodiments, the corresponding operation is performed in response to detecting that the intensity of the corresponding contact increases to above a press input intensity threshold (e.g., a "downward stroke" of the corresponding press input). In some embodiments, the press input includes the intensity of the corresponding contact increasing to above a press input intensity threshold and the intensity of the contact subsequently decreasing to below the press input intensity threshold, and the corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the press input threshold (e.g., an "upward stroke" of the corresponding press input).

[0230] In some implementations, the device employs intensity hysteresis to avoid unintended inputs sometimes referred to as "jitter," wherein the device defines or selects a hysteresis intensity threshold that has a predefined relationship with a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the press input intensity threshold). Therefore, in some implementations, a press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of that contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., an "upstroke" of the corresponding press input). Similarly, in some implementations, a press input is detected only when the device detects that the contact intensity increases from an intensity equal to or below the hysteresis intensity threshold to an intensity equal to or above the press input intensity threshold and optionally the contact intensity subsequently decreases to an intensity equal to or below the hysteresis intensity threshold, and a corresponding operation is performed in response to detecting a press input (e.g., an increase or decrease in contact intensity depending on the environment).

[0231] For ease of explanation, optionally, the description of an operation triggered in response to a press input associated with a press input strength threshold or in response to a gesture including a press input is provided in response to detecting any of the following conditions: the contact strength increases to above the press input strength threshold, the contact strength increases from below a hysteresis strength threshold to above the press input strength threshold, the contact strength decreases to below the press input strength threshold, and / or the contact strength decreases to below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in the example where the operation is described as being performed in response to detecting a decrease in contact strength below the press input strength threshold, the operation is optionally performed in response to detecting a decrease in contact strength below a hysteresis strength threshold corresponding to and less than the press input strength threshold.

[0232] Now let’s turn our attention to the implementation of user interfaces (“UIs”) and associated processes on electronic devices such as portable multifunction devices 100, 300 or 500.

[0233] Figures 6A to 6V Exemplary user interfaces for displaying avatars in sticker application user interfaces and avatar keyboard application user interfaces, according to some embodiments, are shown. The user interfaces in these figures are used to illustrate including... Figure 7 and Figure 8 The process described below is the process in the middle.

[0234] Figures 6A to 6VThe illustration shows exemplary user input and corresponding changes to one or more user interfaces that can be displayed on an electronic device (such as...). Figure 6A The device 600 is shown in the electronic device 600. Device 600 includes a display 601 (in some cases, the display is a touch-sensitive display) and a camera 602 (in some embodiments, the camera includes an image sensor capable of capturing data representing a portion of the spectrum (e.g., visible light, infrared light, or ultraviolet light)). In some embodiments, camera 602 includes multiple image sensors and / or other types of sensors. In addition to capturing data representing sensed light, in some embodiments, camera 602 is capable of capturing other types of data such as depth data. For example, in some embodiments, camera 602 also uses techniques based on speckle, time-of-flight, parallax, or focus to capture depth data. The image data captured by device 600 using camera 602 includes data corresponding to a portion of the spectrum of the scene within the camera's field of view. Additionally, in some embodiments, the captured image data also includes depth data of light data. In some other embodiments, the captured image data contains data sufficient to determine or generate depth data for that portion of the spectrum. In some embodiments, device 600 includes one or more features of device 100, 300, or 500.

[0235] In some examples, the electronic device 600 includes a depth camera, such as an infrared camera, a thermal imaging camera, or a combination thereof. In some examples, the device also includes a light emitting device (e.g., a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. Optionally, the light emitting device is used to illuminate an object during image capture by a visible light camera and a depth camera (e.g., an IR camera), and information from the depth camera and the visible light camera is used to determine depth maps of different portions of the object captured by the visible light camera. In some embodiments, parallax information from both cameras (e.g., two visible light cameras) is used to display the lighting effects described herein for a backward image, and depth information from the depth camera is combined with image data from the visible light camera for a forward image (e.g., a self-portrait). In some embodiments, the same user interface is used when determining depth information using both visible light cameras and when determining depth information using the depth camera, thus providing a consistent user experience even when using drastically different techniques to determine the information used to produce the lighting effects. In some implementations, when displaying a camera user interface with one of the lighting effects applied, the device detects the selection of a camera switching display and switches from a front camera (e.g., a depth camera and a visible light camera) to a rear camera (e.g., two visible light cameras spaced apart from each other) (or vice versa), while maintaining the display of user interface controls to apply the lighting effect and replacing the field of view display of the front camera with the field of view display of the rear camera (or vice versa).

[0236] exist Figure 6A In this device 600, a messaging user interface 603 is displayed. This messaging user interface is the user interface of a messaging application. The messaging user interface 603 includes a message area 603-1 for displaying messages transmitted between the parties in a messaging conversation, a message writing area 603-2 for displaying content being written for transmission in the messaging conversation, and a keyboard area 603-3 for displaying various keyboard interfaces. Figures 6A to 6H In the implementation shown, the message corresponds to a message conversation with the first receiver 607-1.

[0237] exist Figure 6A In the process, device 600 detects input 604 (e.g., a tap gesture) on the power indicator 606 and, in response, displays an avatar keyboard 605 in keyboard areas 603-3, such as... Figure 6B As shown.

[0238] The avatar keyboard 605 includes various graphical objects that can be selected for transmission in the messaging user interface 603. The avatar keyboard 605 includes an emoji area 608 and a sticker area 610. The emoji area displays a set of emojis 609 that can be selected for transmission in the messaging user interface 603. The sticker area displays stickers 612 and a sticker application enablement 614, which, when selected, displays the sticker user interface 615. Figure 6C (As shown).

[0239] like Figure 6B As shown, sticker area 610 includes sticker 612, which can be selected for transmission in messaging user interface 603. The stickers displayed in sticker area 610 each have an appearance based on various avatars available at device 600. These stickers also include the pose or expression of the avatar upon which the sticker's appearance is based. For example, monkey sticker 612-1 is displayed with a monkey avatar having a surprised expression, poop sticker 612-2 is displayed with a poop avatar having heart-shaped eyes, and robot sticker 612-3 is displayed with a robot avatar in a neutral pose.

[0240] In some implementations, device 600 selectively displays various stickers 612 in sticker area 610 based on multiple factors. For example, in some implementations, device 600 selectively displays individual stickers based on usage history, such as recently created stickers (e.g., recommended / suggested stickers for recently created avatars) and stickers frequently used or recently used by the user. In some implementations, device 600 selectively displays different stickers 612, such that various sticker poses are represented in sticker area 610.

[0241] Device 600 also selectively displays various stickers 612 in sticker area 610 based on the type of avatar available at device 600. For example, different types of avatars may include avatars based on non-human characters, avatars based on human characters, user-created (e.g., user-created and / or customized), or predefined (e.g., not user-created or customized). Figure 6B In the illustrated implementation, the monkey sticker 612-1, the poop sticker 612-2, and the robot sticker 612-3 each have an appearance based on a predefined avatar, which is a non-human character.

[0242] Device 600 displays stickers 612 for these avatars available on device 600. Therefore, if a particular avatar is not available on device 600, the sticker area 610 does not include a sticker for that avatar. For example, in Figure 6B In this device, no user-created avatars or avatars based on human characters are included, and therefore no stickers for such avatars are displayed. As discussed in more detail below, when such avatars are available at device 600, sticker area 610 displays one or more stickers for the avatar.

[0243] The sticker area 610 also includes a sticker application display 614. The sticker application display 614 has an appearance that includes various stickers. For example, in Figure 6B In the sticker application capability representation 614, there are representations 614-1 of different stickers. Device 600 selectively displays various sticker representations in the sticker application capability representation 614 based on multiple factors. For example, in some embodiments, when a type of avatar is not available at device 600, the device displays a sticker representation based on an example avatar of a type that is not available at device 600. For example, in Figure 6BIn some implementations, when a user-created or human-based avatar is unavailable at device 600, device 600 displays representations 614-1 of different user-created and human-based avatar stickers. In some implementations, when a new avatar is created, device 600 updates the sticker application display representation 614 to include a representation of stickers based on the newly created avatar. In some implementations, device 600 displays representations of stickers for different types of avatars. In some implementations, device 600 selectively displays different sticker representations, such that various sticker poses are represented in the sticker application display representation 614. In some implementations, the sticker representations are displayed in an animated sequence, where different representations are individually cycled on the sticker application display representation 614.

[0244] exist Figure 6B In the process, device 600 detects input 616 on sticker application enable display 614, and in response, uses, for example Figure 6C The sticker user interface 615 shown replaces the display of the avatar keyboard 605 and the writing area 603-2.

[0245] like Figure 6C As shown, device 600 displays a sticker user interface 615 and a sticker area 620. The sticker user interface has an area 618 with representations 622 of multiple sets of stickers. Each sticker in the sticker area corresponds to a sticker representation selected in the area 618. Representation 622 corresponds to multiple sets of stickers available at device 600. A user can view different sets of stickers by selecting a different representation 622 (e.g., by touching the corresponding representation 622 in area 618, or by swiping horizontally on sticker area 620). When a different representation 622 is selected, device 600 updates area 618 to indicate the selected representation 622 and updates sticker area 620 to display the sticker corresponding to the selected representation. Figure 6C In the first area 618, monkey representation 622-1 is selected, and monkey sticker 624 is displayed in sticker area 620. In some embodiments, sticker 624 is shown with slight animations, such as laughing, blinking, waving, etc. Monkey sticker 624 includes various poses, such as the confused pose shown in the afro monkey sticker 624-1.

[0246] In some implementations, device 600 displays area 618 when the sticker user interface 615 is first displayed, and then hides area 618 (e.g., for subsequent instances of interface 615, area 618 is not initially displayed). The user can cause device 600 to redisplay area 618 by dragging sticker area 620, such as... Figure 6L As shown.

[0247] See now Figures 6C to 6E Region 618 also includes a creation power representation 626, which can be selected to create a new avatar. As shown, region 618 does not include any representation of a user-created or human-type avatar, because no such avatar is currently available at device 600. Therefore, device 600 displays a paddle-shaped region 628 that extends from the creation power representation 626 and has an animation of representations 628-1 of avatars cycling on that paddle region, such as by targeting Figures 6C to 6E The different representations shown indicate that the animation provides the user with an indication that no user-created or human-type avatars are available at device 600, and encourages the user to select paddle area 628 to create an avatar.

[0248] exist Figure 6E In the process, device 600 detects input 630 on the creation display 626 and, in response, displays the avatar creation user interface 632, such as... Figure 6F As shown. Device 600 detects input in the avatar creation user interface 632, typically represented by input 634, to select various avatar features to construct / create a new avatar (round hat avatar 636), such as... Figure 6G As shown. In response to input 638 on the completion display 640, device 600 exits the avatar creation user interface 632 and returns to Figure 6H The messaging user interface 603 shows a sticker user interface 615, which is updated to include representations 622-2 of the round hat avatar 636 selected in area 618 and round hat stickers 642, which have the appearance of round hat avatar 636 but have different poses for each of the corresponding round hat stickers 642. Figure 6C The monkey sticker shown is one of many sticker poses similar to the one depicted. In some implementations, after creating the round hat avatar 636, the new avatar can then be used on device 600, including in other applications such as a contacts application, a camera application, a media viewing application, and other applications on device 600. Additionally, the round hat avatar 636 can be updated, including in other applications.

[0249] exist Figure 6H In this process, device 600 detects input 644 on a thumbs-up hat sticker 642-1, which is a sticker with the appearance of a hat avatar 636 and a "thumbs-up" gesture. In some embodiments, selecting the thumbs-up hat sticker 642-1 causes device 600 to add the sticker to a messaging conversation (e.g., to send to a first recipient 607-1). In some embodiments, selecting the thumbs-up hat sticker 642-1 causes device 600 to display the thumbs-up hat sticker 642-1 in the avatar keyboard 605, such as... Figure 6I As shown.

[0250] Figures 6I to 6V A message delivery user interface 603 is shown for an implementation of a message dialogue between a message recipient 607-2 and the second recipient 607-2. Figure 6I In the device 600, an avatar keyboard 605 is displayed, which has a thumbs-up round hat sticker 642-1 instead of the monkey sticker 612-1. Additionally, the device 600 updates the display of the sticker application's enabled representation 614 to include a representation 614-2 of the round hat avatar 636. In some embodiments, representation 614-2 has the appearance of the recently used thumbs-up round hat sticker 642-1. In some embodiments, representation 614-2 has the appearance of other stickers that can be used for the recently created round hat avatar 636.

[0251] exist Figure 6I In this process, device 600 detects input 646 on the thumbs-up round hat sticker 642-1 and, in response, displays a sticker preview interface showing a round hat sticker preview 650. In some embodiments, a user can perform a tap and hold gesture on the round hat sticker preview 650 to generate a peel-off appearance of the round hat sticker preview 650, which can then be dragged to message area 603-1 to add the round hat sticker to a message conversation. In some embodiments, a user can (e.g., via input 654) choose to send an enable signal 652 to add the thumbs-up round hat sticker 642-1 to a message conversation, such as... Figure 6K As shown.

[0252] exist Figure 6K In this process, device 600 detects input 656 on sticker application enable display 614 and, in response, displays sticker user interface 615. In some embodiments, device 600 stops the display of emoji 609 and displays stickers (e.g., round hat sticker 642 corresponding to round hat avatar 636) in emoji area 608.

[0253] exist Figure 6L In the middle, device 600 displays a sticker user interface 615 with a round cap sticker 642. Figure 6L In the illustrated embodiment, device 600 has the previously displayed sticker user interface (e.g., Figure 6C (in the middle), and therefore area 618 is not initially displayed. Additionally, device 600 has generated a second user-created avatar (e.g., due to receiving a series of inputs to access the avatar creation user interface and relative to the above). Figures 6E to 6G The discussion follows a similar approach to interacting with the avatar creation user interface, which will soon become apparent. In response to drag input 658, the device scrolls the round hat sticker 642, and... Figure 6MThe display shows a region 618 with a representation of 622, which includes a round cap representation 622-2, a round cap representation having a selected state, and a female representation 622-3 corresponding to a set of female stickers based on a user-created female avatar.

[0254] exist Figure 6M In the process, device 600 detects input 660 on the creation enablement display 626 to initiate a process for creating a boy's avatar. The process for creating a boy's avatar is similar to the avatar creation process described above, and for simplicity, will not be repeated here. After device 600 creates the boy's avatar, the device displays as follows: Figure 6N The sticker user interface 615 shown. Figure 6N Area 618 and sticker area 620 are shown. The area has been updated to include boy representations 622-4 with a selected state. The sticker area has been updated to include boy stickers 662, which include a set of poses based on the new boy avatar.

[0255] exist Figure 6N In the middle, device 600 detects scroll input 664, and in response, scrolls the sticker area 620 to display additional boy stickers 662 and an editing display 665, such as Figure 6O As shown.

[0256] exist Figure 6O In the process, device 600 detects input 668 on the editing display 665 and, in response, displays an avatar editing user interface 670 (similar to avatar creation user interface 632), which shows a boy's avatar 672 and a set of optional hairstyle options 674 that can be selected to modify the appearance of the boy's avatar 672, such as... Figure 6P As shown.

[0257] exist Figure 6P In the process, device 600 detects input 676 on the round hairstyle option 674-1, and in response, modifies the boy's avatar 672 to have a round hairstyle, as shown. Figure 6Q As shown. Device 600 detected... Figure 6Q The completion indicator 680 displays input 678, and in response, exits the avatar editing user interface 670 and displays the messaging user interface 603 with an updated sticker user interface 615, as shown. Figure 6R As shown.

[0258] exist Figure 6R In the middle, device 600 displays a sticker user interface 615, which shows a boy sticker 662 updated with a bouncy hairstyle. Device 600 detects drag input 682 on handle 684 and, in response, expands sticker user interface 615, as... Figure 6S As shown.

[0259] Figure 6S An additional boy sticker 662 updated with a rounded hairstyle is shown. Device 600 detects drag input 686 (e.g., dragging downwards) and, in response, scrolls sticker 662 to display additional boy stickers 662, including a thumbs-up boy sticker 662-1 and a heart-shaped eye boy sticker 662-2, as shown. Figure 6T As shown.

[0260] exist Figure 6T In the sticker user interface 615, an area 618 is shown including a boy representation 622-4, which has been updated to include a rounded hairstyle. Device 600 detects input 688 on the robot representation 622-5 and, in response, selects the robot representation 622-5 and replaces the boy sticker 662 with a robot sticker 690, as shown. Figure 6U As shown. In some implementations, in response to one or more horizontal swipe gestures on the sticker area 620, robot representations 622-5 can be selected and the corresponding robot stickers 690 can be displayed.

[0261] exist Figure 6U In the device 600, a robot sticker 690 with an appearance based on a robot avatar (as discussed previously) is displayed. This robot sticker is based on a predefined avatar of a non-human character. In some embodiments, stickers based on such avatars (e.g., predefined avatars or avatars based on non-human characters) include stickers with poses that match those of stickers based on user-created avatars or avatars based on human characters. For example, robot sticker 690 includes various poses, some of which are the same as some poses of boy sticker 662. For example, heart-eyed robot sticker 690-1 has the same pose as heart-eyed boy sticker 662-2 (e.g., both stickers include a smiling facial expression with hearts on the eyes). It should be noted that although robot sticker 690-1 and boy sticker 662-2 have different appearances (e.g., robot sticker 690-1 has a robot appearance, while boy sticker 662-2 has a boy appearance), they also have the same poses. Furthermore, in some embodiments, stickers based on predefined avatars or avatars of non-human characters optionally exclude certain poses included in stickers based on user-created avatars or avatars of human characters. For example, robot sticker 690 does not include the thumbs-up pose. In some embodiments, the excluded sticker poses are those that include human features other than the head (e.g., hands).

[0262] In some implementations, multiple sets of stickers based on predefined avatars or avatars of non-human characters all have stickers with the same or similar poses. For example, all such avatars include heart-shaped eye stickers and exclude thumbs-up stickers. In some implementations, certain stickers have the same pose for different sticker sets, but are customized for the specific avatar on which the sticker's appearance is based. For example, robot sticker 690 includes afro robot sticker 690-2, which resembles the corresponding pose in other sticker sets (e.g., Figure 6E The image shows sticker 624-1 of a monkey with an afro, and... Figure 6T An afro sticker for a boy's avatar is shown, but with a custom appearance corresponding to the characteristics of the avatar on which the sticker's appearance is based. For example, the afro robot sticker 690-2 includes an appearance showing mechanical parts 692 (such as teeth, bolts, and springs springing out of the robot's head). In some embodiments, other avatars of similar types (e.g., predefined, non-human character-based) may include similar poses, but with different custom appearances specifically based on the characteristics of that avatar (e.g., an afro unicorn sticker with glitter).

[0263] In some implementations, stickers based on user-created avatars or avatars of non-human characters all have stickers with the same pose. For example, the sticker pose shown for round hat sticker 642 is the same as the sticker pose shown for boy sticker 662 (e.g., the pose is the same, but the appearance is different based on different avatars).

[0264] exist Figure 6U In the process, device 600 detects input 694 on the round cap representation 622-2, and in response, displays the selected round cap representation 622-2 in sticker area 618 and displays the round cap sticker 642 that replaces the robot sticker 690 in sticker area 620, as shown. Figure 6V As shown. Round hat sticker 642 includes the same pose as boy sticker 662, including a thumbs-up round hat sticker 642-1 corresponding to thumbs-up boy sticker 662-1 and a heart-shaped eye round hat sticker 642-2 corresponding to heart-shaped eye boy sticker 662-2.

[0265] Figure 7 This is a flowchart illustrating a method for displaying an avatar in a sticker application user interface using an electronic device, according to some embodiments. Method 700 is performed on a device (e.g., 100, 300, 500, 600) having a display and input devices. Some operations in method 700 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0266] As described below, method 700 provides an intuitive way to display avatars in a sticker application user interface. This method reduces the cognitive burden on users when displaying avatars in a sticker application user interface, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to display avatars in the sticker application user interface more quickly and efficiently saves power and increases the interval between battery charges.

[0267] An electronic device (e.g., 600) receives (702) a request (e.g., 615) via one or more input devices (e.g., 601) to display a sticker user interface (e.g., 615) (e.g., a single user interface for displaying stickers that can be selected for use in an application such as a messaging application) (e.g., selection of a user interface object (e.g., a display representation) associated with the sticker user interface) (e.g., selection of a representation of a set of stickers) (e.g., a gesture on a set of stickers to display a representation of multiple sets of stickers).

[0268] In response to a request to display a sticker user interface, an electronic device (e.g., 600) displays (704) a sticker user interface (e.g., 615) via a display device (e.g., 601), the sticker user interface including representations of multiple sets of stickers based on user-created avatars (e.g., avatars that can be created and optionally customized by the user) (e.g., 622). In some embodiments, user-created avatars include customizable (e.g., optional or configurable) avatar features. In some embodiments, user-created avatars include avatars modeled as representing human characters, and customizable avatar features generally correspond to human physical characteristics. For example, such avatars may include representations of people with various body types, human features, or characteristics (e.g., an elderly woman with dark skin and long, straight brown hair). Such avatars will also include representations of people with various non-human characteristics generally associated with human appearance (e.g., cosmetic enhancements, hats, glasses, etc.). User-created avatars are not included in avatars generated without input from the user selecting avatar features.

[0269] In some embodiments, representations of multiple sets of stickers based on user-created avatars are displayed in a first area of ​​the user interface (e.g., 618) (e.g., a sticker wheel). In some embodiments, the first area also includes one or more representations (e.g., 622-1) of multiple sets of stickers based on avatars that are not user-created (e.g., avatars that cannot be created or customized by the user). In some embodiments, the sticker wheel is scrollable (e.g., horizontally) (e.g., in response to gestures such as swipe gestures) to display additional representations of the multiple sets of stickers and other options displayed in the sticker wheel. In some embodiments, avatars that cannot be created or customized by the user include avatars modeled as representing non-human characters. In some embodiments, avatars modeled as representing non-human characters include, for example, anthropomorphically constructed non-human characters (e.g., stylized animals, stylized robots, or stylizations of generally inanimate or generally non-human objects). In some embodiments, such avatars include avatars with customizable (e.g., optional or configurable) avatar features that generally correspond to non-human characteristics and features. In some implementations, such avatars will not include representations of people with various body types, human features, or characteristics (e.g., a toddler with a round face and short, wavy hair), even if some of the customizable features of the human avatar include non-human characteristics (e.g., cosmetic enhancements, hats, glasses, or other non-human objects typically associated with human appearance). Displaying a first area with one or more representations of multiple sets of stickers based on avatars not created by the user reduces the amount of input required to perform the task of locating and selecting stickers displayed in the application. Reducing the amount of input required to perform the task enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0270] In some implementations, the first area also includes a creation user interface object (e.g., 626) (e.g., creation power indicator), which, when selected, displays a user interface (e.g., 632) for creating a user-created avatar (e.g., a new user-created avatar). Displaying such a creation user interface object reduces the amount of input required to access the user interface to perform the technical task of generating the avatar. Reducing the amount of input required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0271] In some implementations, after detecting the generation of a new user-created avatar, the electronic device displays a representation of a set of stickers based on the new user-created avatar in a first area (e.g., 622-4).

[0272] In some embodiments, displaying the sticker user interface (e.g., 615) further includes displaying a sticker user interface having a first area (e.g., 618) based on determining that the request to display the sticker user interface is the first received request to display the sticker user interface (e.g., 616) (e.g., the electronic device first receives the request to display the sticker user interface). In some embodiments, displaying the sticker user interface further includes displaying a sticker user interface without the first area (e.g., see [link to relevant documentation]) based on determining that the request to display the sticker user interface is a subsequent received request to display the sticker user interface (e.g., 656) (e.g., the electronic device is not receiving the request to display the sticker user interface for the first time). Figure 6L (The sticker user interface 615).

[0273] In some implementations, the user interface for displaying stickers that do not have a first area (e.g., see...) Figure 6L When the sticker user interface 615 (e.g., the sticker user interface is displayed to show stickers, but not a representation of multiple sets of stickers) is activated, the electronic device receives a first input (e.g., 658). In some embodiments, in response to detecting the first input, based on determining that the first input meets a first set of criteria (e.g., the input includes downward movement and originates from the displayed stickers), the electronic device displays a first area (e.g., updates the sticker user interface to include the first area) (e.g., see [link to relevant documentation]). Figure 6M ).

[0274] In some implementations, a set of stickers based on a first avatar (e.g., round hat avatar 636) has a first set of sticker poses (e.g., the pose shown in sticker 642), and a set of stickers based on a second avatar has a first set of sticker poses (e.g., the pose shown in sticker 662) (e.g., all sets of stickers based on user-created avatars have the same pose and facial expression, but have different appearances based on the specific user-created avatar on which each set of stickers is based). In some implementations, in response to detecting the selection of a corresponding set of representations of multiple sets of stickers based on a user-created avatar, the corresponding set of stickers from the multiple sets of stickers based on the user-created avatar is displayed. Displaying a set of stickers in response to detecting the selection of a representation of a set of stickers based on a user-created avatar reduces the amount of input required to perform the technical task of generating stickers. Reducing the amount of input required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0275] In some embodiments, the sticker user interface (e.g., 615) also includes a representation (e.g., 662-5) of a set of stickers based on a first predefined avatar (e.g., a robot avatar) (e.g., a predefined or pre-existing avatar, an avatar not created by the user, or an avatar that cannot be created by the user). In some embodiments, avatars that cannot be created or customized by the user include avatars modeled to represent non-human characters. In some embodiments, avatars modeled to represent non-human characters include, for example, anthropomorphically constructed non-human characters (e.g., stylized animals, stylized robots, or stylized generally inanimate or generally non-human objects). In some embodiments, such avatars include avatars with customizable (e.g., optional or configurable) avatar features that generally correspond to non-human characteristics and features. In some implementations, the avatar will not include representations of people with various body types, human features, or characteristics (e.g., a toddler with a round face and short, wavy hair), even if some of the customizable features of the human avatar include non-human characteristics (e.g., cosmetic enhancements, hats, glasses, or other non-human objects typically associated with human appearance). In some implementations, the avatar is based on a set of stickers (e.g., 690) with poses different from the first set of stickers (e.g., see [link]). Figure 6T The second set of sticker poses in sticker 662 (see, for example, sticker 662) Figure 6U(e.g., the set of poses and facial expressions for multiple sets of stickers used for a predefined avatar differs from the set of poses and facial expressions for multiple sets of stickers used for a user-created avatar). In some embodiments, a subset of poses and facial expressions for multiple sets of stickers used for a predefined avatar is the same as a subset of poses and facial expressions for multiple sets of stickers used for a user-created avatar (e.g., some sticker poses and facial expressions are common to both the multiple sets of stickers used for a predefined avatar and the multiple sets of stickers used for a user-created avatar). In some embodiments, a subset of poses and facial expressions for a corresponding set of stickers used for a predefined avatar is used to be the same as a subset of poses and facial expressions for other sets of stickers used for the predefined avatar (e.g., some sticker poses and facial expressions are common to different sets of stickers used for the predefined avatar). In some embodiments, in response to detecting a selection of a representation of a set of stickers based on a first non-user-created avatar, a set of stickers based on the first non-user-created avatar is displayed.

[0276] In some implementations, a set of stickers (e.g., 690) based on a first predefined avatar (e.g., an avatar representing an animated character not created by the user, such as a unicorn avatar) includes stickers with a first sticker pose (e.g., afro robot sticker 690-2) (e.g., stickers depicting an afro head pose / expression). In some implementations, the sticker display user interface also includes a representation of a set of stickers based on a second predefined avatar (e.g., see...). Figure 6C (e.g., stickers 624) (e.g., avatars representing animated characters not created by the user, such as robot avatars, avatars that cannot be created or customized by the user). In some embodiments, avatars that cannot be created or customized by the user include avatars modeled to represent non-human characters. In some embodiments, avatars modeled to represent non-human characters include, for example, anthropomorphic non-human characters (e.g., stylized animals, stylized robots, or stylizations of generally inanimate or generally non-human objects). In some embodiments, such avatars include avatars with customizable (e.g., optional or configurable) avatar features that generally correspond to non-human characteristics and features. In some embodiments, such avatars will not include representations of people with various body types, human features, or characteristics (e.g., a toddler with a round face and short, wavy hair), even if some of the customizable features of the human avatar include non-human features (e.g., cosmetic enhancements, hats, glasses, or other non-human objects that are generally associated with human appearance). In some embodiments, a set of stickers based on a second predefined avatar includes stickers with the pose of the first sticker (e.g., see [link to documentation]). Figure 6C(e.g., the afro monkey sticker 624-1). In some embodiments, the sticker having a first sticker pose for a first predefined head image includes graphic elements (e.g., mechanical parts 692) (e.g., glitter or teeth) corresponding to the first predefined head image, which are not included in the sticker having a first sticker pose for a second predefined head image. In some embodiments, multiple sets of stickers for a predefined head image include stickers unique to a set of stickers and incorporating a set of characteristics unique to the predefined head image. For example, regarding the "afro" unicorn sticker, the sticker has the following appearance: the top portion of the unicorn head is removed and an afro state of the unicorn brain is displayed, and the unicorn head includes glitter emanating from the unicorn brain. The appearance of the glitter in the "afro" sticker is unique to the unicorn sticker and corresponds to the mysterious characteristics of the unicorn head image. As another example, regarding the "afro" robot sticker, the sticker has the following appearance: the top of the robot head is removed and an afro state of the robot brain is displayed, and the robot head includes teeth ejected from the robot brain. The appearance of the teeth in the "Afro" sticker is unique to robot stickers and corresponds to the mechanical characteristics of the robot head.

[0277] In some embodiments, the first set of sticker poses includes at least one sticker pose not included in the second set of sticker poses (e.g., thumbs-up sticker 662-1) (e.g., sticker poses including a hand). In some embodiments, excluded sticker poses include poses showing body parts other than the head. Stickers with such poses may include, for example, "thumbs-up" stickers, "fist bump" stickers, "hug" stickers, etc.

[0278] In some embodiments, the sticker user interface further includes a keyboard display area (e.g., 605) that includes multiple emojis (e.g., 609) and representations of multiple sets of stickers based on a user-created avatar (e.g., sticker area 610). In some embodiments, the electronic device detects one representation of the multiple sets of stickers based on the user-created avatar (e.g., Figure 6KThe selection of 614-2) (e.g., 656). In some embodiments, in response to detecting the selection of one representation from a set of stickers based on a user-created avatar, the electronic device displays multiple stickers from a set of stickers based on the user-created avatar in the keyboard display area (e.g., 642). In some embodiments, displaying multiple stickers in the keyboard display area includes stopping the display of emojis. In some embodiments, when multiple stickers are displayed, the representations of the multiple sets of stickers are displayed in different locations in the keyboard display area. Displaying multiple stickers from a set of stickers based on a user-created avatar in the keyboard display area in response to detecting the selection of one representation from a set of stickers based on the user-created avatar reduces the amount of input required to perform the technical task of generating stickers for transmission in a messaging application (e.g., by reducing the number of menu options required to locate and select the desired sticker). Reducing the amount of input required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0279] The display (704) sticker user interface (e.g., 615) includes, based on the determination that the user has created a first set of two or more user-created avatars, including a first avatar (e.g., a round hat avatar 636) (e.g., an avatar created by the user to model a male) and a second avatar (e.g., an avatar created by the user to model a female), display (706) (e.g., simultaneously display) a representation (e.g., 622) of the first multiple sets of stickers (e.g., a static representation of the avatars on which the stickers of the corresponding set are based, such as a representation of user-created avatars). The first multi-set sticker representation includes a representation of a set of stickers based on a first avatar (e.g., 622-2) (e.g., a static representation of a first avatar having a static pose and an appearance based on the first avatar including features used to create the first avatar (e.g., hat, sunglasses, hairstyle / color, skin tone, etc.)) and a representation of a set of stickers based on a second avatar (e.g., 622-3) (e.g., a static representation of a second avatar having a static pose and an appearance based on the second avatar including features used to create the second avatar (e.g., hat, sunglasses, hairstyle / color, skin tone, etc.)). Displaying a multi-set sticker representation when one set of avatars has already been created reduces the amount of input required to perform the technical task of generating and selecting stickers for transmission in a messaging application (e.g., by reducing the number of menu options required to locate and select the desired sticker). Reducing the amount of input required to perform the task enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0280] The display (704) sticker user interface (e.g., 615) includes, based on the determination that a user has created a second group of two or more user-created avatars, including a third avatar not included in the first group of two or more user-created avatars (e.g., boy avatar 672) (e.g., an avatar created by a user to model a child), displaying (708) a representation of a second group of stickers that differs from the representation of the first group of stickers. In some embodiments, the representation of the second group of stickers includes a representation of a group of stickers based on the third avatar not included in the representation of the first group of stickers (e.g., 622-4) (e.g., a static representation of a third avatar having a static pose and an appearance based on the third avatar including features used to create the third avatar (e.g., hat, sunglasses, hairstyle / color, skin tone, etc.). In some embodiments, in response to the detection of the generation of a corresponding avatar (e.g., the first, second, and third avatars), (e.g., automatically, e.g., after no subsequent user input following the creation of the avatar)) a group of stickers based on the corresponding avatar and a corresponding representation of that group of stickers is generated. In some implementations, the representation of the first and / or second sets of stickers includes a representation of an avatar that is not user-created. Displaying different representations of multiple sets of stickers when a new avatar has already been created reduces the amount of input required to perform the technical task of generating and selecting stickers for sending in a messaging application (e.g., by reducing the number of menu options required to locate and select the desired sticker). Reducing the amount of input required to perform the task enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0281] In some implementations, the electronic device detects representations of a set of stickers based on a first avatar (e.g., Figure 6UThe selection of a representation (e.g., 694) is represented by a round cap (e.g., 622-2) (e.g., selection of one of a set of stickers based on a first avatar, a second avatar, or a third avatar). In some embodiments, a representation is selected by default (e.g., when displaying the sticker user interface). In some embodiments, a representation is selected in response to user input. In some embodiments, in response to detecting a selection of a representation of a set of stickers based on a first avatar, the electronic device simultaneously displays the selected representation (e.g., 622-2) and multiple stickers (e.g., 642) from the set of stickers based on the first avatar (e.g., graphical objects having appearances based on an avatar (e.g., a user-created avatar) and having different poses and facial expressions), the multiple stickers having an appearance based on the first avatar. In some embodiments, a sticker corresponds to the selected representation (e.g., a sticker having an appearance based on the avatar associated with the selected representation). When a different representation is selected, the stickers associated with the previously selected representation are replaced with a set of stickers associated with the newly selected representation. In some embodiments, the stickers include additional features that are displayed to modify the appearance of the avatar in order to convey specific expressions, emotions, feelings, etc. For example, stickers could include a heart above the eyes of an avatar to convey love or tears below the eyes to convey sadness. In some implementations, stickers include minor modifications to the appearance of the avatar, such as altering a portion of the avatar while still maintaining the overall recognizable representation of the avatar. An example of such a modification is an "afro" sticker, where the sticker represents an avatar with the top portion of the head removed, displaying an image of the avatar's brain in an "exploded" state.

[0282] In some embodiments, a set of stickers based on a first avatar includes a first sticker (e.g., 642-1) having a first pose (e.g., a thumbs-up pose) and an appearance based on the first avatar, and a second sticker (e.g., 642-2) having a second pose different from the first pose (e.g., heart-shaped eyes) and an appearance based on the first avatar (e.g., round hat avatar 636) (e.g., the stickers in this set have a set of different poses and appearances based on the first avatar). In some embodiments, while displaying a set of stickers based on a first avatar, the electronic device detects a selection of a representation of a set of stickers based on a second avatar (e.g., 622-4). In some embodiments, in response to detecting a selection of a representation of a set of stickers based on a second avatar, the electronic device stops displaying a set of stickers based on a first avatar. In some embodiments, in response to detecting a selection of a representation of a set of stickers based on a second avatar, the electronic device displays a set of stickers based on a second avatar (e.g., 662). In some implementations, a set of stickers based on a second avatar includes a third sticker (e.g., 662-1) with a first pose (e.g., a thumbs-up pose) and an appearance based on the second avatar (e.g., boy avatar 672), and a fourth sticker (e.g., 662-2) with a second pose (e.g., heart-shaped eyes) and an appearance based on the second avatar (e.g., the first set of stickers with the appearance of the first avatar and a set of poses is replaced with a second set of stickers with the same set of poses but with an appearance based on the second avatar). Displaying stickers with the same set of poses but different appearances based on the corresponding avatars (e.g., user-created avatars) allows users to quickly and easily compose messages to express known emotions based on poses, while still respecting the user's personal and artistic preferences for avatar stickers with different appearances. This provides an improved control scheme for generating custom messages that may require less input than using a different control scheme (e.g., a control scheme that requires generating separate, custom poses). Furthermore, this type of control can be completed in real time, for example, during a conversation such as a text conversation or a video conversation, whereas manual control to build stickers would have to be completed before or after the conversation begins. Reducing the amount of input required to perform a task enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0283] In some implementations, one or more of the stickers are animated (e.g., the stickers are shown blinking, waving, making facial expressions, etc.).

[0284] In some embodiments, after displaying the sticker user interface, the electronic device receives a request to redisplay the sticker user interface (e.g., 638). In some embodiments, in response to receiving the request to redisplay the sticker user interface, the electronic device redisplays the sticker user interface via the display device (e.g., the sticker user interface 615 is redisplayed). Figure 6H (in Chinese). In some implementations, redisplaying the sticker user interface includes, based on determining that the user has created a fourth avatar not included in the avatars created by two or more users in the first or second group (e.g., 636) (e.g., the user has created a new avatar), (e.g., simultaneously) displaying a representation of a third multiple group of stickers (e.g., ...). Figure 6H The representation shown is 622). In some embodiments, the representation of the third set of stickers includes a representation of a set of stickers based on a fourth avatar not included in the representation of the first or second set of stickers (e.g., 622-2). (For example, again, after receiving a series of inputs to create a boy avatar 672, the sticker user interface 615 is redisplayed.) Figure 6N This includes newly displayed boy representations 622-4 and boy stickers 662. In some embodiments, displaying a representation of a set of stickers includes simultaneously displaying at least a portion of that set of stickers based on a fourth avatar (e.g., having an appearance based on the fourth avatar). For example, when redisplaying the sticker UI, a representation of a set of stickers based on a fourth avatar is selected, and at least a portion of the stickers in that set are displayed with different appearances based on the fourth avatar. Redisplaying the sticker user interface to display stickers after the user has created an avatar reduces the amount of input required to perform the technical task of generating stickers (e.g., for sending in a messaging conversation). Reducing the amount of input required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0285] In some embodiments, the representation of a set of stickers based on a first avatar (e.g., 622-2) has the appearance of one sticker from that set of stickers based on the first avatar. In some embodiments, the representation of a set of stickers based on a second avatar (e.g., 622-4) has the appearance of one sticker from that set of stickers based on the second avatar. In some embodiments, the representation of a set of stickers based on a third avatar has the appearance of one sticker from that set of stickers based on the third avatar.

[0286] In some implementations, the display sticker user interface also includes a display editing user interface object (e.g., 665) (e.g., an editing display), which, when selected, displays an editing interface for editing the corresponding user-created avatar (e.g., 670).

[0287] In some implementations, the sticker user interface further includes displaying a plurality of stickers (e.g., 662) from a set of stickers based on a corresponding user-created avatar (e.g., 672), wherein the plurality of stickers have an appearance based on a first appearance of the corresponding user-created avatar (e.g., Figure 6O The boy sticker 662). In some embodiments, the display sticker user interface further includes detecting a series of inputs (e.g., 668, 676, 678) corresponding to a request to edit the corresponding user-created avatar from a first appearance to a second appearance (e.g., from a first hairstyle to a second hairstyle) (e.g., including selection of an editable representation and a series of inputs to interact with the editing interface to edit the corresponding user-created avatar). In some embodiments, the display sticker user interface further includes detecting a request to display multiple stickers from a set of stickers based on the corresponding user-created avatar (e.g., 678) (e.g., exiting the editing interface) (e.g., detecting selection of a representation of a set of stickers based on the corresponding user-created avatar). In some embodiments, the display sticker user interface further includes, in response to detecting a request to display multiple stickers from a set of stickers based on the corresponding user-created avatar, displaying multiple stickers from that set of stickers based on the corresponding user-created avatar (e.g., Figure 6R (Sticker 662 in the image). In some implementations, a sticker in a set of stickers has an updated appearance based on a second appearance of the corresponding user-created avatar (e.g., sticker 662 in the image). Figure 6R (e.g., when the corresponding user-created avatar is changed / updated, the stickers in that set of stickers based on the corresponding user-created avatar are changed / updated). In some embodiments, when the corresponding user-created avatar is changed / updated, the appearance of the representation of a set of stickers based on the corresponding user-created avatar is updated. Automatically updating the appearance of the sticker representation after the user has created / updated the avatar reduces the amount of input required to perform the technical task of generating the stickers. Reducing the amount of input required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0288] In some implementations, an editing interface (e.g., 670) is used to edit the appearance of the corresponding user-created avatar in the sticker user interface and other user interfaces (e.g., the editing interface changes the appearance of the corresponding user-created avatar throughout the operating system, including instances where the corresponding user-created avatar is displayed in different applications such as camera applications, video communication applications, messaging applications, media display applications, etc.). For example, in camera applications and video communication applications, a representation of the corresponding user-created avatar may be displayed in the camera's field of view. When the appearance of the corresponding user-created avatar is changed in the sticker user interface, the change to the appearance of the corresponding user-created avatar is also applied to the representation of the corresponding user-created avatar in the camera application and video communication application. In a messaging application, a corresponding user-created avatar may be used to represent participants in a messaging conversation. When the appearance of the corresponding user-created avatar is changed in the sticker user interface, the change to the appearance of the corresponding user-created avatar is also applied to the corresponding user-created avatar in the messaging conversation of the messaging application. Media display applications such as photo viewing applications or video viewing applications may include a representation of the corresponding user-created avatar in a media item. When the appearance of a corresponding user-created avatar changes in the sticker user interface, this change is also applied to the representation of the corresponding user-created avatar in the media item viewed in the media display application. Updating the appearance of avatars across various user interfaces in response to detecting changes in the sticker user interface reduces the amount of input required to perform the technical task of generating or updating avatars for different applications. Reducing the amount of input required to perform this task enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0289] It should be noted that the above description is in contrast to the process described in method 700 (e.g., Figure 7 The details of the above also apply in a similar manner to the methods described below. For example, methods 800, 1000, 1200, 1300, 1500, 1700, and 1800 optionally include one or more features of the various methods described above with reference to method 700. For example, stickers can be displayed and used in a manner similar to that described above. For the sake of brevity, these details will not be repeated below.

[0290] Figure 8This is a flowchart illustrating a method for displaying an avatar in an avatar keyboard application user interface using an electronic device, according to some embodiments. Method 800 is performed on a device (e.g., 100, 300, 500, 600) having a display and one or more input devices. Some operations in method 800 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0291] As described below, method 800 provides an intuitive way to display avatars in an avatar keyboard application user interface. This method reduces the cognitive burden on users when displaying avatars in the avatar keyboard application user interface, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to display avatars more quickly and efficiently in the avatar keyboard application user interface saves power and increases the interval between battery charges.

[0292] An electronic device (e.g., 600) displays (802) content via the display device (e.g., 601) to create a user interface (e.g., 603) (e.g., a document creation user interface or a message writing user interface) (e.g., a single interface screen).

[0293] When displaying the content creation user interface (e.g., 603), the electronic device (e.g., 600) receives (804) a request (e.g., 604) via one or more input devices to display a first display area (e.g., 605) (e.g., a keyboard display area) (e.g., an emoji keyboard display area) (e.g., a sticker keyboard display area). In some embodiments, the first display area includes a plurality of graphical objects (e.g., 612, 609) (e.g., stickers) (e.g., emojis) corresponding to predefined content to be inserted into the content creation user interface.

[0294] Displaying (804) a first display area (e.g., 605) in response to receiving the request includes: displaying (806) via the display device (e.g., 601) a first display area comprising a first subgroup (e.g., 612) of graphic objects (e.g., stickers) (e.g., a subgroup of multiple graphic objects in the first display area), the first subgroup of graphic objects having an appearance based on a set of avatars (e.g., avatars such as avatars modeled to represent human characters, avatars modeled to represent non-human characters, avatars that can be created and / or customized by the user, and avatars that cannot be created or customized by the user) available at the electronic device (e.g., 600) (e.g., stickers having an appearance based on the respective avatar and having different poses and facial expressions). In some embodiments, the sticker includes additional features that are displayed to modify the respective avatar in order to convey a specific expression, emotion, feeling, etc. For example, the sticker may include a heart above the avatar's eyes to convey love or tears below the avatar's eyes to convey sadness. In some implementations, the stickers include minor modifications to the appearance of the avatar, such as altering a portion of the avatar while still maintaining the overall recognizable representation of the avatar. An example of such a modification is an "afro" sticker, where the sticker represents an avatar with the top portion of the head removed, displaying an "exploded" state of the avatar's brain. In some implementations, avatars modeled to represent humans include customizable (e.g., optional or configurable) avatar features that generally correspond to human physical characteristics. For example, such avatars may include representations of people with various body types, human features, or characteristics (e.g., an older woman with dark skin and long, straight brown hair). Such avatars will also include representations of people with various non-human characteristics generally associated with human appearance (e.g., cosmetic enhancements, hats, glasses, etc.). In some implementations, such avatars will not include anthropomorphic constructs such as stylized animals, segmented robots, or stylized objects that are generally inanimate or generally non-human. In some embodiments, avatars modeled to represent non-human characters include, for example, anthropomorphic constructs of non-human characters (e.g., stylized animals, stylized robots, or stylizations of generally inanimate or generally non-human objects). In some embodiments, such avatars include avatars with customizable (e.g., optional or configurable) avatar features that generally correspond to non-human characteristics and features. In some embodiments, such avatars will not include representations of people with various physical, human features or characteristics (e.g., a toddler with a round face and short, wavy hair), even if some of the customizable features of the human avatar include non-human features (e.g., cosmetic enhancements, hats, glasses, or other non-human objects that are generally associated with human appearance).

[0295] The display (806) includes a first display area (e.g., 610) comprising a first subgroup (e.g., 610) of graphical objects (e.g., 612) having an appearance based on a set of avatars available at the electronic device. This includes: based on determining that the set of avatars includes a first type of avatar (e.g., avatars that can be created and / or customized by the user of the electronic device) (e.g., avatars modeled as representing human characters), displaying (808) graphical objects (e.g., 610) in the first subgroup having an appearance based on that first type of avatar (e.g., round hat avatar 636). Figure 6I In some embodiments, when the set of avatars includes a customizable, createable, and / or modeled avatar representing a human character, the displayed stickers include one or more stickers derived from the avatar (e.g., having an avatar-based appearance). In some embodiments, these stickers are referred to as first-type stickers. In some embodiments, first-type stickers include stickers suggested by the electronic device based on the usage history of first-type stickers (e.g., suggesting recently and / or frequently used stickers).

[0296] The display (806) includes a first display area (e.g., 605) comprising a first subgroup of graphical objects (e.g., 612) having an appearance based on a set of avatars available at the electronic device. This includes displaying (810) graphical objects (e.g., 612-1, 612-2, 612-3) in the first subgroup having an appearance based on a second type of avatar (e.g., avatars that cannot be created and / or customized by the user of the electronic device) (e.g., avatars modeled to represent non-human characters), without displaying one (e.g., one or more) of the graphical objects in the first subgroup having an appearance based on avatars of the first type. In some embodiments, when the set of avatars does not include customizable, createable, and / or modeled to represent human avatars, the displayed stickers are derived from non-human characters and / or avatars that are not user-created or customizable (e.g., having an avatar-based appearance). In some embodiments, these stickers are referred to as second-type stickers. When the first type of avatar is unavailable, displaying only the second type of avatar provides feedback to the user that the first type of avatar is not currently available on the device, and encourages the user to create a first type of avatar. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0297] In some implementations, the first type of avatar is a user-created avatar (e.g., a round hat avatar 636, a boy avatar 672) (e.g., an avatar that can be created and optionally customized by the user), and the second type of avatar is a predefined avatar (e.g., a robot, a monkey, poop) (e.g., an avatar of a type other than a user-created avatar) (e.g., an avatar that cannot be created and / or customized by the user of the electronic device). In some implementations, user-created avatars include customizable (e.g., optional or configurable) avatar features.

[0298] In some implementations, the first type of avatar is based on human characters (e.g., round-hat avatar 636, boy avatar 672) (e.g., modeled to represent human avatars), and the second type of avatar is based on non-human characters (e.g., monkeys, poop, robots) (e.g., modeled to represent avatars of animals such as pigs, cats, dogs, sharks, etc.; mystical characters such as unicorns, dragons, and aliens; anthropomorphic objects such as robots and poop; and / or stylized expressions such as emojis). In some implementations, avatars modeled to represent non-human characters include, for example, anthropomorphic constructs of non-human characters (e.g., stylized animals, stylized robots, or stylizations of generally inanimate or generally non-human objects). In some implementations, such avatars include avatars with customizable (e.g., optional or configurable) avatar features that generally correspond to non-human characteristics and features. In some implementations, such an avatar will not include representations of people with various body types, human features, or characteristics (e.g., a toddler with a round face and short, wavy hair), even if some of the customizable features of the human avatar include non-human characteristics (e.g., cosmetic enhancements, hats, glasses, or other non-human objects that are typically associated with human appearance).

[0299] In some embodiments, displaying the first display area further includes displaying a sticker user interface object (e.g., 614) (e.g., a sticker display). In some embodiments, the electronic device receives input pointing to the sticker user interface object (e.g., 616, 656). In some embodiments, in response to receiving input pointing to the sticker user interface object, the electronic device stops displaying the first display area (e.g., 605). In some embodiments, in response to receiving input pointing to the sticker user interface object, the electronic device displays the sticker user interface (e.g., 615) (e.g., displaying a user interface that can be selected for a sticker in the content creation user interface). In some embodiments, the sticker user interface includes a second plurality of graphical objects (e.g., stickers 624, stickers 642) corresponding to predefined content to be inserted into the content creation user interface. In some embodiments, the sticker user interface is displayed simultaneously with a portion of the content creation user interface (e.g., a message display area). In some embodiments, the sticker user interface replaces the sticker keyboard display area. Displaying a set of graphical objects corresponding to predefined content to be inserted into the content creation user interface allows users to quickly and easily compose messages to express known feelings based on the predefined content. This provides an improved control scheme for generating messages, which may require less input than using a different control scheme (e.g., a control scheme that needs to generate separate, custom content). Furthermore, this type of control can be completed in real-time during, for example, a dialogue such as a text conversation or a video conversation, whereas manual control to construct graphical objects would have to be done before or after the conversation begins.

[0300] In some implementations, displaying a sticker user interface object includes displaying a sticker user interface object having a first appearance (e.g., Figure 6B In 614), the sticker user interface object includes multiple representations of a first type of avatar (e.g., 614-1, 614-2) (e.g., the sticker display representation includes a representation of the sticker based on the appearance of the first type of avatar). In some embodiments, the display representation also includes one or more representations of a second type of avatar. Displaying multiple representations of the first type of avatar in the sticker user interface object provides the user with a selection of the sticker user interface object, allowing the user to access feedback on the stickers of the first type of avatar. Providing improved feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0301] In some implementations, the display has a first appearance (e.g., Figure 6BAfter the sticker user interface object (e.g., 614) is created, the electronic device receives a series of inputs corresponding to a request to create a third avatar of the first type (e.g., 636). In some embodiments, the electronic device receives a request to redisplay the first display area (e.g., 644). In some embodiments, in response to receiving the request to redisplay the first display area, the electronic device displays an image with a second appearance (e.g., Figure 6I The sticker user interface object (e.g., 614) includes a representation of a third avatar of the first type (e.g., 614-2) (e.g., redisplaying a keyboard display area with the sticker user interface object, which includes a representation of a sticker with an appearance based on the created first type avatar). In some embodiments, the sticker user interface object also includes one or more representations of the first type avatar. In some embodiments, the sticker user interface object also includes one or more representations of a second type avatar. Redisplaying the sticker user interface object with a different appearance updated to reflect the newly created avatar provides the user with feedback that creating an additional avatar will update the sticker user interface object to display the additional avatar. Providing improved feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0302] In some embodiments, displaying a sticker user interface object with a first appearance (e.g., 614) includes an animated sequence of displaying representations of a first type of avatar and representations of a second type of avatar. In some embodiments, the animation includes showing a representation of the first sticker on a sticker user interface object with animation (e.g., laughing, moving, etc.), and then replacing the representation of the first sticker with an animated representation of a second sticker, etc. In some embodiments, the animated representation of the sticker includes representations of both the first and second types of stickers. Displaying a sticker user interface object with animated representations of avatars that are displayed cyclically provides feedback to the user that no such avatar is available on the device and informs the user that the selection of the sticker user interface object will allow the user to create such an avatar. Providing improved feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0303] In some embodiments, the sticker user interface (e.g., 615) further includes displaying a creation user interface object (e.g., 626) (e.g., creating a representation) based on determining that the set of avatars does not include any avatars of the first type. This creation user interface object, when selected, displays a creation user interface (e.g., 632) for creating avatars of the first type. In some embodiments, when the set of avatars does not include any avatars of the first type, multiple representations of multiple stickers for second type avatars (e.g., ...) are used. Figure 6C The representation 622 (e.g., 622-1, 622-5) is displayed in the sticker user interface. In some embodiments, one of the representations of multiple sets of stickers for the second type of avatar is displayed as having a selected state, and a set of stickers of the second type corresponding to the selected representation (e.g., 624, 690) is displayed.

[0304] In some implementations, displaying the first display area may also include displaying multiple emojis (e.g., 609) (e.g., predefined emojis in a predefined category or selected predefined emojis based on previous user activity (e.g., recently used / frequently used emojis)).

[0305] In some embodiments, after displaying a graphical object in the first subgroup having an appearance based on a second type of avatar different from the first type, but not displaying one of the graphical objects in the first subgroup having an appearance based on the first type of avatar, the electronic device receives a series of inputs (e.g., 630, 634, 638) corresponding to a request to create a first avatar of the first type (e.g., detecting the creation of a first type of avatar (e.g., a user-created avatar)). In some embodiments, in response to receiving a series of inputs, the electronic device creates a first avatar of the first type (e.g., 636) and adds the first avatar to the group of avatars. In some embodiments, after creating the first type of avatar, the electronic device receives a request to redisplay a first display area (e.g., 605) (e.g., 644). In some embodiments, in response to receiving a request to redisplay the first display area, the electronic device displays a first display area having a first subgroup of graphical objects (e.g., 612). In some embodiments, the first subgroup of graphical objects includes a first graphical object having an appearance based on a first type of first avatar (e.g., 612). Figure 6I(e.g., redisplaying the keyboard, and the subgroups of graphical objects now include stickers with an appearance based on the newly created avatar). In some embodiments, stickers based on a first type of avatar (e.g., the newly created avatar) are given higher priority than stickers of a second type. In some embodiments, the keyboard display area includes stickers suggested by the electronic device based on the sticker priority. Displaying the first display area updated with stickers for the newly created avatar provides feedback, informing the user that the sticker can be selected for use in a message. Additionally, this reduces the amount of input required to perform the technical task of generating stickers for the newly created avatar. Providing improved feedback and reducing the amount of input required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling users to use the device more quickly and efficiently.

[0306] In some embodiments, after displaying a graphic object (e.g., 612) in the first subgroup having an appearance based on a second type of avatar different from the first type, the electronic device receives a series of inputs (e.g., 644) corresponding to the use (e.g., sending a sticker, creating a sticker, etc.) of a graphic object (e.g., 642-1) corresponding to the second avatar of the first type (e.g., a sticker having an appearance based on the second avatar of the first type) in a content creation user interface (e.g., in a document creation user interface or a message composition user interface). In some embodiments, after receiving a series of inputs corresponding to the use of a graphic object corresponding to the second avatar of the first type, the electronic device receives a request to redisplay the first display area. In some embodiments, in response to receiving a request to redisplay the first display area, the electronic device displays a first display area having a first subgroup of graphic objects, the first subgroup of graphic objects including graphic objects corresponding to the second avatar of the first type (e.g., 642-1) (e.g., the keyboard is redisplayed, and the subgroup of graphic objects now includes the sticker being used). In some implementations, the keyboard display area includes stickers suggested by the electronic device based on the usage history of stickers of a first type (or second type) (e.g., suggesting recently used and / or frequently used stickers). Displaying a first display area updated with previously used stickers reduces the amount of input required to locate and send stickers in subsequent communications. Reducing the amount of input required to perform tasks enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). This also reduces power consumption and extends the device's battery life by enabling users to use the device more quickly and efficiently.

[0307] In some implementations, a first instance of a graphical object (e.g., one of a set of avatars available on an electronic device) is displayed. Figure 6IWhen input 642-1 is received, the electronic device receives input of a first type (e.g., a tap gesture) pointing to a first instance of a corresponding graphic object in the graphic objects (e.g., 646). In some embodiments, in response to receiving the first type of input, the electronic device displays a second instance of the corresponding graphic object in the graphic objects (e.g., 650) (e.g., displaying a preview of the corresponding graphic object in the graphic objects without transmitting a sticker corresponding to the corresponding graphic object in the graphic objects). In some embodiments, other graphic objects (e.g., emojis) in the keyboard display area respond to the first type of input (e.g., a tap gesture). For example, an emoji can be selected using a tap gesture. Displaying a second instance of the corresponding graphic object in response to receiving the first type of input maintains consistency for interacting with the graphic objects and the emojis displayed on the keyboard. This provides an intuitive interface for interacting with the different graphical objects presented in the keyboard display area, which increases familiarity and thus enhances device operability, and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.

[0308] In some implementations, when displaying a second instance of a corresponding graphic object in a graphic object (e.g., 650), the electronic device receives a second input pointing to the second instance of the corresponding graphic object in the graphic object, wherein the second input includes a stationary first portion followed by a second portion including movement of the second input. In some implementations, in response to receiving the second input, based on determining that the second input meets a first criterion (e.g., the first portion of the second input is stationary at the position of the second instance of the corresponding graphic object in the graphic object for a threshold amount of time, and the second portion of the second input includes movement to a position corresponding to a message delivery area of ​​the content creation user interface), the electronic device sends a sticker corresponding to the corresponding graphic object in the graphic object to the receiving user. For example, in Figure 6J and Figure 6KIn this scenario, when a user touches and long-presses preview 650 (triggering device 600 to select a sticker to send), device 600 sends the round hat sticker 642-1 to the second recipient 607-2 in a messaging conversation, and then drags the contact to message area 603-1. In some implementations, in response to receiving a second input, and based on determining that the second input does not meet a first criterion, the electronic device abandons sending a sticker corresponding to the corresponding graphic object in the graphic object to the recipient user. For example, referring to the previous example, if the user does not maintain contact on preview 650 for a sufficient period to trigger device 600 to select a sticker, device 600 will not send sticker 642-1 to the second recipient 607-2 in the messaging conversation, even if the user drags the contact to message area 603-1. Discontinuing to send a sticker when the second input does not meet the first criterion reduces the possibility of accidental sticker transmission, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends the device's battery life by enabling users to use the device more quickly and efficiently.

[0309] In some embodiments, displaying a second instance of a corresponding graphic object in the graphic objects (e.g., 650) further includes displaying a sending user interface object (e.g., 652) (e.g., sending a power indicator). In some embodiments, the electronic device receives input directed to the sending user interface object (e.g., 654). In some embodiments, in response to receiving input directed to the sending user interface object, the electronic device sends a sticker corresponding to a corresponding graphic object in the graphic objects to the receiving user.

[0310] It should be noted that the above is relative to method 800 (e.g., Figure 8 The details of the process described herein also apply in a similar manner to the methods described above and below. For example, methods 700, 1000, 1200, 1300, 1500, 1700, and 1800 optionally include one or more features of the various methods described above with reference to method 800. For example, stickers can be displayed and used in a manner similar to that described above. For the sake of brevity, these details will not be repeated below.

[0311] Figures 9A to 9AG Exemplary user interfaces for displaying profile pictures in a contacts application user interface, according to some embodiments, are shown. The user interfaces in these figures are used to illustrate the following description, including... Figure 10 The process of...

[0312] Figure 9AAn electronic device 600 is shown displaying (e.g., on a monitor 601) a contacts user interface 902, which shows a list of contacts available in a contacts application on the electronic device. The contacts application is an application used to manage contactable users.

[0313] exist Figure 9A In the middle, the device detects input 904 (e.g., a tap gesture) and, in response, in Figure 9B The image shows the new contact user interface 906, used to create new contacts for the Contacts application.

[0314] like Figure 9B As shown, the new contact user interface 906 includes a data field 908 and an area 910, which optionally displays a representation of the new contact. This representation can be displayed to represent contacts in various user interfaces, such as, for example, messaging applications, contact lists, emails, or other instances that can represent contacts.

[0315] Figure 9C This displays information about the new contact entered in the Name field of the new contact user interface. For example... Figure 9C As shown, "Jane Appleseed" is entered as the corresponding first and last name of the new contact. After receiving the name information, device 600 updates area 910 to display a monogram representing the initials "JA" corresponding to the new contact's name "Jane Appleseed".

[0316] exist Figure 9C In the middle, device 600 detects input 912 on the phone number option (e.g., a tap gesture) and displays a keyboard for entering the phone number of the new contact Jane Appleseed, such as Figure 9D As shown.

[0317] In some implementations, the new contact representation can be edited by selecting area 910 from the new contact user interface 906. For example, Figure 9D Input 914 is shown on the detection area 910 of device 600.

[0318] In response to the detection of input 914 on area 910, device 600 displays a representation editing user interface 915 for modifying the appearance of the representation of the new contact (e.g., Jane Appleseed), such as Figure 9E As shown. It should be understood that in some implementations, the user interface 915 can be used to edit the appearance of the representation of existing contacts (e.g., not new contacts).

[0319] User interface 915 includes a current representation 916. Current representation 916 represents the current appearance of the contact's representation. Therefore, upon exiting user interface 915 (e.g., in response to selecting the complete power-on representation 917), the contact's representation as shown in various user interfaces of device 600 (e.g., a representation appearing in area 910 of the new contact user interface 906) will have the appearance shown in current representation 916. As discussed in detail below, current representation 916 is updated in response to a series of user inputs in user interface 915. However, in response to detecting a selection to cancel the power-on representation 918, such updates may be discarded, and the contact's representation reverts to its previous appearance (e.g., the appearance shown in area 910 before displaying user interface 915).

[0320] exist Figure 9E In the current context, 916 has the appearance of the letter combination options previously displayed in the new contact user interface 906. Figures 9F to 9AF A series of inputs for modifying the current representation 916 are shown according to various embodiments of this disclosure.

[0321] User interface 915 also includes multiple sets of optional options for modifying the current representation 916. A first set of options 921 includes options that, when selected, initiate a process for selecting an avatar to set as the current representation 916. A second set of options 922 typically includes options that, when selected, initiate a process for selecting a non-avatar representation as the current representation 916. Examples of non-avatar representations include images and letter combinations. In some embodiments, the second set of options 922 may include subgroups of options determined based on usage history. For example, subgroups may include previously (e.g., recently) used options and / or options frequently used by the user. In such embodiments, subgroups of options may include avatar options (e.g., recently used avatar options). In some embodiments, the second set of options 922 may include options recommended to the user based on information available for new contacts. For example, options may include a picture of the contact, an image / sticker / avatar sent to or received from the contact, an avatar associated with the contact, or other representations previously used for the contact. In some embodiments, options are recommended based on information available at device 600, such as, for example, the content of messaging metadata used in communication with the contact.

[0322] exist Figure 9EIn this implementation, device 600 detects input 924 (e.g., a tap gesture) on the monkey avatar option 921-1 and, in response, displays an interface for selecting a pose option for the chosen avatar option. In some embodiments, device 600 may display different types of pose interfaces. For example, if camera 602 of device 600 is configured to capture depth data (e.g., data for capturing changes in a user's facial pose), device 600 displays a real-time pose interface that allows the user to control the displayed avatar to achieve a desired pose. In embodiments where camera 602 is not configured to capture depth data, device 600 displays a pre-recorded pose interface that includes multiple predefined poses for the selected avatar option.

[0323] exist Figure 9F In the middle, device 600 displays a real-time gesture interface 926, which includes an avatar 928 with an appearance corresponding to the selected avatar option, a capture power indicator 930, and a cancel power indicator 932. Figure 9F In this context, avatar 928 corresponds to monkey avatar option 921-1 selected via input 924. As discussed in more detail below, avatar 928 tracks the movement of the user's face (e.g., captured via camera 602) and updates based on changes in the user's facial pose. Figure 9F In the image 928, the avatar has a smile pose controlled by the user's face (e.g., the user's face has a similar smile pose).

[0324] exist Figure 9F In the process, device 600 detects input 934 on the cancellation display 932 and, in response, returns to the display editing user interface 915 without updating the current display 916, as shown. Figure 9G As shown.

[0325] exist Figure 9G In the middle, device 600 detected input 936 on the female avatar option 921-2, and in response, displayed Figure 9H The real-time gesture user interface 926 in the image 928 has a corresponding... Figure 9G The appearance of the selected female avatar option 921-2. Figure 9H In this embodiment, the avatar 928 has a pose with its tongue protruding. The avatar 928 is controlled by the user's face via changes in facial pose detected by the camera 602 (e.g., changes in facial expression and movement of facial features). Therefore, the user can control the display of various pose options for the avatar 928 by moving their facial features within the field of view of the camera 602, which causes the device 600 to display corresponding changes in the pose of the avatar 928.

[0326] exist Figure 9IIn this process, device 600 detects (e.g., via camera 602) that the user's face has a pose including smiling and head tilting, and modifies avatar 928 to present the same pose. Device 600 detects input 938 on capture power display 930, which causes device 600 to select the current pose of avatar 928 (e.g., select the pose of avatar 928 when capture power display 930).

[0327] Figure 9J An alternative embodiment for selecting a pose for the chosen female avatar option 921-2 using a pre-recorded pose interface 940 is shown. If, for example, camera 602 is not configured to capture depth data (e.g., data for tracking the user's face), the pre-recorded pose interface 940 is displayed instead of the real-time pose interface 926 (e.g., in response to input 936 on the female avatar option 921-2). In the pre-recorded pose interface 940, device 600 displays various predefined avatar poses 942-1 to 942-6. Figure 9J In the process, device 600 detects and selects input 944 of a predefined head pose 642-3 corresponding to a smiling head tilt pose.

[0328] In capture Figure 9I or Figure 9J After selecting the avatar pose, device 600 displays a zoomable user interface 946 for changing the position and scale of the selected avatar pose 948, such as... Figure 9K As shown. In some embodiments, the avatar pose 948 moves in response to a swipe gesture detected when the zoom interface 946 is displayed (e.g., moves within a circular frame). In some embodiments, the avatar pose 948 zooms in or out in response to a pinch or spread gesture detected when the zoom interface 946 is displayed. After detecting input 950 to confirm the position and scale of the selected avatar pose 948, the device 600 displays... Figure 9L The background options 952-1 to 952-6 are used to detect the selection of the background option represented by the avatar (e.g., via input 954 at background option 952-3) and return to... Figure 9M The text in the image represents the user interface (915).

[0329] like Figure 9M As shown, the user interface 915 is updated based on the selection and customization of the female avatar option 921-2. For example, based on... Figures 9G to 9L Various choices and / or inputs are made to update the appearance of the current representation 916. Specifically, the current representation 916 is updated from... Figure 9G The appearance of the letter combination shown has been changed to Figure 9MThe image shows the appearance of a female avatar option with a smiling and tilted head pose. This avatar is positioned and scaled and has the selected background option 952-3. Furthermore, the second set of options 922 is updated to include a letter combination option 922-1 representing the previous appearance of the current avatar 916. That is, letter combination option 922-1 is an optional non-avatar option; if this option is selected, the current avatar 916 is updated to have the appearance of the previously displayed letter combination option (e.g., as shown in the image). Figure 9G (As shown). The remaining non-avatar options in the second group of options 922 are shifted to accommodate the display of the letter combination option 922-1 in the group of options, and the previously displayed non-avatar options are removed from the group (for example, to avoid pushing the first group of options 921 away from the display 601).

[0330] In some implementations, users can create avatars to select as the current representation. For example, in... Figure 9M In the middle, device 600 detected input 956 for selecting avatar creation option 921-3 and displayed... Figure 9N The avatar creation user interface 958 (similar to) Figure 6F The avatar creation user interface 632 shown is illustrated. Device 600 detected... Figure 9N The avatar creation user interface 958 typically uses input 959 to construct / create avatars. Figure 9O The new avatar 960 is shown. In response to input 962 on the completion display 963, device 600 exits the avatar creation user interface 958 and returns to Figure 9P The representation editing user interface 915 shows the current representation 916 being updated to have the appearance of the new avatar 960. In some embodiments, after the new avatar 960 is created, the new avatar can then be made available on the electronic device 600, including in other applications such as messaging applications, camera applications, media viewing applications, and other applications on device 600. Additionally, the new avatar 960 can be updated, and updates to the new avatar 960 can be included in other applications.

[0331] In addition, Figure 9P The first set of options 921 is updated to include a new avatar option 921-4, which is a representation of the new avatar 960, and the second set of options 922 is updated to include a female avatar option 922-2, which corresponds to the previous appearance of the current representation 916. As previously described, some of the optional options in the second set of options 922 are shifted to accommodate the addition of the female avatar option 922-2. In some embodiments, previously used representation options are added to the second set of options 922 at a position in the top row 922a (e.g., adjacent camera option 922-3 or adjacent letter combination option 922-1).

[0332] exist Figure 9P In the process, device 600 detects input 964 on letter combination option 922-1, and... Figure 9Q The background option 965 is displayed. After detecting a selection of the background option (e.g., via input 967), device 600... Figure 9R The text displays different font options 966 for the letters used in the alphabetical representation. After detecting a font style selection (e.g., via input 969), the text continues... Figure 9S In the middle, device 600 again displays the representation editing user interface 915, which shows the current representation 916 updated with the selected letter combination option, and the new avatar option 922-4 added to the second group of options 922.

[0333] exist Figure 9S In this embodiment, device 600 detects input 968 on photo option 922-5. In some embodiments, photo option 922-5 represents a thumbnail of a photo available at device 600. In some embodiments, the photo is identified (e.g., via automatic image recognition) as a photo of a contact (e.g., a photo of Jane Appleseed). In some embodiments, photo option 922-5 is a contact representation of a recently used contact.

[0334] In response to the detection of input 968, device 600 Figure 9T The device displays a filter user interface 970 with different filter options 972 that can be selected and applied to the selected photo. The device detects input 974 that selects one of the filter options 972 and applies the selected filter option to the photo, such as... Figure 9U As shown. Currently, it indicates 915. Figure 9U The image is displayed as having the selected photo option 922-5, but modified with the selected filter option 972. The second set of options 922 is updated with the most recent letter combination option 922-6, which represents the previous appearance of the current representation 916.

[0335] exist Figure 9U In the middle, device 600 detects input 975 on the completion indicator 917 and exits the indicator editing user interface 915.

[0336] exist Figure 9V In the device 600, a contact card 976 with a contact representation 978 (e.g., Jane Appleseed) is displayed. Figure 9U The current representation in the text is the appearance of 916.

[0337] exist Figure 9VIn the middle, device 600 detects input 980 on the edit indicator 982, and in response, displays... Figure 9W The text in the image represents the user interface (915).

[0338] exist Figure 9W In the process, device 600 detects input 984 on camera option 922-3 and, in response, displays camera user interface 986, which shows a representation of image data captured in the field of view of the camera (e.g., camera 602) of device 600. Device 600 detects input 987 on capture capability display 988 and captures using zoom user interface 990. Figure 9Y Image 989 is shown in the image.

[0339] exist Figure 9Y In the process, device 600 detects input 991 to select a zoomed (and moved) portion of image 989, which is in Figure 9Z The display shows various filter options 992. Device 600 detects the selection of the no-filter option 992-1 and, in response, displays... Figure 9AA The text indicates that the user interface 915 will be edited. Figures 9Y to 9Z The captured and selected unfiltered image is set as the current representation 916. The second set of options 922 is updated to include filtered photo options 922-7, which represent the previous appearance of the current representation 916.

[0340] exist Figure 9AA In the process, device 600 detects input 993 on all power indicators 994, which are power indicators used to access the photo library available on device 600. In response, device 600 displays... Figure 9AB The photo album user interface is 995. Figure 9AB and Figure 9AC In the process, device 600 detects the selection of photos from the albums available at device 600, and... Figure 9AD The selected photo is displayed in the zoom user interface 990 (representation 996).

[0341] In response to Figure 9AD In the input 997, device 600 detects the scaling and movement (e.g., cropping) of representation 996, and the selection of the scaled and moved image.

[0342] exist Figure 9AE In the process, device 600 displays filter options, detects the selection of one of the filter options, and displays the options described above. Figures 9AA to 9AE The current representation 916 of the image generated in the steps shown. Update the second set of options 922 with the previous photo options 922-8.

[0343] Device 600 detected input 998 on the completion indicator 917, and... Figure 9AG The middle displays the usefulness of Figures 9AA to 9AE The appearance of the contact representation generated by the selection made in the middle is updated. The contact representation 978 is the contact card 976.

[0344] Figure 10 This is a flowchart illustrating a method for displaying an avatar in a contact application user interface using an electronic device, according to some embodiments. Method 1000 is performed at a device (e.g., 100, 300, 500, 600) having a display and one or more input devices (e.g., 601, 602). Some operations in method 1000 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0345] As described below, method 1000 provides an intuitive way to display avatars in the contacts application user interface. This method reduces the cognitive burden on users when displaying avatars in the contacts application user interface, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to display avatars in the contacts application user interface more quickly and efficiently saves power and increases the interval between battery charges.

[0346] An electronic device (e.g., 600) displays (1002) a contactable user editable user interface (e.g., 915) via a display device (e.g., 601) (e.g., an interface for editing information for contactable users (e.g., for contact via telephone, email, message, etc.) at the electronic device; a single interface screen). In some embodiments, the contactable user editable user interface includes (e.g., simultaneously includes) one or more representation options for contactable users (e.g., 921) (e.g., a user interface object (powered representation)), including avatar representation options (e.g., 921-2) (e.g., an avatar representation option is a user interface object (e.g., a powered representation) that, when selected, initiates a process for selecting an avatar to be used as a representation for contactable users). In some embodiments, the avatar representation option has the appearance of an avatar (e.g., avatars such as, for example, avatars modeled to represent human characters, avatars modeled to represent non-human characters, avatars that can be created and / or customized by the user, and avatars that cannot be created or customized by the user). In some embodiments, avatars modeled to represent humans include customizable (e.g., optional or configurable) avatar features that typically correspond to human physical characteristics. For example, such avatars may include representations of people with various body types, human features, or characteristics (e.g., an elderly woman with dark skin and long, straight brown hair). Such avatars will also include representations of people with various non-human characteristics typically associated with human appearance (e.g., cosmetic enhancements, hats, glasses, etc.). In some embodiments, such avatars will not include anthropomorphic constructs, such as stylized animals, segmented robots, or stylized objects that are generally inanimate or generally non-human. In some embodiments, avatars modeled to represent non-human characters include, for example, anthropomorphic constructs of non-human characters (e.g., stylized animals, stylized robots, or stylized objects that are generally inanimate or generally non-human). In some embodiments, such avatars include avatars with customizable (e.g., optional or configurable) avatar features that typically correspond to non-human characteristics and features. In some implementations, such an avatar will not include representations of people with various body types, human features, or characteristics (e.g., a toddler with a round face and short, wavy hair), even if some of the customizable features of the human avatar include non-human characteristics (e.g., cosmetic enhancements, hats, glasses, or other non-human objects that are typically associated with human appearance).

[0347] In some embodiments, the user-editable interface (e.g., 915) also includes a first representation of the user (e.g., a visually associated image, letter combination, or other symbol provided to the user). In some embodiments, the representation of the user is displayed in other user interfaces (e.g., in a phone application UI, a messaging application UI, etc.) to indicate the user (typically in a small area on the screen). In some embodiments, the first representation of the user is replaced by an avatar selected for use as the representation of the user in the user-editable interface (e.g., 921-2).

[0348] In some embodiments, the one or more representation options include non-avatar options (e.g., 922-1, 922-3, 922-5, 922-6, 922-7, 922-8) (e.g., contactable user representation options that do not correspond to an avatar) (e.g., non-avatar options are associated with a photo, letter combination, or other options that are not used to select an avatar as a contactable user representation in the contactable user interface). In some embodiments, an electronic device (e.g., 600) detects the selection of a non-avatar option (e.g., 964, 968, 984, 993) via the one or more input devices (e.g., 601). In some embodiments, in response to detecting the selection of a non-avatar option, the electronic device initiates a process for selecting a representation option other than an avatar (e.g., a photo, letter combination, etc.) to be used as a contactable user representation in the contactable user interface.

[0349] In some implementations, the one or more representation options include multiple options selected based on information about the contactable user (e.g., 922).

[0350] In some implementations, the multiple options selected based on information about the contactable user include recently used representations of the contactable user (e.g., 922-1, 922-2, 922-4, 922-6, 922-7, 922-8) (e.g., representations of the contactable user previously used within a predetermined amount of time or a predetermined number of instances of selected representations of the contactable user). In some implementations, after a representation of the contactable user is selected, that representation is added to the set of recently used representations of the contactable user. Adding a recently selected representation of the contactable user to the set of recently used representations of the contactable user reduces the amount of input required to subsequently use the recently selected representation (e.g., reduces the input required to generate or access the representation). Reducing the amount of input enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.

[0351] In some implementations, the multiple options selected based on information about the contactable user include media items (e.g., 922-5) available at the electronic device that are identified as associated with the contactable user (e.g., a photo of the contactable user). In some implementations, the multiple options selected based on information about the contactable user include media items available at the electronic device that are identified as associated with the contactable user and meet specific quality criteria (e.g., the photo primarily captures the contactable user, the photo is in focus, etc.). In some implementations, the media item corresponds to the contactable user. For example, the media item includes a photo of the user. Also, the media item was previously (e.g., recently) sent to or received from the contactable user. Displaying media items of recent communication as a potential representation of the contactable user reduces the amount of input required for subsequent use of the media item (e.g., reducing the input required to generate or access the representation). Reducing the amount of input enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.

[0352] In some implementations, the contactable user information includes information from messaging communication sessions with the contactable user. In some implementations, items include stickers, photos, or other content including messaging metadata from communications with the contactable user.

[0353] In some implementations, the one or more representation options include letter combination representation options (e.g., 922-1, 922-6) (e.g., a representation of a contactable user having an abbreviation corresponding to the name (e.g., first name, last name, middle name) associated with the contactable user).

[0354] In some implementations, the one or more representation options include media item options (e.g., 922-5, 922-7, 922-8) (e.g., selecting a photo associated with a contactable user from a set of photos associated with a contactable user (e.g., a set of photos associated with a contactable user)).

[0355] In some embodiments, after detecting a selection of a media item option, an electronic device (e.g., 600) displays, via a display device (e.g., 601), a plurality of filter options for applying filter effects to the media item associated with the selected media item option. In some embodiments, filter effects are applied to media items by overlaying the filter effect onto the media item. In some embodiments, the filter effect alters both the background applied to the media item and any applied visual effects (e.g., avatars, stickers, etc.) that may be included in the media item. In some embodiments, the filter alters the appearance of the media item (e.g., applying a comic strip filter, sketch filter, black and white filter, grayscale filter, etc.). In some embodiments, the filter is a filter that reduces the realism of the media item (e.g., a sketch filter or a comic strip filter). In some embodiments, the filter is a filter that reduces the 3D effect of the media item (e.g., flattening). Displaying filter options for modifying a media item after selection reduces the amount of input required to customize the selected media item when using different control schemes (e.g., navigating to a custom user interface and selecting a control scheme for displaying and modifying different controls for the selected media item). Reducing the amount of input required to perform tasks enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This also reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0356] The electronic device detects (1004) the selection of an avatar representation option (e.g., 921-2) via the one or more input devices (e.g., 601) (e.g., 936).

[0357] In response to the detection of a selection of an avatar representation option (e.g., 921-2), an electronic device (e.g., 600) initiates (1006) a process for selecting an avatar to be used as a representation of a contactable user in a contactable user interface.

[0358] As part of the process for selecting an avatar to be used as a representation of a contactable user in a contactable user interface, the electronic device receives (1008) one or more inputs (e.g., 936, 938, 944, 950, 954, 956, 959, 962, ...) via the one or more input devices (e.g., 601, 602). Figure 9H The sequence of image data (controlling avatar 928) corresponds to a simulated 3D avatar (e.g., 928, ...). Figure 9M 916, 960, Figure 9P (916 in the middle).

[0359] In some implementations, in response to the selection of a simulated 3D avatar, an electronic device (e.g., 600) displays (1010) via a display device (e.g., 601) a posing user interface (e.g., 926, 940) including one or more controls (e.g., 942-1 to 942-6) (e.g., 930) for selecting a pose of the simulated 3D avatar from a plurality of different poses (e.g., multiple user interface objects (e.g., energy representations) corresponding to different predefined poses) (e.g., a capture energy representation for capturing a pose using the camera of the electronic device). Displaying a pose interface for selecting a pose of the avatar from a plurality of different poses after selecting an avatar reduces the amount of input required to customize the selected avatar when using different control schemes (e.g., the need to navigate to a custom user interface and select a control scheme for displaying and modifying different controls for the selected avatar). Reducing the amount of input required to perform a task enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0360] In some embodiments, the one or more controls include a first pose user interface object (e.g., 942-1) (e.g., a first pose energy representation) corresponding to a first predefined pose and a second pose user interface object (e.g., 942-2) (e.g., a second pose energy representation) corresponding to a second predefined pose different from the first predefined pose. In some embodiments, in response to detecting a selection of a pose energy representation, a simulated 3D avatar is set (e.g., displayed) to have a pose corresponding to the selected pose energy representation.

[0361] In some embodiments, the one or more input devices include a keyboard (e.g., 602). In some embodiments, the one or more controls include a capture user interface object (e.g., 630) (e.g., a capture display representation), which, when selected, selects a pose of a simulated 3D avatar based on a facial pose detected in the camera's field of view when the capture user interface object is selected (e.g., see...). Figure 9H and Figure 9I In some implementations, the display posing user interface includes displaying a capture energy representation and a simulated 3D avatar, wherein the simulated 3D avatar responds to facial pose changes detected in the camera's field of view (e.g., different detected facial poses correspond to multiple different poses from which a pose can be assigned to the 3D avatar). When the capture energy representation is selected, the electronic device selects a pose for the simulated 3D avatar. The capture user interface object that displays the selected avatar pose based on the face detected in the camera's field of view provides a control scheme for constructing a user-friendly representation on the display of the electronic device, wherein the system detects and processes input in the form of changes in the user's facial features (and the magnitude and / or direction of those changes) and provides the desired output in the form of the appearance of the avatar (e.g., a modified pose) through an iterative feedback loop, while eliminating the need for manual processing of the user interface (e.g., providing touch input on the display). This provides the user with improved visual feedback on how to manipulate the display to control and / or construct the avatar using facial movements. This enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently. Additionally, this control scheme may require less input to generate or control the avatar's pose compared to using different animation control schemes (e.g., control schemes that require manipulating individual control points for each frame of an animated sequence). Furthermore, this type of control can be completed in real-time, for example, during dialogues such as text or video conversations, whereas manual animation control of the avatar would have to be completed before or after the conversation begins.

[0362] In some implementations, from multiple different postures (e.g., Figure 9I (938 in the example) Figure 9J After selecting a pose for the simulated 3D avatar in (944), the electronic device sets the simulated 3D avatar with the selected pose as a representation that can be contacted by the user (e.g., Figure 9M(916 in the example) (e.g., displaying a representation of a contactable user with the appearance of a simulated 3D avatar having a selected pose; associating the simulated 3D avatar in the selected pose with a contactable user such that the 3D avatar in the selected pose is used to represent the contactable user).

[0363] In some implementations, displaying the posing user interface (e.g., 926) includes displaying the first avatar (e.g., 921-1) in the posing user interface based on determining that a first avatar (e.g., 921-1) is selected as a simulated 3D avatar (e.g., selecting an avatar representation corresponding to the first avatar). Figure 9F At least one representation of 928 in (e.g., a representation showing a first avatar with at least one selected pose). In some embodiments, displaying the posing user interface includes, based on determining that a second avatar (e.g., 921-2) is selected as a simulated three-dimensional avatar (e.g., selecting an avatar representation corresponding to the second avatar), displaying the second avatar (e.g., 921-2) in the posing user interface. Figure 9H At least one representation of (928) (e.g., a representation showing a second avatar with at least one selected pose) (e.g., a representation not showing the first avatar). Displaying a representation of the second avatar with a selected pose but not a representation of the first avatar provides the user with visual feedback that the pose change affects the appearance of the second avatar, not the first avatar. Providing improved feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending the device's battery life by enabling users to use the device more quickly and efficiently.

[0364] In some implementations, before displaying the user-editable interface (e.g., 915), the electronic device (e.g., 600) detects a series of inputs corresponding to a request to create a first user-created avatar (e.g., in the user interface of different applications). Figure 6F Avatar creation user interface 632 Figures 11A to 11ADThe electronic device detects a series of inputs for creating a first user-created avatar in the avatar editing user interface (11002) (e.g., 956). In some embodiments, the electronic device receives a request to display a contactable user editing user interface (e.g., 914) (e.g., 962). In some embodiments, in response to receiving a request to display a contactable user editing user interface, the electronic device displays a contactable user editing user interface including the first user-created avatar (e.g., 921-1, 921-4). Displaying a contactable user editing user interface including the first user-created avatar after the user has created / updated the avatar reduces the number of inputs required to perform the technical task of generating an avatar to be used as a representation of the contactable user. Reducing the number of inputs required to perform the task enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0365] In some implementations, the sequence of one or more inputs corresponding to the selection of a simulated 3D avatar includes inputs corresponding to selecting (e.g., 936) the first user-created (921-4, 921-2) avatar from a set of user-created avatars (e.g., 921-4, 921-2, 921-4, 921-2, 921-4). Figure 9G The girl's profile picture is adjacent to option 921-3. Figure 9G The avatar with the soaring hairstyle is adjacent to the female avatar option 921-2).

[0366] In some implementations, the sequence of one or more inputs corresponding to the selection of a simulated 3D avatar includes a set of inputs (e.g., 956, 959, 962) corresponding to the creation of a new avatar (e.g., creating an avatar in response to detecting a series of user inputs directed to the avatar creation user interface). In some implementations, a new avatar is created after the contactable user-editable user interface is displayed (e.g., when the contactable user-editable user interface is displayed, a series of inputs are received to access the avatar creation user interface and create a new avatar). In some implementations, the newly created avatar is selected as the simulated 3D avatar. Creating a new avatar in the contactable user-editable user interface reduces the number of inputs required to perform the technical task of generating a contactable user representation. This provides an improved control scheme for generating custom representations that may require fewer inputs than using different control schemes (e.g., a control scheme that requires navigating to different applications to create and customize an avatar, which can then be loaded into the contactable user-editable user interface for selection). Furthermore, this type of control can be completed in real time, for example, during a dialogue such as a text conversation or a video conversation, whereas manual control to build stickers would have to be completed before or after the conversation begins. This enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends the device's battery life by enabling users to use the device more quickly and efficiently.

[0367] In some implementations, after selecting a pose for the simulated 3D avatar from multiple different poses, the electronic device (e.g., 600) displays background options (e.g., ...) via the display device (e.g., 601). Figure 9L (See 952-3), the background option, when selected (e.g., 954), changes the appearance (e.g., color, shape, and / or texture) of the background area that can be contacted by the user (see, for example, 952-3). Figure 9M (Updated 916 in the middle).

[0368] In some implementations, displaying a posing user interface including the one or more controls includes determining that the one or more input devices (e.g., 602) include a depth camera sensor (e.g., Figure 1A The depth camera sensor 175 (e.g., a depth camera) displays a simulated 3D avatar with a dynamic appearance via the display device (e.g., 601). Figure 9H and Figure 9IThe simulated 3D avatar (928) changes pose in response to changes in facial pose detected in the field of view of a depth camera sensor (e.g., the simulated 3D avatar mirrors changes in facial pose detected by the depth camera). In some embodiments, the one or more controls are displayed as having a capture indication that, when selected, captures the pose of the simulated 3D avatar based on the facial pose detected in the field of view of the depth camera when the capture indication is selected. In some embodiments, displaying a posing user interface including the one or more controls includes: based on determining that the one or more input devices do not include a depth camera sensor, displaying via the display device a third pose user interface object (e.g., 942-3) (e.g., a third pose energy representation) (e.g., a first pose energy representation) corresponding to a third predefined pose (e.g., a first predefined pose) and a fourth pose user interface object (e.g., 942-4) (e.g., a fourth pose energy representation) (e.g., a second pose energy representation) corresponding to a fourth predefined pose (e.g., a second predefined pose) different from the third predefined pose (e.g., a second predefined pose). (For example, the one or more controls are displayed as a plurality of energy representations, each of which has the appearance of a simulated 3D avatar having one of the predefined poses.) In some embodiments, in response to detecting a selection of one of the pose energy representations, the simulated 3D avatar is set to a pose corresponding to the selected pose energy representation. The pose user interface, which displays a third pose user interface object corresponding to a third predefined pose and a fourth pose user interface object corresponding to a fourth predefined pose, provides a variety of optional pose options. This reduces the amount of input required to select a pose to perform the technical task of creating a user-friendly representation. Reducing the amount of input required to per...

Claims

1. A method comprising: At an electronic device that has a display device and one or more communication devices: Receive the first message via the one or more communication devices; Following the receipt of the first message, a request to display the first message is received; and In response to receiving the request to display the first message: Based on the determination that a first contactable user meets a set of prompting criteria, wherein the set of prompting criteria includes a first prompting criterion that is met when updated contact information corresponding to the first contactable user has been received, the following is simultaneously displayed on the display device: The first message, and The updated contact information can be used as a visual indicator for the first contactable user; and Based on the determination that the first contactable user does not meet the set of prompting criteria, the following is displayed on the display device: The first message, without displaying updated contact information, can be used as a visual indication for the first contactable user.

2. The method of claim 1, wherein the set of prompting criteria includes a second prompting criterion satisfied when the first message is part of a messaging conversation, wherein the first contactable user is a participant in the messaging conversation.

3. The method according to any one of claims 1 to 2, further comprising: Detecting updated contact information can be used to activate the visual indication for the first contactable user; as well as In response to the detection of updated contact information available for activation of the visual indication for the first contactable user: Based on the determination that the first contactable user does not correspond to an existing entry in a set of contactable users associated with the user of the electronic device, an optional option is displayed for creating a new entry for the first contactable user in the set of contactable users associated with the user of the electronic device using the contact information.

4. The method according to any one of claims 1 to 2, further comprising: Detecting updated contact information can be used to activate the visual indication for the first contactable user; as well as In response to the detection of updated contact information available for activation of the visual indication for the first contactable user: Based on the determination that the first contactable user corresponds to an existing entry in the set of contactable users associated with the user of the electronic device, and the received contact information including a modified graphical representation of the first contactable user and the modified name of the contactable user, a plurality of selectable options including two or more of the following are displayed: Selectable options for updating existing entries in the set of contactable users associated with the user of the electronic device using the modified graphical representation of the first contactable user and the modified name of the contactable user. An optional option for updating existing entries in the set of contactable users associated with the user of the electronic device using the modified graphical representation of the first contactable user, without using the modified name of the contactable user to update the existing entries, and An optional option for updating existing entries in the set of contactable users associated with the user of the electronic device using the modified name of the first contactable user, without using the modified graphical representation of the contactable user to update the existing entries.

5. The method according to any one of claims 1 to 2, further comprising: Detecting updated contact information can be used to activate the visual indication for the first contactable user; as well as In response to the detection of updated contact information available for activation of the visual indication for the first contactable user: Based on determining that the first contactable user corresponds to an existing entry in the set of contactable users associated with the user of the electronic device, and that the received contact information includes a modified graphic representation of the first contactable user but excludes the modified name of the contactable user, the existing entry is updated using the modified graphic representation of the contactable user.

6. The method according to any one of claims 1 to 2, further comprising: Detecting updated contact information can be used to activate the visual indication for the first contactable user; as well as In response to the detection of updated contact information available for activation of the visual indication for the first contactable user: Based on determining that the first contactable user corresponds to an existing entry in the set of contactable users associated with the user of the electronic device, and that the received contact information includes the modified name of the first contactable user but not the modified graphic representation of the contactable user, the existing entry is updated using the modified name of the contactable user.

7. The method of any one of claims 1 to 2, wherein the received contact information includes a modified graphical representation of the first contactable user, and wherein the first contactable user corresponds to an existing entry in a set of contactable users associated with the user of the electronic device, the method further comprising: Receive user input that updates the existing entry using the modified graphical representation of the first contactable user; In response to receiving user input that updates the existing entry using the modified graphical representation of the first contactable user: The modified graphical representation of the first contactable user is used to update the existing entries in the set of contactable users associated with the user of the electronic device; and The electronic device can display a selectable power indicator, and if the selectable power indicator is selected, the electronic device can automatically update the graphical representation of the first contactable user in the future.

8. The method according to any one of claims 1 to 2, wherein the contact information includes the name of the first contactable user or a graphical representation of the first contactable user, and the method further includes: The set of contactable users associated with the user of the electronic device is updated using the modified name of the first contactable user or the modified graphical representation of the contactable user; and The contact information, including the modified name or the modified graphic representation, of the set of contactable users associated with the user of the electronic device can be used in multiple applications of the electronic device.

9. The method according to any one of claims 1 to 2, wherein the electronic device is associated with a user, the method further comprising: In response to receiving the request to display the first message: Based on the determination that the first contactable user does not meet a set of sharing criteria, wherein the set of sharing criteria includes a first sharing criterion that is met when the first contactable user corresponds to an approved recipient: Simultaneously displaying the updated contact information of the user on the electronic device can be used to transmit instructions to the first contactable user and the first message.

10. The method of any one of claims 1 to 2, wherein the first message is received in a conversation including the first contactable user and the second contactable user, and wherein the electronic device has received updated contact information for the first contactable user and updated contact information for the second contactable user, the method further comprising: Detecting updated contact information can be used to activate the visual indication for the first contactable user; as well as In response to the detection of updated contact information available for activation of the visual indication of the first contactable user, the following is displayed: The updated contact information can be used for a second visual indication of the first contactable user, wherein the second visual indication includes a visual representation of at least a portion of the received updated contact information for the first contactable user; and The updated contact information can be used for a third visual indication of the second contactable user, wherein the third visual indication includes a visual representation of at least a portion of the received updated contact information for the second contactable user.

11. The method according to any one of claims 1 to 2, wherein the contact information of the first contactable user includes information corresponding to an avatar.

12. The method according to any one of claims 1 to 2, wherein the updated contact information can be used to simultaneously display the visual indication of the first contactable user and a visual representation of at least a portion of a messaging conversation comprising multiple messages, the multiple messages including a second message transmitted to the first contactable user and a third message received from the first contactable user.

13. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more communication devices, the one or more programs including instructions for performing the method according to any one of claims 1 to 12.

14. An electronic device comprising: Display devices; One or more communication devices; One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 12.

15. An electronic device comprising: Components for performing the method according to any one of claims 1 to 12.

16. A computer program product comprising one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more communication devices, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 12.

Images