Creative Camera

Abstract

To provide a more prompt and efficient method and interface for displaying a visual effect on an electronic device.SOLUTION: A visual effect is displaying on an image data, in which the visual effect includes a custom avatar 835 displayed on the face of a user and a sticker 858-1 applied to the image data. The visual effect also includes a screen effect and is changed based on depth data within the image data.SELECTED DRAWING: Figure 8AJ

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is related to U.S. Provisional Patent Application No. 62 / 668,227, entitled "Creative Camera," filed May 7, 2018, the contents of which are incorporated herein by reference in their entirety.

[0002] FIELD OF THE DISCLOSURE This disclosure relates generally to computer user interfaces, and more particularly to techniques for displaying visual effects. [Background technology]

[0003] Visual effects are used to enhance the user experience when capturing and viewing media using electronic devices. Visual effects can change the appearance of image data or can present idealized or completely fictional representations of the environment captured in the image. Summary of the Invention

[0004] However, some techniques for displaying visual effects using electronic devices are generally cumbersome and inefficient. For example, some existing techniques use complex and time-consuming user interfaces that may involve multiple key presses or keystrokes. Existing techniques require more time than necessary, wasting user time and device power. The latter problem is particularly acute in battery-operated devices.

[0005] The present technology thus provides electronic devices with faster, more efficient methods and interfaces for displaying visual effects. Such methods and interfaces optionally complement or replace other methods for displaying visual effects. Such methods and interfaces reduce the cognitive burden on users and create more efficient human-machine interfaces. In the case of battery-operated computing devices, such methods and interfaces conserve power and increase the time between battery charges.

[0006] A method is described that is executed on an electronic device having a camera, a display device, and one or more input devices. The method includes displaying, via the display device, a messaging user interface including a camera affordance of a message conversation including at least a first participant, detecting, via the one or more input devices, a first input directed at the camera affordance, and in response to detecting the first input, displaying a camera user interface including a capture affordance, detecting, via the one or more input devices, a second input directed at the capture affordance, and in response to detecting the second input, capturing image data using the camera, ceasing display of the capture affordance, displaying a send affordance in a position within the camera user interface previously occupied by the capture affordance, detecting, via the one or more input devices, a third input directed at the send affordance, and in response to detecting the third input, initiating a process of transmitting the captured image data to the first participant.

[0007] A non-transitory computer-readable storage medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device including a camera, a display device, and one or more input devices, the one or more programs including instructions to: display, via the display device, a messaging user interface including a camera affordance of a message conversation including at least a first participant; detect, via the one or more input devices, a first input directed at the camera affordance, and in response to detecting the first input, display a camera user interface including a capture affordance; detect, via the one or more input devices, a second input directed at the capture affordance, and in response to detecting the second input, capture image data using the camera, stop displaying the capture affordance, display a send affordance in a position within the camera user interface previously occupied by the capture affordance; detect, via the one or more input devices, a third input directed at the send affordance, and in response to detecting the third input, initiate a process to send the captured image data to the first participant.

[0008] A transient computer-readable storage medium is described, the transient computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device including a camera, a display device, and one or more input devices, the one or more programs including instructions to: display, via the display device, a messaging user interface including a camera affordance of a message conversation including at least a first participant; detect, via the one or more input devices, a first input directed at the camera affordance, and in response to detecting the first input, display a camera user interface including a capture affordance; detect, via the one or more input devices, a second input directed at the capture affordance, and in response to detecting the second input, capture image data using the camera, stop displaying the capture affordance, display a send affordance in a position within the camera user interface previously occupied by the capture affordance; detect, via the one or more input devices, a third input directed at the send affordance, and in response to detecting the third input, initiate a process to send the captured image data to the first participant.

[0009] An electronic device is described that includes a camera, a display device, one or more input devices, one or more processors, and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions to: display, via the display device, a messaging user interface including a camera affordance of a message conversation including at least a first participant; detect, via the one or more input devices, a first input directed at the camera affordance, and in response to detecting the first input, display a camera user interface including a capture affordance; detect, via the one or more input devices, a second input directed at the capture affordance, and in response to detecting the second input, capture image data using the camera, stop displaying the capture affordance, display a send affordance in a position within the camera user interface previously occupied by the capture affordance; detect, via the one or more input devices, a third input directed at the send affordance, and in response to detecting the third input, initiate a process to send the captured image data to the first participant.

[0010] An electronic device is described that includes a camera, a display device, one or more input devices, means for displaying, via the display device, a messaging user interface including a camera affordance for a message conversation including at least a first participant, means for detecting, via the one or more input devices, a first input directed at the camera affordance, and in response to detecting the first input, means for displaying the camera user interface including a capture affordance, means for detecting, via the one or more input devices, a second input directed at the capture affordance, and in response to detecting the second input, means for capturing image data using the camera, ceasing display of the capture affordance, and displaying a send affordance in a location within the camera user interface previously occupied by the capture affordance, means for detecting, via the one or more input devices, a third input directed at the send affordance, and in response to detecting the third input, means for initiating transmission of the captured image data to the first participant.

[0011] A method is described, the method being executed on an electronic device having a camera and a display. The method includes: displaying a camera user interface via the display, the camera user interface including a camera viewing area including a representation of image data captured via the camera and a first affordance associated with a first camera viewing mode; detecting a gesture directed toward the first affordance while a subject is located within a field of view of the camera and representations of the subject and a background are displayed in the camera viewing area; activating the first camera viewing mode in response to detecting the gesture directed toward the first affordance, the activating the first camera viewing mode including displaying an avatar selection area including a selected one of a plurality of avatar options and displaying a representation of the selected avatar option over a representation of the subject in the camera viewing area; detecting a change in a posture of the subject while the first camera viewing mode is active; and in response to detecting the change in the posture of the subject, modifying an appearance of the displayed representation of the selected avatar option based on the detected change in the posture of the subject while maintaining display of the background.

[0012] A non-transitory computer-readable storage medium is described. A non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a camera and a display device, the one or more programs including instructions for: displaying, via the display device, a camera user interface including a camera viewing area including a representation of image data captured via the camera and a first affordance associated with a first camera viewing mode; detecting a gesture directed toward the first affordance while a subject is located within the field of view of the camera and representations of the subject and a background are displayed in the camera viewing area; activating the first camera viewing mode in response to detecting the gesture directed toward the first affordance, wherein activating the first camera viewing mode includes displaying an avatar selection area including a selected one of a plurality of avatar options and displaying a representation of the selected avatar option over a representation of the subject in the camera viewing area; detecting a change in the subject's posture while the first camera viewing mode is active; and in response to detecting the change in the subject's posture, modifying an appearance of the displayed representation of the selected avatar option based on the detected change in the subject's posture while maintaining display of the background.

[0013] A transitory computer-readable storage medium is described. A temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a camera and a display device, the one or more programs including instructions for: displaying, via the display device, a camera user interface including a camera viewing area including a representation of image data captured via the camera and a first affordance associated with a first camera viewing mode; detecting a gesture directed toward the first affordance while a subject is located within the camera's field of view and representations of the subject and a background are displayed in the camera viewing area; activating the first camera viewing mode in response to detecting the gesture directed toward the first affordance, wherein activating the first camera viewing mode includes displaying an avatar selection area including a selected one of a plurality of avatar options and displaying a representation of the selected avatar option over a representation of the subject in the camera viewing area; detecting a change in the subject's posture while the first camera viewing mode is active; and in response to detecting the change in the subject's posture, modifying an appearance of the displayed representation of the selected avatar option based on the detected change in the subject's posture while maintaining display of the background.

[0014] An electronic device is described that includes a camera, a display device, one or more processors, and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for displaying, via the display device, a camera user interface including a camera viewing area including a representation of image data captured via the camera and a first affordance associated with a first camera viewing mode, detecting a gesture directed toward the first affordance while a subject is located within the field of view of the camera and representations of the subject and a background are displayed in the camera viewing area, activating the first camera viewing mode in response to detecting the gesture directed toward the first affordance, displaying an avatar selection area including a selected one of a plurality of avatar options and displaying a representation of the selected avatar option over a representation of the subject in the camera viewing area, and detecting a change in a posture of the subject while the first camera viewing mode is active, and in response to detecting the change in the posture of the subject, modifying an appearance of the displayed representation of the selected avatar option based on the detected change in the posture of the subject while maintaining display of the background.

[0015] An electronic device is described that includes a camera, a display, one or more input devices, and means for displaying a camera user interface via the display, the camera user interface including a camera viewing area including a representation of image data captured via the camera and a first affordance associated with a first camera viewing mode, means for detecting a gesture directed toward the first affordance while a subject is located within the field of view of the camera and representations of the subject and a background are displayed in the camera viewing area, and means for activating the first camera viewing mode in response to detecting the gesture directed toward the first affordance, where activating the first camera viewing mode includes displaying an avatar selection area including a selected one of a plurality of avatar options and displaying a representation of the selected avatar option over a representation of the subject in the camera viewing area, means for detecting a change in a posture of the subject while the first camera viewing mode is active, and means for modifying an appearance of the displayed representation of the selected avatar option based on the detected change in the posture of the subject while maintaining display of the background in response to detecting the change in the posture of the subject.

[0016] A method is described, the method being executed on an electronic device having a display device, comprising: displaying, via the display device, a media user interface including a media display area including a representation of a media item and an effect affordance; detecting a gesture directed at the effect affordance; and, in response to detecting the gesture directed at the effect affordance, displaying, concurrently with the representation of the media item, a plurality of effect options for applying an effect to the media item, wherein, in accordance with a determination that the media item is associated with corresponding depth data, the plurality of effect options include respective effect options for applying an effect based on the depth data; and, in accordance with a determination that the image data does not include depth data, respective effect options are unavailable for activation in the plurality of effect options.

[0017] A non-transitory computer-readable storage medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for displaying, via the display device, a media user interface including a media display area including a representation of a media item and an effect affordance, detecting a gesture directed at the effect affordance, and, in response to detecting the gesture directed at the effect affordance, displaying, concurrently with the representation of the media item, a plurality of effect options for applying an effect to the media item, wherein, in accordance with a determination that the media item is associated with corresponding depth data, the plurality of effect options include respective effect options for applying an effect based on the depth data, and, in accordance with a determination that the image data does not include depth data, the respective effect option is unavailable for activation in the plurality of effect options.

[0018] A temporary computer-readable storage medium is described, the temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for displaying, via the display device, a media user interface including a media display area including a representation of a media item and an effect affordance, detecting a gesture directed at the effect affordance, and, in response to detecting the gesture directed at the effect affordance, displaying, concurrently with the representation of the media item, a plurality of effect options for applying an effect to the media item, wherein, in accordance with a determination that the media item is associated with corresponding depth data, the plurality of effect options include respective effect options for applying an effect based on the depth data, and, in accordance with a determination that the image data does not include depth data, the respective effect option is unavailable for activation in the plurality of effect options.

[0019] An electronic device is described that includes a display device, one or more processors, and a memory that stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for displaying, via the display device, a media user interface including a media display area including a representation of a media item and an effect affordance, detecting a gesture directed at the effect affordance, and, in response to detecting the gesture directed at the effect affordance, displaying a plurality of effect options for applying an effect to the media item concurrently with the representation of the media item, wherein, in accordance with a determination that the media item is associated with corresponding depth data, the plurality of effect options include respective effect options for applying an effect based on the depth data, and, in accordance with a determination that the image data does not include depth data, the respective effect option is unavailable for activation in the plurality of effect options.

[0020] An electronic device is described that includes: a display device; one or more input devices; means for displaying, via the display device, a media user interface including a media display area including a representation of a media item and an effect affordance; means for detecting a gesture directed at the effect affordance; and means for displaying, in response to detecting the gesture directed at the effect affordance, a plurality of effect options for applying an effect to the media item concurrently with the representation of the media item, wherein, in accordance with a determination that the media item is associated with corresponding depth data, the plurality of effect options include respective effect options for applying an effect based on the depth data, and in accordance with a determination that the image data does not include depth data, the respective effect options are unavailable for activation in the plurality of effect options.

[0021] A method is described, the method being executed on an electronic device having a display device, the method including: displaying, via the display device, a live video communication user interface of a live video communication application, the live video communication user interface including a representation of a subject participating in the live video communication session and a first affordance; detecting a gesture directed toward the first affordance; and, in response to detecting the gesture directed toward the first affordance, activating a camera effects mode to increase a size of the representation of the subject participating in the live video communication session.

[0022] A non-transitory computer-readable storage medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device including a display device, the one or more programs including instructions for displaying, via the display device, a live video communication user interface of a live video communication application, the live video communication user interface including a representation of a subject participating in the live video communication session and a first affordance, detecting a gesture directed toward the first affordance, and, in response to detecting the gesture directed toward the first affordance, activating a camera effects mode and increasing a size of the representation of the subject participating in the live video communication session.

[0023] A temporary computer-readable storage medium is described, the temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device including a display device, the one or more programs including instructions for displaying, via the display device, a live video communication user interface of a live video communication application, the live video communication user interface including a representation of a subject participating in the live video communication session and a first affordance, detecting a gesture directed toward the first affordance, and, in response to detecting the gesture directed toward the first affordance, activating a camera effects mode and increasing a size of the representation of the subject participating in the live video communication session.

[0024] An electronic device is described that includes a display device, one or more processors, and a memory that stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for displaying, via the display device, a live video communication user interface of a live video communication application, the live video communication user interface including a representation of a subject participating in the live video communication session and a first affordance, detecting a gesture directed toward the first affordance, and, in response to detecting the gesture directed toward the first affordance, activating a camera effects mode and increasing a size of the representation of the subject participating in the live video communication session.

[0025] An electronic device is described that includes a display device, one or more input devices, means for displaying, via the display device, a live video communication user interface of a live video communication application, the live video communication user interface including a representation of a subject participating in the live video communication session and a first affordance, means for detecting a gesture directed toward the first affordance, and means for activating a camera effects mode to increase a size of a representation of a subject participating in the live video communication session in response to detecting the gesture directed toward the first affordance.

[0026] A method is described that is implemented in an electronic device having a camera and a display device. The method includes: displaying, via the display device, a representation of image data captured via one or more cameras, the representation including a representation of a subject, and the image data corresponding to depth data including depth data of the subject; and displaying, via the display device, a representation of a virtual avatar displayed in place of at least a portion of the representation of the subject, the virtual avatar being positioned at a simulated depth relative to the representation of the subject, the simulated depth being determined based on the depth data of the subject, wherein displaying the representation of the virtual avatar includes displaying, based on the depth data, a first portion of the virtual avatar in accordance with a depth-based display standard, the first portion of the virtual avatar being positioned at a simulated depth relative to a corresponding first portion of the subject. pursuant to a determination that the depth data of the subject satisfies a set of depth-based display criteria, including a requirement that the depth data of the subject indicate that the first portion of the virtual avatar has a simulated depth that is in front of the first portion of the subject, including a representation of the first portion of the virtual avatar displayed in place of the first portion of the subject as part of the representation of the virtual avatar, such that the depth-based display criteria are satisfied; and pursuant to a determination, based on the depth data, that the first portion of the virtual avatar does not satisfy the set of depth-based display criteria for the first portion of the subject, excluding the representation of the first portion of the virtual avatar from the representation of the virtual avatar and displaying the first portion of the subject in an area that would have been occupied by the first portion of the virtual avatar.

[0027] A non-transitory computer-readable storage medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device comprising a camera and a display device, the one or more programs including instructions for displaying, via the display device, a representation of image data captured via the one or more cameras, the representation including a representation of a subject, and the image data corresponding to depth data including depth data of the subject, and displaying, via the display device, a representation of a virtual avatar displayed in place of at least a portion of the representation of the subject, the virtual avatar being positioned at a simulated depth relative to the representation of the subject determined based on the depth data of the subject, and displaying the representation of the virtual avatar includes displaying, based on the depth data, a first portion of the virtual avatar at a simulated depth relative to the representation of the subject. pursuant to a determination that the depth data of the subject satisfies a set of depth-based display criteria, including a requirement that the depth data of the subject indicate that the first portion of the virtual avatar has a simulated depth that is in front of a corresponding first portion of the subject, including a representation of the first portion of the virtual avatar displayed in place of the first portion of the subject as part of the representation of the virtual avatar, such that the depth-based display criteria are satisfied; and pursuant to a determination, based on the depth data, that the first portion of the virtual avatar does not satisfy the set of depth-based display criteria for the first portion of the subject, excluding the representation of the first portion of the virtual avatar from the representation of the virtual avatar and displaying the first portion of the subject in an area that would have been occupied by the first portion of the virtual avatar.

[0028] A transient computer-readable storage medium is described, the transient computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a camera and a display device, the one or more programs including instructions for displaying, via the display device, a representation of image data captured via the one or more cameras, the representation including a representation of a subject, and the image data corresponding to depth data including depth data of the subject, and displaying, via the display device, a representation of a virtual avatar displayed in place of at least a portion of the representation of the subject, the virtual avatar being positioned at a simulated depth relative to the representation of the subject determined based on the depth data of the subject, and displaying the representation of the virtual avatar includes displaying, based on the depth data, a first portion of the virtual avatar at a simulated depth relative to the representation of the subject, the first portion of the virtual avatar being positioned at a simulated depth relative to the representation of the subject determined based on the depth data. In accordance with a determination that the depth data of the subject satisfies a set of depth-based display criteria, including a requirement that the depth data of the subject indicate that the first portion of the virtual avatar has a simulated depth that is in front of a corresponding first portion of the subject, including a representation of the first portion of the virtual avatar displayed in place of the first portion of the subject as part of the representation of the virtual avatar, such that the depth-based display criteria are satisfied; and in accordance with a determination based on the depth data that the first portion of the virtual avatar does not satisfy the set of depth-based display criteria for the first portion of the subject, excluding the representation of the first portion of the virtual avatar from the representation of the virtual avatar and displaying the first portion of the subject in an area that would have been occupied by the first portion of the virtual avatar.

[0029] An electronic device is described, the electronic device including a camera, a display device, one or more processors, and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for displaying, via the display device, a representation of image data captured via the one or more cameras, the representation including a representation of a subject, and the image data corresponding to depth data including depth data of the subject, and displaying, via the display device, a representation of a virtual avatar displayed in place of at least a portion of the representation of the subject, the virtual avatar being positioned at a simulated depth relative to the representation of the subject determined based on the depth data of the subject, and displaying the representation of the virtual avatar includes displaying, based on the depth data, a first portion of the virtual avatar at a simulated depth relative to the representation of the subject. The method includes, in accordance with a determination that the depth data of the subject satisfies a set of depth-based display criteria, including a requirement that the depth data of the subject indicate that the first portion of the virtual avatar has a simulated depth that is in front of a corresponding first portion of the subject, including a representation of the first portion of the virtual avatar displayed in place of the first portion of the subject as part of the representation of the virtual avatar, such that the depth-based display criteria are satisfied; in accordance with a determination based on the depth data that the first portion of the virtual avatar does not satisfy the set of depth-based display criteria for the first portion of the subject, excluding the representation of the first portion of the virtual avatar from the representation of the virtual avatar; and displaying the first portion of the subject in an area that would have been occupied by the first portion of the virtual avatar.

[0030] An electronic device is described, the electronic device including: a camera; a display device; means for displaying, via the display device, a representation of image data captured via the one or more cameras, the representation including a representation of a subject, and the image data corresponding to depth data including depth data of the subject; and means for displaying, via the display device, a representation of a virtual avatar displayed in place of at least a portion of the representation of the subject, the virtual avatar being positioned at a simulated depth relative to the representation of the subject determined based on the depth data of the subject, wherein displaying the representation of the virtual avatar includes displaying, based on the depth data, a first portion of the virtual avatar according to a depth-based display standard, the first portion of the virtual avatar being positioned at a simulated depth relative to the representation of the subject. pursuant to a determination that the depth data of the subject satisfies a set of depth-based display criteria, including a requirement that the depth data of the subject indicate that the subject has a simulated depth that is in front of the corresponding first portion, including a representation of the first portion of the virtual avatar displayed in place of the first portion of the subject, such that the depth-based display criteria are satisfied; and pursuant to a determination, based on the depth data, that the first portion of the virtual avatar does not satisfy the set of depth-based display criteria for the first portion of the subject, excluding the representation of the first portion of the virtual avatar from the representation of the virtual avatar and displaying the first portion of the subject in an area that would have been occupied by the first portion of the virtual avatar.

[0031] Executable instructions for performing these functions are optionally contained in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors. Executable instructions for performing these functions are optionally contained in a transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0032] This provides devices with faster, more efficient methods and interfaces for displaying visual effects, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can complement or replace other methods for displaying visual effects.

[0033] For a better understanding of the various embodiments described, reference should be made to the following detailed description in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout the drawings, in which: [Brief explanation of the drawings]

[0034] [Figure 1A] 1 is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

[0035] [Figure 1B] FIG. 1 is a block diagram illustrating exemplary components for event processing according to some embodiments.

[0036] [Figure 2] 1 illustrates a portable multifunction device with a touch screen according to some embodiments.

[0037] [Figure 3] FIG. 1 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments.

[0038] [Figure 4A] 1 illustrates an exemplary user interface for a menu of applications on a portable multifunction device in accordance with some embodiments.

[0039] [Figure 4B] 1A-1C illustrate exemplary user interfaces for multifunction devices with touch-sensitive surfaces separate from displays in accordance with some embodiments.

[0040] [Figure 5A] FIG. 1 illustrates a personal electronic device according to some embodiments.

[0041] [Figure 5B] 1 is a block diagram illustrating a personal electronic device according to some embodiments.

[0042] [Figure 6A] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6B] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6C] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6D] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6E] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6F] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6G] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6H] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6I] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6J] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6K] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6L] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6M] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6N] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6O] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6P] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6Q] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6R] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6S] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6T] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6U]1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6V] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6W] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6X] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6Y] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6Z] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AA] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AB] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AC] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AD] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AE] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AF] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AG] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AH] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AI] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AJ] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AK] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AL] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AM] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AN] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AO] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AP] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AQ] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AR]1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AS] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AT] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AU] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AV] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AW] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AX] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AY] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6AZ] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BA] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BB] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BC] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BD] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BE] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BF] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BG] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BH] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BI] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BJ] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BK] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BL] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BM] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BN] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BO]1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BP] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application. [Figure 6BQ] 1 illustrates an exemplary user interface for displaying visual effects in a messaging application.

[0043] [Figure 7A] 6A-6BQ are flow diagrams illustrating methods for displaying visual effects in messaging applications.The user interfaces of FIGS. 6A-6BQ are used to illustrate the processes described below, including the process of FIG. [Figure 7B] 6A-6BQ are flow diagrams illustrating methods for displaying visual effects in messaging applications.The user interfaces of FIGS. 6A-6BQ are used to illustrate the processes described below, including the process of FIG.

[0044] [Figure 8A] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8B] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8C] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8D] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8E] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8F]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8G] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8H] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8I] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8J] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8K] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8L] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8M] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8N] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8O] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8P] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8Q] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8R]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8S] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8T] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8U] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8V] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8W] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8X] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8Y] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8Z] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AA] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AB] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AC] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AD]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AE] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AF] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AG] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AH] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AI] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AJ] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AK] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AL] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AM] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AN] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AO] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AP]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AQ] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AR] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AS] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AT] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AU] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AV] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AW] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AX] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AY] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8AZ] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BA] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BB]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BC] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BD] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BE] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BF] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BG] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BH] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BI] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BJ] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BK] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BL] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BM] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BN]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BO] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BP] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 8BQ] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application.

[0045] [Figure 9A] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a camera application. [Figure 9B] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a camera application.

[0046] [Figure 10A] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10B] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10C] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10D] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10E] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10F]10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10G] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10H] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10I] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10J] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10K] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10L] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10M] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10N] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10O] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10P] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10Q] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10R]10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10S] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10T] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10U] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10V] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10W] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10X] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10Y] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10Z] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AA] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AB] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AC]10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AD] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AE] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AF] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AG] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AH] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AI] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AJ] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AK] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode. [Figure 10AL] 10A-10C illustrate exemplary user interfaces for displaying visual effects in a media item viewing mode.

[0047] [Figure 11A] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a media item viewing mode. [Figure 11B] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a media item viewing mode.

[0048] [Figure 12A] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12B] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12C] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12D] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12E] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12F] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12G] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12H] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12I] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12J] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12K] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12L]1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12M] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12N] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12O] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12P] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12Q] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12R] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12S] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12T] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12U] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12V] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12W] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12X]1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12Y] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12Z] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AA] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AB] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AC] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AD] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AE] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AF] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AG] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AH] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AI] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AJ]1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AK] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AL] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AM] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AN] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AO] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session. [Figure 12AP] 1 illustrates an exemplary user interface for displaying visual effects in a live video communication session.

[0049] [Figure 13A] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a live video communication session. [Figure 13B] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a live video communication session.

[0050] [Figure 14A] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14B] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14C]FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14D] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14E] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14F] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14G] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14H] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14I] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14J] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14K] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14L] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application. [Figure 14M] FIG. 1 illustrates an exemplary user interface for displaying visual effects in a camera application.

[0051] [Figure 15A] FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a camera application. [Figure 15B]FIG. 1 is a flow diagram illustrating a method for displaying visual effects in a camera application. DETAILED DESCRIPTION OF THE INVENTION

[0052] In the following description, example methods, parameters, etc. are set forth. However, it is to be understood that the purpose of such description is not to limit the scope of the present disclosure, but rather to provide a description of example embodiments.

[0053] There is a need for electronic devices that provide efficient methods and interfaces for displaying visual effects. For example, while programs for displaying visual effects already exist, these programs are inefficient and difficult to use compared to the following techniques that allow users to display visual effects in various applications. Such techniques can reduce the cognitive burden on users of displaying visual effects in applications, thereby increasing productivity. Furthermore, such techniques can reduce processor and battery power that would otherwise be wasted on redundant user input.

[0054] 1A-1B, 2, 3, 4A-4B, and 5A-5B below provide a description of an example device for performing techniques for managing event notifications.

[0055]

[0013] Figures 6A-6BQ illustrate example user interfaces for displaying visual effects in a messaging application. Figures 7A and 7B are flow diagrams illustrating methods for displaying visual effects in a messaging application, according to some embodiments. The user interfaces of Figures 6A-6BQ are used to illustrate processes described below, including the processes of Figures 7A and 7B.

[0056] 8A-8BQ illustrate example user interfaces for displaying visual effects in a camera application. Figures 9A and 9B are flow diagrams illustrating methods for displaying visual effects in a camera application, according to some embodiments. The user interfaces of Figures 8A-8BQ are used to illustrate processes described below, including the processes of Figures 9A and 9B.

[0057] 10A-10AL illustrate exemplary user interfaces for displaying visual effects in a camera application. Figures 11A and 11B are flow diagrams illustrating methods for displaying visual effects in a media item viewing mode, according to some embodiments. The user interfaces of Figures 10A-10AL are used to illustrate the processes described below, including the processes of Figures 11A and 11B.

[0058] 12A-12AP illustrate exemplary user interfaces for displaying visual effects in a live video communication session. Figures 13A and 13B are flow diagrams illustrating methods for displaying visual effects in a live video communication session, according to some embodiments. The user interfaces of Figures 12A-12AP are used to illustrate processes described below, including the processes of Figures 13A and 13B.

[0059] 14A-14M illustrate example user interfaces for displaying visual effects in a camera application. Figures 15A and 15B are flow diagrams illustrating methods for displaying visual effects in a camera application according to some embodiments. The user interfaces of Figures 14A-14M are used to illustrate processes described below, including the processes of Figures 15A and 15B.

[0060] In the following description, terms such as "first" and "second" are used to describe various elements, but these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first touch can be referred to as a second touch, and similarly, a second touch can be referred to as a first touch, without departing from the scope of the various embodiments described. A first touch and a second touch are both touches, but are not the same touch.

[0061] The terminology used in the description of the various embodiments set forth herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments set forth and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. As used herein, the term "and / or" should also be understood to refer to and include any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0062] Optionally, the term "if" is interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are optionally interpreted to mean "upon determining," "in response to determining," "upon detecting (the stated condition or event)," or "in response to detecting (the stated condition or event)," depending on the context.

[0063] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile telephone, that also includes other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads), are also optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touchscreen displays and / or touchpads).

[0064] In the following discussion, electronic devices are described that include a display and a touch-sensitive surface, however, it should be understood that the electronic device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0065] The device typically supports a variety of applications such as one or more of a drawing application, a presentation application, a word processing application, a website creation application, a disc authoring application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an email application, an instant messaging application, a training support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0066] Various applications running on the device optionally use at least one common physical user-interface device, such as a touch-sensitive surface. One or more features of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or changed for each application and / or within each application. In this way, the common physical architecture (such as the touch-sensitive surface) of the device optionally supports various applications with user interfaces that are intuitive and transparent to the user.

[0067] Attention now turns to embodiments of portable devices with touch-sensitive displays. FIG. 1A is a block diagram illustrating portable multifunction device 100 with touch-sensitive display system 112, according to some embodiments. Touch-sensitive display 112 may be conveniently referred to as a “touch screen,” and may also be known or referred to as a “touch-sensitive display system.” Device 100 includes memory 102 (optionally including one or more computer-readable storage media), memory controller 122, one or more processing units (CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input control devices 116, and external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors 165 for detecting the intensity of a contact on device 100 (e.g., a touch-sensitive surface, such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0068] As used in this specification and in the claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or a surrogate (proxy) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a value range that includes at least four distinct values ​​and more typically includes hundreds (e.g., at least 256) distinct values. The intensity of a contact is optionally determined (or measured) using various techniques and various sensors or combinations of sensors. For example, one or more force sensors beneath or adjacent to the touch-sensitive surface are optionally used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average) to determine an estimate of the force of the contact. Similarly, a pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or change in the contact area detected on the touch-sensitive surface, the capacitance and / or change in the capacitance of the touch-sensitive surface proximate the contact, and / or the resistance and / or change in the capacitance of the touch-sensitive surface proximate the contact are optionally used as surrogates for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measure for the force or pressure of the contact is used directly to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measure). In some implementations, the surrogate measure for the force or pressure of the contact is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of contact as an attribute of user input allows a user access to additional device functionality that may not otherwise be accessible by the user on reduced-size devices with limited area for displaying affordances (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls such as knobs or buttons).

[0069] As used herein and in the claims, the term “tactile output” refers to a physical displacement of a device relative to its previous position, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, that will be detected by a user with the user's sense of touch. For example, in a situation where a device or a component of the device is in contact with a touch-sensitive user surface (e.g., the fingers, palm, or other part of the user's hand), the tactile output produced by the physical displacement is interpreted by the user as a tactile sensation that corresponds to a perceived change in the physical properties of the device or component of the device. For example, movement of the touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a “downclick” or “upclick” of a physical actuator button. In some cases, a user feels a tactile sensation such as a “downclick” or “upclick” even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's action. As another example, movement of the touch-sensitive surface is optionally interpreted or perceived by a user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. A user's interpretation of such touch depends on the user's personal sensory perception, but there are many sensory perceptions of touch that are common to the majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., "upclick," "downclick," "roughness"), unless otherwise specified, the generated tactile output corresponds to a physical displacement of the device, or a component of the device, that produces the described sensory perception for a typical (or average) user.

[0070] It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of components. The various components shown in Figure 1A are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and / or application specific integrated circuits.

[0071] Memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0072] A peripheral interface 118 can be used to connect input and output peripherals of the device to the CPU 120 and memory 102. The one or more processors 120 operate or execute various software programs and / or instruction sets stored in memory 102 to perform various functions for device 100 and to process data. In some embodiments, the peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0073] The RF (radio frequency) circuitry 108 transmits and receives RF signals, also called electromagnetic signals. The RF circuitry 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuitry 108 optionally includes well-known circuitry for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, and memory. The RF circuitry 108 optionally communicates wirelessly with other devices and with networks, such as the Internet, also known as the World Wide Web (WWW), an intranet, and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs). The RF circuitry 108 optionally includes well-known circuitry for detecting near field communication (NFC) fields, such as near field communication radio. The wireless communication optionally uses any of a plurality of communication standards, protocols, and technologies, including Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (WDMA), and the like.access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth®, Bluetooth® Low Energy (BTLE), Wireless Fidelity (Wi-Fi)® (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet message access protocol (IMAP) and / or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP)), Session Initiation Protocol for Instant Messaging and Presence Leveraging (SIP)), and protocols for cellular communication (e.g., cellular datagrams, cellular networks, and mobile devices). Such communication protocols include, but are not limited to, SIMPLE (Simple Message Extensions), Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

[0074] Audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and device 100. Audio circuit 110 receives voice data from peripherals interface 118, converts the voice data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves audible to humans. Audio circuit 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuit 110 converts the electrical signals into voice data and transmits the voice data to peripherals interface 118 for processing. The voice data is optionally retrieved from and / or transmitted to memory 102 and / or RF circuit 108 by peripherals interface 118. In some embodiments, audio circuit 110 further includes a headset jack (e.g., 212 in FIG. 2 ). The headset jack provides an interface between audio circuitry 110 and a removable audio input / output peripheral, such as an output-only headphone or a headset with both an output (e.g., single or double ear headphones) and an input (e.g., a microphone).

[0075] I / O subsystem 106 couples input / output peripherals of device 100, such as touchscreen 112 and other input control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes one or more input controllers 160 for display controller 156, light sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and other input or control devices. One or more input controllers 160 receive / send electrical signals from / to other input control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, input controller(s) 160 are optionally coupled to any (or none) of a keyboard, infrared port, USB port, and pointer device such as a mouse. The one or more buttons (e.g., 208 in FIG. 2) optionally include up / down buttons for adjusting the volume of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button (e.g., 206 in FIG. 2).

[0076] A quick press of a push button optionally unlocks the touchscreen 112 or optionally initiates a process of using gestures on the touchscreen to unlock the device, as described in U.S. Patent Application No. 11 / 322,549, filed December 23, 2005, U.S. Patent No. 7,657,849, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," which is incorporated herein by reference in its entirety. A longer press of a push button (e.g., 206) optionally powers the device 100 on or off. The functions of one or more of the buttons are optionally user-customizable. The touchscreen 112 is used to implement virtual or soft buttons and one or more soft keyboards.

[0077] Touch-sensitive display 112 provides an input and output interface between the device and a user. Display controller 156 receives and / or sends electrical signals to touchscreen 112. Touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, optionally, some or all of the visual output corresponds to user-interface objects.

[0078] Touchscreen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic and / or tactile contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction sets in memory 102) detects contact (and any movement or cessation of contact) on touchscreen 112 and translates the detected contact into interaction with user interface objects (e.g., one or more softkeys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the point of contact between touchscreen 112 and the user corresponds to the user's finger.

[0079] Touchscreen 112 optionally uses LCD (liquid crystal display), LPD (light emitting polymer display), or LED (light emitting diode) technology, although other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally detect contact and any movement or disruption thereof using any of a number of now known or later developed touch sensing technologies, including, but not limited to, capacitive, resistive, infrared, and surface ultrasonic technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as found in the iPhone® and iPod Touch® by Apple Inc. of Cupertino, California.

[0080] The touch-sensitive display in some embodiments of touchscreen 112 is optionally similar to the multi-touch-sensing touchpads described in the following U.S. Patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman), and / or U.S. Patent Application Publication No. 2002 / 0015024 A1, each of which is incorporated by reference in its entirety herein. However, touchscreen 112 displays visual output from device 100, whereas touch-sensitive touchpads do not provide visual output.

[0081] Touch-sensitive displays in some embodiments of touchscreen 112 are described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, entitled "Multipoint Touch Surface Controller"; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, entitled "Multipoint Touchscreen"; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, entitled "Gestures For Touch Sensitive Input Devices"; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, entitled "Gestures For Touch Sensitive Input Devices"; (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, entitled "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices"; (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, entitled "Virtual Input Device Placement On A Touch Screen User Interface"; (7) US Patent No. 11 / 228,700, filed September 16, 2005, entitled "Operation Of A Computer With A Touch Screen Interface," (8) US Patent No. 11 / 228,737, filed September 16, 2005, entitled "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard," and (9) US Patent No. 11 / 367,749, filed March 3, 2006, entitled "Multi-Functional Hand-Held Device," all of which are incorporated herein by reference in their entirety.

[0082] Touchscreen 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. A user optionally contacts touchscreen 112 using any suitable object or appendage, such as a stylus, a finger, or the like. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures, which may be less precise than stylus-based input due to the large contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor position or commands to perform the user's desired action.

[0083] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating certain functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touchscreen 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0084] Device 100 also includes a power system 162 for providing power to the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with generating, managing, and distributing electrical power within a portable device.

[0085] Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor connected to light sensor controller 158 in I / O subsystem 106. Light sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. Light sensor 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), light sensor 164 optionally captures still images or video. In some embodiments, a light sensor is located on the back of device 100, opposite touchscreen display 112 on the front of the device, to enable the touchscreen display as a viewfinder for still and / or video image acquisition. In some embodiments, the light sensor is optionally located on the front of the device to obtain an image of the user for a videoconference while the user simultaneously views other videoconference participants on the touchscreen display. In some embodiments, the position of the light sensor 164 can be changed by the user (e.g., by rotating the lens and sensor within the device housing), thereby using a single light sensor 164 in conjunction with a touchscreen display for both video conferencing and capturing still and / or video images.

[0086] Device 100 also optionally includes one or more depth camera sensors 175. FIG. 1A shows a depth camera sensor coupled to camera controller 169 in I / O subsystem 106. Depth camera sensor 175 receives data from the environment and generates a three-dimensional model of an object (e.g., a face) in a scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, depth camera sensor 175 is optionally used in conjunction with imaging module 143 (also referred to as a camera module) to determine a depth map of different portions of an image captured by imaging module 143. In some embodiments, a depth camera sensor is located on the front of device 100 to optionally obtain images of a user with depth information for video conferencing, such as allowing a user to capture a selfie using depth map data while viewing other video conference participants on a touchscreen display. In some embodiments, depth camera sensor 175 is located on the back of the device, or on the back and front of device 100. In some embodiments, the position of the depth camera 175 can be changed by the user (e.g., by rotating the lens and sensor within the device housing), and the depth camera sensor 175 can be used in conjunction with a touchscreen display for both video conferencing and capturing still and / or video images.

[0087] In some embodiments, a depth map (e.g., a depth map image) contains information (e.g., values) about the distance of objects in a scene from a viewpoint (e.g., a camera, light sensor, depth camera sensor). In one embodiment of a depth map, each depth pixel defines a location in the z-axis of the viewpoint where its corresponding two-dimensional pixel is located. In some embodiments, a depth map is composed of pixels, where each pixel is defined by a value (e.g., 0 to 255). For example, a value of "0" represents the pixel located furthest in a "three-dimensional" scene, and a value of "255" represents the pixel closest to the viewpoint (e.g., a camera, light sensor, depth camera sensor) in the "three-dimensional" scene. In other embodiments, a depth map represents the distance between objects in a scene and the plane of the viewpoint. In some embodiments, a depth map contains information about the relative depth of various features of an object of interest in the viewpoint of a depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears on a user's face). In some embodiments, a depth map contains information that enables a device to determine the contours of an object of interest in the z-direction.

[0088] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 in I / O subsystem 106. Contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor 165 receives contact intensity information (e.g., pressure information or a proxy for pressure information) from the surrounding environment. In some embodiments, at least one contact intensity sensor is located on or proximate to the touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite touchscreen display 112, which is located on the front of device 100.

[0089] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is optionally connected to input controller 160 within I / O subsystem 106. Proximity sensor 166 may optionally perform as described in U.S. patent application Ser. No. 11 / 241,839, "Proximity Detector In Handheld Device," Ser. No. 11 / 240,788, "Proximity Detector In Handheld Device," Ser. No. 11 / 620,702, "Using Ambient Light Sensor To Augment Proximity Sensor Output," Ser. No. 11 / 586,862, "Automated Response To And Sensing Of User Activity In Portable Devices," and Ser. No. 11 / 638,251, "Methods And Systems For Automatic Configuration Of Peripherals," which are incorporated herein by reference in their entireties. In some embodiments, the proximity sensor turns off and disables touchscreen 112 when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call).

[0090] Device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows tactile output generators coupled to haptic feedback controller 161 in I / O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile output on the device). Contact intensity sensor 165 receives haptic feedback generation instructions from haptic feedback module 133 and generates a tactile output on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is located on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112) and, optionally, generates a tactile output by moving the touch-sensitive surface vertically (e.g., in / out of the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100, opposite touchscreen display 112, which is located on the front of device 100.

[0091] Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows accelerometer 168 coupled to peripherals interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I / O subsystem 106. Accelerometer 168 optionally functions as described in U.S. Patent Application Publication Nos. 20050190059, "Acceleration-based Theft Detection System for Portable Electronic Devices," and 20060017692, "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer," both of which are incorporated herein by reference in their entireties. In some embodiments, information is displayed on the touchscreen display in portrait or landscape orientation based on an analysis of data received from the one or more accelerometers. In addition to accelerometer(s) 168, device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for obtaining information about the position and orientation (e.g., vertical or horizontal) of device 100.

[0092] In some embodiments, the software components stored in memory 102 include an operating system 126, a communications module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and applications (or instruction sets) 136. Additionally, in some embodiments, as shown in FIGS. 1A and 3, memory 102 (FIG. 1A) or memory 370 (FIG. 3) stores device / global internal state 157. Device / global internal state 157 includes one or more of the following: an active application state, indicating which applications, if any, are currently active; a display state, indicating which applications, views, or other information are occupying various regions of touchscreen display 112; a sensor state, including information obtained from the device's various sensors and input control devices 116; and location information regarding the device's position and / or orientation.

[0093] Operating system 126 (e.g., an embedded operating system such as Darwin®, RTXC®, LINUX®, UNIX®, OS X®, iOS®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0094] Communications module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by RF circuitry 108 and / or external port 124. External port 124 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) is adapted to couple to other devices directly or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector the same as or similar to, and / or compatible with, the 30-pin connector used on iPod® (trademark of Apple Inc.) devices.

[0095] Contact / motion module 130 (in cooperation with display controller 156) optionally detects contact with touch screen 112 and other touch-sensing devices (e.g., a touchpad or physical click wheel). Contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact occurs (e.g., detecting a finger-down event), determining the intensity of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is contact movement and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-drag events), and determining whether the contact has stopped (e.g., detecting a finger-up event or an interruption of the contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by the series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions are optionally applied to a single contact (e.g., a single finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, contact / motion module 130 and display controller 156 detect contacts on the touchpad.

[0096] In some embodiments, contact / motion module 130 uses a set of one or more intensity thresholds for determining whether an action has been performed by a user (e.g., for determining whether a user has “clicked” on an icon). In some embodiments, at least a subset of the intensity thresholds are determined according to software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without modifying the physical hardware of device 100). For example, the mouse “click” threshold of a trackpad or touchscreen display may be set to any of a wide range of predefined thresholds without modifying the trackpad or touchscreen display hardware. Additionally, in some implementations, a user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once with a system-level click “intensity” parameter).

[0097] Contact / motion module 130 optionally detects gesture input by a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different movements, timing, and / or intensity of the detected contact). Thus, gestures are optionally detected by detecting particular contact patterns. For example, detecting a finger tap gesture includes detecting a finger down event (e.g., at an icon location), followed by detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger down event. As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger down event, followed by detecting one or more finger drag events, followed by detecting a finger lift (lift-off) event.

[0098] Graphics module 132 includes various known software components for rendering and displaying graphics on touchscreen 112 or other display, including components for modifying the visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, video, animation, etc.

[0099] In some embodiments, graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. Graphics module 132 receives one or more codes specifying the graphics to be displayed, along with coordinate data and other graphic characteristic data, as appropriate, from an application or the like, and then generates screen image data for output to display controller 156.

[0100] The tactile feedback module 133 includes various software components that generate instructions used by the tactile output generator(s) 167 to generate tactile outputs at one or more locations on the device 100 in response to user interaction with the device 100.

[0101] Text input module 134 is optionally a component of graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0102] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for use in location-based telephony, to the camera 143 as photo / video metadata, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0103] The application 136 optionally includes the following modules (or sets of instructions), or a subset or superset thereof: • a contacts module 137 (sometimes called an address book or contact list); ●Telephone module 138, ●Videoconferencing module 139, ● an email client module 140; ● Instant messaging (IM) module 141; ●Training support module 142, a camera module 143 for still and / or video images, ● Image management module 144; ●Video player module, ●Music player module, ● Browser module 147, ● Calendar module 148, A widget module 149, optionally including one or more of a weather widget 149-1, a stocks widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5 and other widgets acquired by the user, and a user-created widget 149-6; a widget creator module 150 for creating user-created widgets 149-6; ● Search module 151, A video and music player module 152 that integrates a video player module and a music player module; ● Memo module 153, Map module 154, and / or ●Online video module 155.

[0104] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA® enabled applications, encryption, digital rights management, voice recognition, and voice duplication.

[0105] The contacts module 137, along with the touch screen 112, the display controller 156, the touch module 130, the graphics module 132, and the text input module 134, is optionally used to manage an address book or contact list (e.g., stored in the memory 102 or in the application internal state 192 of the contacts module 137 in the memory 370), including adding a name(s) to the address book, deleting a name(s) from the address book, associating phone number(s), email address(es), street address(es), or other information with a name, associating an image with a name, categorizing and sorting names, providing phone numbers or email addresses to initiate and / or facilitate communication by telephone 138, videoconferencing module 139, email 140, or IM 141, and the like.

[0106] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to enter a series of characters corresponding to a telephone number, access one or more telephone numbers in contacts module 137, modify an entered telephone number, dial the corresponding telephone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As noted above, wireless communication optionally uses any of a number of communication standards, protocols, and technologies.

[0107] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, light sensor 164, light sensor controller 158, touch / motion module 130, graphics module 132, text input module 134, contacts module 137, and telephone module 138, videoconferencing module 139 includes executable instructions for initiating, conducting, and terminating videoconferences between a user and one or more other participants in accordance with the user's instructions.

[0108] In cooperation with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, email client module 140 contains executable instructions for composing, sending, receiving, and managing emails in response to user instructions. In cooperation with image management module 144, email client module 140 makes it very easy to compose and send emails with still or video images captured by camera module 143.

[0109] In cooperation with RF circuitry 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for entering text corresponding to an instant message, modifying entered text, sending the corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, sent and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0110] In conjunction with the RF circuitry 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, the training support module 142 creates workouts (e.g., with time, distance, and / or calorie burn goals), communicates with training sensors (sports devices), receives training sensor data, calibrates sensors used to monitor workouts, selects and plays music for workouts, and displays, stores, and transmits workout data.

[0111] In conjunction with touch screen 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, camera module 143 contains executable instructions to capture still images or video (including video streams) and store them in memory 102, change the characteristics of the still images or video, or delete the still images or video from memory 102.

[0112] In conjunction with touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing) or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.

[0113] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, browser module 147 contains executable instructions for browsing the Internet according to user instructions, including retrieving, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0114] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 contains executable instructions to create, display, modify, and store calendars and data associated with calendars (e.g., calendar entries, to-do lists, etc.) in accordance with user instructions.

[0115] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is a mini-application (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) that is optionally downloaded and used by a user, or a mini-application (e.g., user-created widget 149-6) created by a user. In some embodiments, a widget includes a Hypertext Markup Language (HTML) file, a Cascading Style Sheets (CSS) file, and a JavaScript file. In some embodiments, a widget includes an Extensible Markup Language (XML) file and a JavaScript file (e.g., Yahoo! Widget).

[0116] In conjunction with RF circuitry 108, touch screen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 is optionally used by a user to create widgets (e.g., turn user-specified portions of a web page into widgets).

[0117] In cooperation with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, search module 151 contains executable instructions for searching memory 102 for text, music, sound, images, video, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with a user's instructions.

[0118] In conjunction with touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 contains executable instructions that enable a user to download and play pre-recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing videos (on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (a trademark of Apple Inc.).

[0119] In conjunction with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, notes module 153 contains executable instructions for creating and managing notes, to-do lists, and the like, in accordance with user instructions.

[0120] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is optionally used to receive, display, modify, and store maps and data associated with maps (e.g., driving directions, data for businesses and other points of interest at or near a particular location, and other location-based data) in accordance with user instructions.

[0121] In cooperation with touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, online video module 155 contains instructions that enable a user to access, view, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen or on an external display connected via external port 124), and send and otherwise manage emails containing links to particular online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module 141 is used to send links to particular online videos, rather than email client module 140. Additional description of online video applications can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," the contents of which are incorporated by reference herein in their entireties.

[0122] Each of the above-identified modules and applications corresponds to an executable instruction set and method described herein (e.g., computer-implemented methods and other information processing methods described herein) that performs one or more of the above functions. These modules (e.g., instruction sets) need not be implemented as separate software programs, procedures, or modules; thus, various embodiments optionally combine or otherwise rearrange various subsets of these modules. For example, a video player module is optionally combined with a music player module into a single module (e.g., video and music player module 152 of FIG. 1A ). In some embodiments, memory 102 optionally stores a subset of the above-identified modules and data structures. Additionally, memory 102 optionally stores additional modules and data structures not described above.

[0123] In some embodiments, device 100 is a device on which operation of a predefined set of functions is performed exclusively through a touchscreen and / or touchpad. By using the touchscreen and / or touchpad as the primary input control device for operation of device 100, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.

[0124] The set of predefined functions performed exclusively through the touchscreen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 to a main menu, home menu, or root menu from any user interface displayed on device 100. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

[0125] 1B is a block diagram illustrating exemplary components for event processing, according to some embodiments. In some embodiments, memory 102 (FIG. 1A) or memory 370 (FIG. 3) includes event sorter 170 (e.g., in operating system 126) and corresponding application 136-1 (e.g., any of applications 137-151, 155, 380-390 described above).

[0126] Event sorter 170 receives the event information and determines application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event sorter 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192 that indicates the current application view(s) that are displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event sorter 170 to determine which application(s) are currently active, and application internal state 192 is used by event sorter 170 to determine which application(s) are currently active, and application internal state 192 is used by event sorter 170 to determine which application view(s) to deliver the event information to.

[0127] In some embodiments, application internal state 192 includes additional information such as one or more of resume information used when application 136-1 resumes execution, user interface state information indicating information being displayed or prepared for display by application 136-1, a state queue to allow the user to return to a previous state or view of application 136-1, and a redo / undo queue of previous actions taken by the user.

[0128] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user touch on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer(s) 168, and / or microphone 113 (through audio circuitry 110). Information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

[0129] In some embodiments, event monitor 171 sends requests to peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, peripherals interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or for longer than a predetermined duration).

[0130] In some embodiments, event sorter 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173 .

[0131] Hit view determination module 172 provides software procedures for determining where a sub-event occurred within one or more views when touch-sensitive display 112 displays more than one view. A view consists of controls and other elements that a user can see on the display.

[0132] Another aspect of a user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of each application) in which touches are detected optionally correspond to programmatic levels within the application's programmatic or view hierarchy. For example, the lowest-level view in which touches are detected is optionally referred to as a hit view, and the set of events that are recognized as appropriate inputs is optionally determined based at least in part on the hit view of the initial touch that initiates the touch-based gesture.

[0133] Hit view determination module 172 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should process the sub-events. In most situations, the hit view is the lowest-level view in which the first sub-event occurs (e.g., the first sub-event in a sequence of sub-events that form an event or potential event). Once a hit view is identified by hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0134] Active event recognizer determination module 173 determines which view(s) in the view hierarchy should receive the particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive the particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that contain the physical location of the sub-event are actively participating views, and therefore, all actively participating views should receive the particular sequence of sub-events. In other embodiments, even if the touch sub-event is completely confined to the region associated with a particular view, views higher in the hierarchy remain actively participating views.

[0135] Event dispatcher module 174 dispatches event information to event recognizers (e.g., event recognizer 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information obtained by corresponding event receivers 182 in an event queue.

[0136] In some embodiments, operating system 126 includes event sorter 170. Alternatively, application 136-1 includes event sorter 170. In still other embodiments, event sorter 170 is a stand-alone module or part of another module stored in memory 102, such as contact / motion module 130.

[0137] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each containing instructions for handling touch events that occur within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more of event recognizers 180 are part of a separate module, such as a user interface kit or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handler 190 includes one or more of data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event sorter 170. Event handler 190 optionally utilizes or invokes data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 includes one or more corresponding event handlers 190. Also, in some embodiments, one or more of the data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0138] A corresponding event recognizer 180 receives event information (e.g., event data 179) from event sorter 170 and identifies an event from the event information. Event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (optionally including sub-event delivery instructions).

[0139] Event receiver 182 receives event information from event sorter 170. The event information includes information about a sub-event, e.g., a touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, the event includes a device rotation from one orientation to another (e.g., from portrait to landscape or vice versa), and the event information includes corresponding information about the device's current orientation (also called the device's attitude).

[0140] The event comparator 184 compares the event information with predefined event or sub-event definitions and determines the event or sub-event, or determines or updates the state of the event or sub-event, based on the comparison. In some embodiments, the event comparator 184 includes an event definition 186. The event definition 186 includes a definition of an event (e.g., a predefined series of sub-events), such as Event 1 (187-1), Event 2 (187-2), etc. In some embodiments, the sub-events in Event 187 include, for example, a touch start, a touch end, a touch movement, a touch stop, and multiple touches. In one example, the definition for Event 1 (187-1) is a double tap on a displayed object. The double tap includes, for example, a first touch on a displayed object during a predetermined phase (touch start), a first lift-off (touch end) during the predetermined phase, a second touch on a displayed object during the predetermined phase (touch start), and a second lift-off (touch end) during the predetermined phase. In another example, the definition of event 2 (187-2) is a drag operation on a displayed object, including, for example, a touch (or contact) on the displayed object during a predetermined phase, a movement of the touch across the touch-sensitive display 112, and a lift-off of the touch (end of the touch). In some embodiments, the event also includes information about one or more associated event handlers 190.

[0141] In some embodiments, event definitions 187 include definitions of events for each user interface object. In some embodiments, event comparator 184 performs a hit test to determine the user interface object associated with the sub-event. For example, in an application view in which three user interface objects are displayed on touch-sensitive display 112, if a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, event comparator 184 uses the results of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and object that triggers the hit test.

[0142] In some embodiments, the definition for the corresponding event (187) also includes a delay action that delays delivery of the event information until after it has been determined whether a set of sub-events corresponds to the event recognizer's event type.

[0143] When the corresponding event recognizer 180 determines that the sequence of sub-events does not match any of the events in the event definition 186, the corresponding event recognizer 180 enters an event disabled, event failed, or event finished state and thereafter ignores the next sub-event of the touch-based gesture. In this situation, other event recognizers, if any, that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0144] In some embodiments, corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate to actively participating event recognizers how the event delivery system should perform sub-event delivery. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or are allowed to interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are delivered to various levels in the view or programmatic hierarchy.

[0145] In some embodiments, each event recognizer 180 activates an event handler 190 associated with an event when one or more specific sub-events of the event are recognized. In some embodiments, each event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is separate from sending (and deferring sending) the sub-events to the respective hit view. In some embodiments, the event recognizer 180 pops a flag associated with the recognized event, and the event handler 190 associated with the flag catches the flag and performs predefined processing.

[0146] In some embodiments, event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with the set of sub-events or to an actively participating view. The event handler associated with the set of sub-events or to an actively participating view receives the event information and performs predefined processing.

[0147] In some embodiments, data updater 176 creates and updates data used by application 136-1. For example, data updater 176 updates phone numbers used in contacts module 137 or stores video files used in a video player module. In some embodiments, object updater 177 creates and updates objects used by application 136-1. For example, object updater 177 creates new user interface objects or updates the positions of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on the touch-sensitive display.

[0148] In some embodiments, event handler(s) 190 include or have access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the respective application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0149] It should be understood that the foregoing discussion regarding event processing of a user's touch on a touch-sensitive display also applies to other forms of user input for operating multifunction device 100 with input devices, not all of which are initiated on the touchscreen. For example, mouse movements and mouse button presses, optionally coordinated with single or multiple keyboard presses or holds, touch movements such as tapping, dragging, scrolling on a touchpad, pen stylus input, device movement, verbal commands, detected eye movements, biometric input, and / or any combination thereof, are optionally utilized as inputs corresponding to sub-events that define the event to be recognized.

[0150] FIG. 2 illustrates portable multifunction device 100 having touchscreen 112, according to some embodiments. The touchscreen optionally displays one or more graphics within user interface (UI) 200. In this embodiment, as well as embodiments described below, a user can select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers 202 (not drawn to scale) or one or more styluses 203 (not drawn to scale). In some embodiments, selection of one or more graphics occurs when the user loses contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, upward and / or downward), and / or rolling of a finger in contact with device 100 (right to left, left to right, upward and / or downward). In some implementations or situations, accidental contact with a graphic does not select the graphic. For example, a swipe gesture sweeping over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0151] Device 100 also optionally includes one or more physical buttons, such as a "home" or menu button 204. As mentioned above, menu button 204 is optionally used for navigation to any application 136 in a set of applications optionally running on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0152] In some embodiments, device 100 includes touchscreen 112, menu button 204, pushbutton 206 for turning power to the device on / off and locking the device, volume control button(s) 208, subscriber identity module (SIM) card slot 210, headset jack 212, and external docking / charging port 124. Pushbutton 206 is optionally used to turn power on / off on the device by pressing and holding the button down for a predetermined time, lock the device by pressing and releasing the button before the predetermined time has elapsed, and / or unlock the device or initiate an unlocking process. In an alternative embodiment, device 100 also receives verbal input through microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of a contact on touchscreen 112 and / or one or more tactile output generators 167 for generating a tactile output for a user of device 100.

[0153] FIG. 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface, according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia playback device, a navigation device, an educational device (such as a child's learning toy), a gaming system, or a control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 includes input / output (I / O) interface 330, which includes display 340, which is typically a touchscreen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 (e.g., similar to tactile output generator(s) 167 described above with reference to FIG. 1A ) for generating tactile output on device 300, sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and / or contact intensity sensors similar to contact intensity sensor(s) 165 described above with reference to FIG. 1A ). Memory 370 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or a subset of, the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (FIG. 1A). Additionally, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, ​​word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, while memory 102 of portable multifunction device 100 (FIG. 1A) optionally does not store these modules.

[0154] 3 are optionally stored in one or more of the aforementioned memory devices. Each of the above-identified modules corresponds to an instruction set that performs the functions described above. The above-identified modules or programs (e.g., instruction sets) optionally need not be implemented as separate software programs, procedures, or modules; thus, various embodiments combine or otherwise rearrange various subsets of these modules. In some embodiments, memory 370 optionally stores a subset of the above-identified modules and data structures. Additionally, memory 370 optionally stores additional modules and data structures not described above.

[0155] Attention is now directed to embodiments of user interfaces that are optionally implemented on portable multifunction device 100, for example.

[0156] 4A shows an exemplary user interface for a menu of applications on portable multifunction device 100, according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof: signal strength indicator(s) 402 for wireless communication(s), such as cellular and Wi-Fi signals; ●Time 404, ●Bluetooth (registered trademark) indicator 405, ● Battery status indicator 406, A tray 408 containing icons for frequently used applications, such as: an icon 416 for the phone module 138, labeled "Phone," optionally including an indicator 414 of the number of missed calls or voicemail messages; an icon 418 for the email client module 140, labeled "Mail," optionally including an indicator 410 of the number of unread emails; ○ An icon 420 for the browser module 147, labeled "Browser", and ○ An icon 422 for the video and music player module 152, also referred to as the iPod® (trademark of Apple Inc.) module 152, labeled "iPod," and -Icons for other applications, such as: ○ An icon 424 for the IM module 141 labeled "Messages"; ○ An icon 426 for the calendar module 148, labeled "Calendar"; ○ Icon 428 for the image management module 144, labeled "Photos"; ○ An icon 430 for the camera module 143, labeled "camera"; ○ Icon 432 for the online video module 155, labeled "Online Video"; Icon 434 for stock price widget 149-2, labeled "Stock Price" ○ Icon 436 for map module 154, labeled "Map"; ○ Icon 438 for weather widget 149-1, labeled "Weather" ○ Icon 440 for alarm clock widget 149-4, labeled "Clock" ○ An icon 442 for the training support module 142, labeled "Training Support"; ○ An icon 444 for the notes module 153, labeled "Notes," and An icon 446 for a settings application or module, labeled "Settings," that provides access to settings related to the device 100 and its various applications 136.

[0157] 4A are merely exemplary. For example, icon 422 for video and music player module 152 is labeled "Music" or "Music Player." Other labels are optionally used for various application icons. In some embodiments, the label for each application icon includes the name of the application corresponding to the respective application icon. In some embodiments, the label for a particular application icon is different from the name of the application corresponding to the particular application icon.

[0158] 4B shows an exemplary user interface on a device (e.g., device 300 of FIG. 3) that includes a touch-sensitive surface 451 (e.g., tablet or touchpad 355 of FIG. 3) that is separate from a display 450 (e.g., touchscreen display 112). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of a contact on touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating a tactile output to a user of device 300.

[0159] Although some of the following examples are described with reference to input on touchscreen display 112 (when the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as shown in FIG. 4B . In some embodiments, this touch-sensitive surface (e.g., 451 in FIG. 4B ) has a major axis (e.g., 452 in FIG. 4B ) that corresponds to a major axis (e.g., 453 in FIG. 4B ) on the display (e.g., 450). According to these embodiments, the device detects contact with touch-sensitive surface 451 (e.g., 460 and 462 in FIG. 4B ) at locations that correspond to respective locations on the display (e.g., in FIG. 4B , 460 corresponds to 468 and 462 corresponds to 470). In this manner, when the touch-sensitive surface is separate from the display, user input (e.g., contacts 460 and 462 and their movement) detected by the device on the touch-sensitive surface (e.g., 451 in FIG. 4B ) is used by the device to operate a user interface on the display (e.g., 450 in FIG. 4B ) of the multifunction device. It should be understood that similar methods are optionally used for the other user interfaces described herein.

[0160] Additionally, while the following examples are described primarily with reference to finger input (e.g., finger touches, finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of the finger inputs are replaced with input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced with a mouse click (e.g., instead of a touch), followed by cursor movement along the path of the swipe (e.g., instead of moving the contact). As another example, a tap gesture is optionally replaced with a mouse click while the cursor is positioned over the location of the tap gesture (e.g., instead of detecting a touch followed by ceasing contact detection). Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice are optionally used simultaneously, or a mouse and finger touches are optionally used simultaneously.

[0161] FIG. 5A shows an exemplary personal electronic device 500. Device 500 includes a main body 502. In some embodiments, device 500 may include some or all of the features described with respect to devices 100 and 300 (e.g., FIGS. 1A-4B ). In some embodiments, device 500 includes a touch-sensitive display screen 504, hereafter referred to as touchscreen 504. Instead of, or in addition to, touchscreen 504, device 500 includes a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or the touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). The one or more intensity sensors in touchscreen 504 (or the touch-sensitive surface) can provide output data representing the intensity of the touch. A user interface of device 500 can respond to a touch based on the intensity of the touch, meaning that touches of different intensities can invoke different user interface actions on device 500.

[0162] Exemplary techniques for detecting and processing touch intensity can be found, for example, in related applications, International Patent Application No. PCT / US2013 / 040061, filed May 8, 2013, entitled "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application" (published as WO 2013 / 169849), and International Patent Application No. PCT / US2013 / 069483, filed November 11, 2013, entitled "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships" (published as WO 2014 / 105276), each of which is incorporated herein by reference in its entirety.

[0163] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508, if included, can be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, can allow device 500 to be attached to, for example, hats, eyewear, earrings, necklaces, shirts, jackets, bracelets, watch bands, chains, pants, belts, shoes, wallets, backpacks, etc. These attachment mechanisms can allow device 500 to be worn by a user.

[0164] FIG. 5B illustrates an exemplary personal electronic device 500. In some embodiments, device 500 may include some or all of the components described with respect to FIGS. 1A, 1B, and 3. Device 500 has a bus 512 operably coupling I / O unit 514 to one or more computer processors 516 and memory 518. I / O unit 514 may be connected to a display 504, which may have touch-sensing components 522 and, optionally, an intensity sensor 524 (e.g., a contact intensity sensor). Additionally, I / O unit 514 may connect to a communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near-field communication (NFC), cellular, and / or other wireless communication technologies. Device 500 may include input mechanism 506 and / or input mechanism 508. Input mechanism 506 is optionally, for example, a rotatable input device or a depressible and rotatable input device. Input mechanism 508 is optionally a button in some embodiments.

[0165] Input mechanism 508 is optionally a microphone in some embodiments. Personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which may be operably connected to I / O section 514.

[0166] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 516, may cause the computer processors to perform the techniques described below, including processes 700, 900, 1100, 1300, and 1500 (FIGS. 7A-7B, 9A-9B, 11A-11B, 13A-13B, and 15A-15B). A computer-readable storage medium may be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic, optical, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical disks based on CD, DVD, or Blu-ray technology, and persistent solid-state memory such as flash, solid-state drives, etc. Personal electronic device 500 is not limited to the components and configuration of Figure 5B and may include other or additional components in multiple configurations.

[0167] As used herein, the term "affordance" refers to a user-interactive graphic user interface object that is optionally displayed on a display screen of device 100, 300, and / or 500 (FIGS. 1A, 3, and 5A-5B). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each optionally constitute an affordance.

[0168] As used herein, the term “focus selector” refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations involving a cursor or other position marker, when input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B ) and the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor acts as a “focus selector,” and the particular user interface element is adjusted according to the detected input. In some implementations involving a touchscreen display (e.g., touch-sensitive display system 112 in FIG. 1A or touchscreen 112 in FIG. 4A ) that allows direct interaction with user interface elements on the touchscreen display, contact detected on the touchscreen acts as a “focus selector,” such that when input (e.g., a press input from a contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one region of the user interface to another region of the user interface (e.g., by using the tab key or arrow keys to move focus from one button to another) without corresponding cursor movement or contact movement on the touchscreen display, and in those implementations, the focus selector moves to follow the movement of focus between different regions of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally a user interface element (or contact on a touchscreen display) that is controlled by the user to communicate the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user intends to interact with).For example, when a pressure input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., cursor, contact, or selection box) over a corresponding button indicates that the user intends to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0169] As used herein and in the claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on a plurality of intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predefined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) relative to a predefined event (e.g., after detecting the contact, before detecting lift-off of the contact, before or after detecting the start of contact movement, before detecting the end of the contact, before or after detecting an increase in the intensity of the contact, and / or before or after detecting a decrease in the intensity of the contact). The characteristic intensity of the contact is optionally based on one or more of the maximum intensity of the contact, the mean value of the intensity of the contact, the average value of the intensity of the contact, the top 10% of the intensity of the contact, half the maximum intensity of the contact, 90% of the maximum intensity of the contact, etc. In some embodiments, the duration of the contact is used to determine the characteristic intensity (e.g., when the characteristic intensity is an average of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action is performed by the user. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact having a characteristic intensity that does not exceed the first threshold results in a first action being performed, a contact having a characteristic intensity that exceeds the first intensity threshold but not the second intensity threshold results in a second action being performed, and a contact having a characteristic intensity that exceeds the second threshold results in a third action being performed. In some embodiments, the comparison between the characteristic intensity and the one or more thresholds is not used to determine whether to perform the first or second action, but rather to determine whether to perform one or more actions (e.g., whether to perform each action or to forgo performing each action).

[0170] In some embodiments, portions of the gesture are identified for purposes of determining the characteristic intensity. For example, the touch-sensitive surface optionally receives successive swipe contacts that transition from a start position to an end position, where the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end position is optionally based on only a portion of the swipe contact (e.g., only the portion of the swipe contact at the end position), rather than the entire successive swipe contact. In some embodiments, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some situations, the smoothing algorithms remove small increases or decreases in the intensity of the swipe contact for purposes of determining the characteristic intensity.

[0171] The intensity of the contact on the touch-sensitive surface is optionally characterized against one or more intensity thresholds, such as a contact-detection intensity threshold, a light pressure intensity threshold, a deep pressure intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light pressure intensity threshold corresponds to an intensity at which the device performs an operation typically associated with clicking a physical mouse button or trackpad. In some embodiments, the deep pressure intensity threshold corresponds to an intensity at which the device performs an operation different from an operation typically associated with clicking a physical mouse button or trackpad. In some embodiments, if the contact is detected with a characteristic intensity below the light pressure intensity threshold (e.g., and above a slight contact-detection intensity threshold below which the contact is no longer detected), the device moves a focus selector in accordance with the movement of the contact on the touch-sensitive surface without performing an action associated with the light pressure intensity threshold or the deep pressure intensity threshold. Generally, unless otherwise specified, these intensity thresholds are consistent across different sets of user interface diagrams.

[0172] An increase in the characteristic intensity of the contact from an intensity below the light pressure intensity threshold to an intensity between the light pressure intensity threshold and the deep pressure intensity threshold may be referred to as a "light press" input. An increase in the characteristic intensity of the contact from an intensity below the deep pressure intensity threshold to an intensity above the deep pressure intensity threshold may be referred to as a "deep press" input. An increase in the characteristic intensity of the contact from an intensity below the contact-detection intensity threshold to an intensity between the contact-detection intensity threshold and the light pressure intensity threshold may be referred to as detecting a contact on the touch surface. A decrease in the characteristic intensity of the contact from an intensity above the contact-detection intensity threshold to an intensity below the contact-detection intensity threshold may be referred to as detecting a lift-off of the contact from the touch surface. In some embodiments, the contact-detection intensity threshold is zero. In some embodiments, the contact-detection intensity threshold is greater than zero.

[0173] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including the respective pressure input or in response to detecting the respective pressure input performed on the respective contact (or contacts), where the respective pressure inputs are detected based at least in part on detecting an increase in intensity of the contact (or contacts) above a pressure input intensity threshold. In some embodiments, the respective actions are performed in response to detecting an increase in intensity of the respective contact (e.g., a "downstroke" of the respective pressure input) above the pressure input intensity threshold. In some embodiments, the pressure input includes an increase in intensity of the respective contact above the pressure input intensity threshold and a subsequent decrease in intensity of the contact below the pressure input intensity threshold, where the respective actions are performed in response to detecting a subsequent decrease in intensity of the respective contact (e.g., an "upstroke" of the respective pressure input) below the pressure input intensity threshold.

[0174] In some embodiments, the device employs intensity hysteresis to avoid accidental inputs, sometimes referred to as “jitter,” and the device defines or selects a hysteresis intensity threshold that has a predefined relationship to the pressure input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units below the pressure input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the pressure input intensity threshold). Thus, in some embodiments, the pressure input includes an increase in the intensity of each contact above the pressure input intensity threshold and a subsequent decrease in the intensity of each contact below the hysteresis intensity threshold corresponding to the pressure input intensity threshold, and a respective action is performed in response to detecting a subsequent decrease in the intensity of each contact below the hysteresis intensity threshold (e.g., an “upstroke” of each pressure input). Similarly, in some embodiments, a pressure input is detected only when the device detects an increase in the intensity of the contact from an intensity below the hysteresis intensity threshold to an intensity above the pressure input intensity threshold, and optionally a subsequent decrease in the intensity of the contact to an intensity below the hysteresis intensity, and a respective action is performed in response to detection of that pressure input (e.g., an increase in the intensity of the contact or a decrease in the intensity of the contact, as the case may be).

[0175] For ease of explanation, descriptions of actions performed in response to a pressure input associated with a pressure input intensity threshold or in response to a gesture including a pressure input are, optionally, triggered in response to detecting any of: an increase in the intensity of the contact above the pressure input intensity threshold; an increase in the intensity of the contact from an intensity below a hysteresis intensity threshold to an intensity above the pressure input intensity threshold; a decrease in the intensity of the contact below the pressure input intensity threshold; and / or a decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to the pressure input intensity threshold. Additionally, in examples where an action is described as being performed in response to detecting a decrease in the intensity of the contact below a pressure input intensity threshold, the action is optionally performed in response to detecting a decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to and lower than the pressure input intensity threshold.

[0176] Attention is now directed to embodiments of user interfaces (“UIs”) and associated processes implemented on an electronic device such as portable multifunction device 100, device 300, or device 500.

[0177] 6A-6BQ illustrate example user interfaces for displaying visual effects in a messaging application, according to some embodiments, and are used to illustrate processes described below, including those in FIGS.

[0178] FIG. 6A depicts device 600 having display 601, in some cases a touch-sensitive display. In some embodiments, device 600 also includes camera 602, which minimally includes an image sensor capable of capturing data representing a portion of the light spectrum (e.g., visible light, infrared light, or ultraviolet light). In some embodiments, camera 602 includes multiple image sensors and / or other types of sensors. In some embodiments, camera 602, in addition to capturing data representing detected light, can capture other types of data, such as depth data. For example, in some embodiments, camera 602 also captures depth data using speckle, time-of-flight, parallax, or focus-based techniques. Image data that device 600 captures using camera 602 includes data corresponding to a portion of the light spectrum for a scene within the camera's field of view. In addition, in some embodiments, the captured image data also includes depth data of the light data. In some other embodiments, the captured image data includes sufficient data to determine or generate depth data for data of a portion of the light spectrum. In some embodiments, device 600 includes one or more features of devices 100, 300, or 500.

[0179] In some examples, the electronic device 600 includes a depth camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device (e.g., a light projector), such as an IR flood light, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate the subject during image capture by the visible light camera and the depth camera (e.g., the IR camera), and information from the depth camera and the visible light camera is used to determine a depth map of different portions of the subject captured by the visible light camera. In some embodiments, the depth map (e.g., a depth map image) includes information (e.g., values) about the distance of objects in the scene from the viewpoint (e.g., the camera). In one depth map embodiment, each depth pixel defines a position in the Z-axis of the viewpoint where its corresponding two-dimensional pixel is located. In some examples, the depth map is composed of pixels, each pixel defined by a value (e.g., 0 to 255). For example, a value of "0" represents the pixel located furthest away in a "3D" scene, and a value of "255" represents the pixel in the "3D" scene closest to the viewpoint (e.g., camera). In other examples, the depth map represents the distance between an object in a scene and the plane of the viewpoint. In some embodiments, the depth map includes information about the relative depth of various features of an object of interest in the viewpoint of the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of a user's face). In some embodiments, the depth map includes information that enables a device to determine the contours of the object of interest in the z-direction. In some embodiments, the lighting effects described herein are displayed using disparity information from two cameras (e.g., two visible light cameras) for the rear image and using depth information from the depth camera combined with image data from the visible light camera for the front image (e.g., a selfie image). In some embodiments, the same user interface is used when determining depth information using two visible light cameras and when determining depth information using a depth camera, providing a consistent experience to the user even when using significantly different techniques for determining information used in generating lighting effects.In some embodiments, while displaying a camera user interface with one of the lighting effects applied, the device detects selection of a camera switching affordance and switches from the front camera (e.g., a depth camera and a visible light camera) to the rear camera (e.g., two visible light cameras spaced apart) (or vice versa) while applying the lighting effect and maintaining display of user interface controls for replacing the display of the front camera's field of view with the rear camera's field of view (or vice versa).

[0180] 6A , device 600 displays a messaging user interface 603 of a messaging application. Messaging user interface 603 includes a message display area 604 that includes a message 605 to be transmitted to participants (represented by recipient identifiers 606) in a message conversation. Messaging user interface 603 also includes a message composition field 608 for displaying input (e.g., text input, multimedia input, etc.) to send to participants in the message conversation. Messaging user interface 603 also includes a camera application affordance 609, an application dock affordance 610, a keyboard display area 612, and a text suggestion area 614.

[0181] In FIG. 6B, device 600 detects input 616 (eg, touch input on display 601) at a location corresponding to camera application affordance 609.

[0182] 6C , in response to detecting input 616, device 600 launches a camera application associated with camera application affordance 609 and displays camera application user interface 615. Camera application user interface 615 includes an image display area 620 that displays a representation of image data, such as, for example, streamed image data (e.g., a live camera preview, a live camera recording, or a live video communication session) representing objects located within the field of view of a camera (e.g., the rear-facing camera or camera 602), or media items, such as, for example, photos or video recordings. In the embodiment shown in FIG. 6C , image display area 620 shows a live camera preview 620-1′ from the rear-facing camera of device 600.

[0183] Camera application user interface 615 also includes an area above image display area 620 that includes camera-specific affordances 617 and a completion affordance 618 for exiting camera application user interface 615. The camera-specific affordances include affordance 617-1 associated with a camera flash function, affordance 617-2 associated with a camera mode function, affordance 617-3 associated with a timer function, and affordance 617-4 associated with a filter function.

[0184] Camera application user interface 615 also includes a camera options region 625 disposed below image display region 620. Camera options region 625 includes a camera selector affordance 627 for switching between cameras (e.g., the rear camera and camera 602) and camera options affordances 619 associated with different capture modes in which the camera can record image data. For example, video affordance 619-1 is associated with functionality for activating the camera's video recording capture mode, and photo affordance 619-2 is associated with functionality for activating the camera's still image capture mode. In the embodiments discussed below with respect to FIGS. 6C-6AX, device 600 is in the still image capture mode of operation associated with photo affordance 619-2. However, unless otherwise noted, these embodiments also apply to the video recording mode associated with video affordance 619-1.

[0185] Camera options region 625 further includes effects affordance 622 for enabling or disabling a mode of device 600 in which device 600 is enabled or disabled to display visual effects in image display region 620. This mode of device 600 is sometimes referred to herein as effects mode.

[0186] Camera options region 625 also includes a capture affordance 621 that can be selected to capture image data represented in image display region 620. In some embodiments, device 600 captures image data in a manner based on the currently enabled capture mode (e.g., video recording capture mode or image capture mode). In some embodiments, device 600 captures image data depending on the type of gesture detected on capture affordance 621. For example, if device 600 detects a tap gesture on capture affordance 621, device 600 captures a still image of the image data represented in image display region 620 at the time the tap gesture occurred. If device 600 detects a tap-and-hold gesture on capture affordance 621, device 600 captures a video recording of the image data represented in image display region 620 for the duration of the tap-and-hold gesture. In some embodiments, video recording stops when a finger is lifted from the affordance. In some embodiments, the video recording continues until a subsequent input (e.g., a tap input) is detected at a location corresponding to the affordance. In some embodiments, the captured image (e.g., a still image or a video recording) is inserted into the message composition field 608 and subsequently sent to the participants in the message conversation. In some embodiments, the captured image is sent directly to the participants in the message conversation without inserting the captured image into the message composition field 608.

[0187] In FIG. 6D, device 600 detects input 623 on effect affordance 622 (e.g., a tap gesture on display 601).

[0188] 6E , in response to detecting input 623, device 600 activates camera 602 (e.g., switches from the rear-facing camera) and updates image display area 620 to display a live camera preview 620-1 from camera 602, showing a representation of object 632 located within the field of view of camera 602 and a background 636 displayed behind object 632. As discussed herein, in some embodiments, image data captured using camera 602 includes depth data that can be used to determine the depth of objects within the field of view of camera 602. In some embodiments, device 600 analyzes objects (e.g., in the image data) based on the detected depths of these objects and uses this determination to apply visual effects as discussed herein. For example, device 600 can classify object 632 as being in the foreground of live camera preview 620-1 and objects located behind the user as being in the background of live camera preview 620-1. These background objects are collectively referred to herein as background 636.

[0189] Device 600 also highlights effects affordance 622, indicating that visual effects are enabled for display, and updates camera options area 625 by replacing camera options affordance 619 with visual effects option affordance 624. The visual effects option affordances include avatar effects affordance 624-1 and sticker effects affordance 624-2. Visual effects option affordance 624 corresponds to different visual effects that can be applied to the image displayed in image display area 620. Selecting one of the visual effects option affordances (e.g., 624-1 or 624-2) causes a menu to be displayed with visual effect options that correspond to the selected visual effects option affordance.

[0190] A user can enable or disable the effects mode of device 600 by selecting effects affordance 622. When effects affordance 622 is highlighted, the effects mode of device 600 is enabled and displays visual effects in image display area 620. When a user taps on the highlighted affordance 622, effects affordance 622 is no longer highlighted and the effects mode is disabled such that visual effects are not enabled for display in image display area 620. In some embodiments, when effects mode is enabled, device 600 updates the image shown in image display area 620 to display one or more visual effects applied to the image (including visual effects applied to a live image stream), and when effects mode is disabled, device 600 removes or hides the visual effects from the image shown in image display area 620.

[0191] In FIG. 6F, device 600 detects input 626 (eg, a tap gesture on display 601) on avatar effect affordance 624-1.

[0192] In FIG. 6G , in response to detecting input 626, device 600 displays avatar options menu 628 with a scrollable list of avatar options 630. Avatar options menu 628 also includes a selection area 629 for indicating a selection of one of the avatar options 630. As shown in FIG. 6G , robot avatar option 630-3 is located in selection area 629, indicating that robot avatar option 630-1 is selected. Other avatar options 630 shown in FIG. 6G include customized female avatar option 630-1, null avatar option 630-2, rabbit avatar option 630-4, and alien avatar option 630-5. As discussed in more detail below, null avatar option 630-2 corresponds to an avatar option for not displaying an avatar on a subject in image display area 620. In some embodiments, null avatar option 630-2 is the avatar option selected by default when avatar options menu 628 is displayed.

[0193] The avatar options 630 correspond to virtual avatar visual effects that are applied to the subject's representation in the image display area 620. Specifically, when selected, each avatar option 630 corresponds to a virtual avatar that is transposed over the subject's face in the image display area, while other portions of the image in the image display area (e.g., the background or other parts of the user, such as the user's body) remain displayed. A user (e.g., subject 632) positioned within the field of view of camera 602 can control the visual aspects of the virtual avatar by changing the pose (e.g., rotation or orientation) of their face, including the movement of various facial features (e.g., winking, sticking out their tongue, smiling, etc.). Details for controlling the display and movement of virtual avatars are provided in U.S. Patent Application No. 15 / 870,195, which is incorporated herein by reference for all purposes.

[0194] In some embodiments, the virtual avatar is a graphically portrayable representation of the user (e.g., a graphical representation of the user). In some embodiments, the virtual avatar is non-photorealistic (e.g., cartoonish). In some embodiments, the virtual avatar includes an avatar face having one or more avatar features (e.g., avatar facial features). In some embodiments, the avatar features correspond to (e.g., are mapped to) one or more physical features of the user's face such that detected movement of the user's physical features (e.g., as determined based on a camera, such as a depth-sensing camera) affects the avatar feature (e.g., affects the graphical representation of the feature).

[0195] In some examples, a user can manipulate the characteristics or features of a virtual avatar using camera sensors (e.g., camera module 143, light sensor 164) and, optionally, depth sensors (e.g., depth camera sensor 175). As the user's physical characteristics (e.g., facial features) and position (e.g., head position, head rotation, or head tilt) change, the electronic device detects the changes and modifies the displayed image of the virtual avatar to reflect the changes in the user's physical characteristics and position. In some embodiments, changes to the user's physical characteristics and position indicate different expressions, emotions, context, tone, or other non-verbal communication. In some embodiments, the electronic device modifies the displayed image of the virtual avatar to represent these expressions, emotions, context, tone, or other non-verbal communication.

[0196] In some embodiments, the virtual avatar is a customizable avatar (e.g., customizable avatar 835). A customizable avatar is a virtual avatar that a user can select and customize, for example, to achieve a desired appearance (e.g., to look like the user). Customizable avatars generally have the appearance of human characters rather than non-human characters such as anthropomorphic compositions of animals or other non-human objects. Additionally, avatar characteristics can be created or modified as needed using an avatar editing user interface (e.g., the avatar editing user interface discussed below with respect to FIGS. 8AA-8AB). In some embodiments, customizable avatars can be generated and configured to achieve customized physical appearances, physical compositions, or modeled behaviors.

[0197] In some embodiments, the virtual avatar is a non-customizable avatar. Non-customizable avatars are virtual avatars that can be selected by a user, but are generally not inherently configurable, although their appearance can be changed via face tracking, as described in more detail below. Instead, non-customizable avatars are pre-configured and generally do not have feature components that can be changed by a user. In some embodiments, a non-customizable avatar has the appearance of a non-human character, such as an anthropomorphic composition of an animal or other non-human object (see, e.g., robot avatar 633, rabbit avatar 634). A non-customizable avatar cannot be created by a user or modified to achieve significant changes in the physical appearance, physical composition, or modeled behavior of the non-customizable avatar.

[0198] 6G, robot avatar 630-3 has been selected, so device 600 displays robot avatar 633 positioned on the face of subject 632 (e.g., a user) displayed in image display area 620. Device 600 displays the avatar such that it remains fixed on the subject's face as it moves within the field of view of camera 602. Device 600 also continuously modifies the avatar based on detected changes in the user's face, including changes in facial pose or expression, while maintaining the displayed subject's body and background 636 within the image shown in image display area 620.

[0199] 6H , if device 600 detects that the user's head has moved beyond the field of view of camera 602, device 600 displays prompt 638, for example, including a representation 640 of a selected avatar (e.g., a robot avatar) and a message 642 instructing the user to reposition their head within the field of view of camera 602. In some embodiments, prompt 638 is displayed with a blurred background 644 (including blurring user 632 and background 636). In some embodiments, displaying prompt 638 includes displaying an animation of the selected avatar returning to a center position in image display area 620 and showing slowed movement of the avatar and its features that appear to stop based on a physical model of the avatar's features.

[0200] Once device 600 detects the user's head returning to the field of view of camera 602, the device continues to update the selected avatar (e.g., robot avatar 633) based on the detected changes in the user's face, as shown in Figure 61. In some embodiments, once device 600 detects the user's face returning to the field of view of camera 602, device 600 displays avatar 633 moving from a center position in image display area 620 to the position of the user's face, and resumes changing the avatar based on the detected changes in the user's face.

[0201] 6I-6K, device 600 detects input 644 (e.g., a scrolling gesture on display 601) on a displayed list of avatar options 630. As input 644 moves leftward across display 601, device 600 displays avatar options 630 that scroll left to reveal additional avatar options (e.g., a poop avatar option and a fox avatar option). In some embodiments, when one of the avatar options enters selection region 629, the device generates haptic feedback (e.g., a tactile output with or without audio output) to indicate to the user that termination of input 644 at that time may result in selection of the avatar option currently located within selection region 629. For example, in FIG. 6K, device 600 generates haptic feedback 646 when alien avatar option 630-5 is located within selection region 629.

[0202] In FIG. 6J , device 600 detects the end of input 644 and determines that alien avatar option 630-5 is located within selection region 629. As a result, device 600 replaces robot avatar 633 with alien avatar 635 corresponding to the selected alien avatar option 630-5 while maintaining display of the subject's body and background 636. In some embodiments, replacing the previously selected avatar (e.g., robot avatar 633) with the currently selected avatar (e.g., alien avatar 635) includes displaying an animation of the previously selected avatar moving off-screen (e.g., outside of image display region 620) and an animation of the currently selected avatar moving from avatar options menu 628 to the user's face in image display region 620. In some embodiments, face tracking of the currently selected avatar begins as the avatar moves from avatar options menu 628 to the user's face. In some embodiments, the background is blurred while the avatar switching operation is occurring.

[0203] 6M-6N, device 600 detects a vertical gesture 646 (e.g., a vertical swipe or touch-and-drag on display 601) on avatar options menu 628 and, in response, expands avatar options menu 628 to expanded view 628-1, as shown in FIG. 6N. In FIG. 6N, avatar options 630 are arranged in a matrix and can be selected (e.g., gesture 648 selects robot avatar option 630-3) to display a different avatar in image display area 620, as shown in FIG. 6O.

[0204] As shown in FIG. 6P, in response to gesture 648, device 600 displays avatar options menu 628 including selected robot avatar option 630-3 (located in selection area 629) and robot avatar 633 displayed on the user's head within image display area 620 (e.g., by inverting the menu).

[0205] In Figure 6Q, device 600 detects input 649 (e.g., a tap gesture on display 601) on close icon 650 to close application options menu 628 and return to camera user interface 615 shown in Figure 6R, which shows camera options area 625, while maintaining the display of robot avatar 633 of the user's face in an image stream (e.g., a live camera preview) on image display area 620. Camera options area 625 shows capture affordance 621, visual effects options affordance 624, and highlight effects affordance 622. As shown below in Figures 6S and 6T, the user can select avatar menu affordance 624-1 to redisplay avatar options menu 628.

[0206] For example, in Figure 6S, device 600 detects input 652 (e.g., a tap gesture on display 601) on avatar effects affordance 624-1. In Figure 6T, in response to detecting input 652, device 600 displays avatar options menu 628 with avatar options 630 that replaces camera options area 625, which includes capture affordance 621, effects affordance 622, and visual effects options affordance 624. Avatar options menu 628 can be closed again by selecting close affordance 650, as described above, to return again to camera application user interface 615, as shown in Figure 6U.

[0207] In FIG. 6U, device 600 detects input 654 (e.g., a tap gesture on display 601) on sticker effect affordance 624-2. In FIG. 6V, in response to detecting input 654, device 600 displays sticker options menu 656 having a scrollable list of stickers 658. Stickers are static graphical objects that a user can select and apply to an image in image display area 620. In some embodiments, stickers can be selected by a touch-and-drag gesture, starting on the selected sticker, dragging the sticker to the desired location, and then placing the sticker in the desired position by ending the drag (e.g., lifting the finger). In some embodiments, a sticker can be selected by a tap gesture, and then the corresponding sticker is displayed in a position on image display area 620. An example of such an embodiment is shown in FIGS. 6W-6X, which show a tap gesture 660 on helmet sticker 658-1, which is then displayed in the center of image display area 620.

[0208] 6Y-6Z show that a placed sticker (e.g., sticker 658-1) can be repositioned on image display area 620 by dragging the sticker to a new position (see gesture 662 moving sticker 658-1 to a new position in FIG. 6Z).

[0209] FIG. 6AA shows that a placed sticker (eg, sticker 658-1) can be rotated using a two-finger gesture 664.

[0210] Figure 6AB shows that a placed sticker (e.g., sticker 658-1) can be resized using a pinch or de-pinch gesture. In Figure 6AB, helmet sticker 658-1 is enlarged in response to a de-pinch gesture 666. Resized, rotated, and repositioned helmet sticker 658-1 is shown placed on the head of the robot avatar in Figure 6AC.

[0211] In FIG. 6AD, device 600 detects input 668 (e.g., a tap gesture on display 601) on close icon 670 to close sticker options menu 656 and return to camera application user interface 615, which includes camera options area 625 showing capture affordance 621, visual effects options affordance 624, and highlight effects affordance 622 indicating that visual effects are enabled to display, as shown in FIG. 6AE. In FIG. 6AE, device 600 shows a representation of the user in an image on image display area 620, but modified with visual effects including robot avatar 633 (e.g., selected from avatar options menu 628) and helmet sticker 658-1 (e.g., selected from sticker options menu 656). The display of the visual effects is dynamic. For example, device 600 continuously changes robot avatar 633 in response to detected changes in the user's face. In some embodiments, the positioning of sticker 658-1 is also dynamic and changes based on detected movement of the user, as discussed in more detail below.

[0212] When visual effects are enabled for device 600 (e.g., effect affordance 622 is highlighted), deselecting the highlighted effect affordance 622 (e.g., selecting affordance 622 when highlighted to disable the visual effect) can remove or hide the applied visual effects (e.g., avatars and stickers) from image display area 620. For example, in FIG. 6AF, the device detects input 672 (e.g., a tap gesture on display 601) on highlight effect affordance 622. In response, device 600 removes the displayed visual effects (e.g., helmet sticker 658-1 and robot avatar 633 in FIG. 6AF) from image display area 620, as shown in FIG. 6AG, while maintaining the display of an image stream (e.g., live camera preview 620-1) including subject 632 and background 636 within the field of view of camera 602.

[0213] In some embodiments, after the visual effects mode is disabled (e.g., by deselecting highlight effect affordance 622), the removed visual effects can be restored, for example, by reselecting effect affordance 622 within a predetermined time. For example, in FIG. 6AH , the device detects input 674 (e.g., a tap gesture on display 601) on the unhighlight effect affordance 622. In response to detecting input 674, device 600 highlights effect affordance 622 and re-displays helmet sticker 658-1 and robot avatar 633 while continuing to display an image stream (e.g., live camera preview 620-1) including the body of subject 632 and background 636, as shown in FIG. 6AI.

[0214] 6AJ, device 600 detects input 676 (e.g., a tap gesture on display 601) on capture affordance 621. In response to detecting input 676, device 600 captures an image of live camera preview 620-1 of FIG. 6AJ and displays media item 620-2 of FIG. 6AK (e.g., a still image of the state of live camera preview 620-1 at the time device 600 detected input 676 on capture affordance 621).

[0215] In some embodiments, when an image is captured (e.g., stored as a media item), device 600 encodes depth data into the media item. Storing the depth data in the media allows for later application of depth-based effects (e.g., effects based on the position of an object (e.g., a user's face) in the z-direction). In some embodiments, when an image is captured while an effect is applied, the effect is encoded directly into the visual (e.g., RGB) information to improve compatibility with other devices.

[0216] 6AK, device 600 displays camera application user interface 615 showing a captured image (e.g., media item 620-2) in image display area 620 (e.g., media item 620-2 replaces live camera preview 620-1 shown in image display area 620). Media item 620-2 is a representation of live camera preview 620-1 at the time capture affordance 621 was selected. As such, media item 620-2 includes helmet sticker 658-1 and robot avatar 633 displayed over the face of subject 632 and background 636.

[0217] Device 600 also replaces the camera-specific affordance (e.g., affordance 617 shown in FIG. 6AJ) with a re-capture affordance 679 and a recipient identifier 606, which indicates the currently selected recipient of media item 620-2 to which the user should subsequently send the media item, as discussed below.

[0218] The device 600 displays a camera options area 625 that includes a visual effects option affordance 624. The visual effects option affordance 624 can be selected to display a respective options menu that can be used to modify the captured media item 620-2 (as well as the recorded video media item 620-4, described below).

[0219] Device 600 also updates camera options region 625 to replace capture affordance 621 and camera selector affordance 627 with markup affordance 677, edit affordance 678, and send affordance 680. Mark affordance 677 allows the user to mark up media item 620-2. Edit affordance 678 allows the user to edit media item 620-2, such as by cropping the image or adjusting other properties of media item 620-2. As shown in FIG. 6AK, send affordance 680 is displayed in the same location / area where capture affordance 621 was displayed, such that send affordance 680 replaces capture affordance 621 on the display.

[0220] Send affordance 680 allows a user to immediately send media item 620-2 to a recipient indicated by recipient identifier 606. For example, in FIG. 6AL, device 600 detects input 682 on send affordance 680 (e.g., a tap gesture on display 601) and immediately sends media item 620-2 to John, the contact corresponding to recipient identifier 606, as shown in messaging user interface 603 of FIG. 6AM. In some embodiments, device 600 sends media item 620-2 without displaying messaging user interface 603. In some embodiments, such as that shown in FIG. 6AM, the device sends media item 620-2 and displays messaging user interface 603 with media item 620-2 in message display area 604 to indicate that media item 620-2 has been sent to a participant (e.g., John) in the message conversation, and displays application dock 690 positioned above keyboard display area 612. The application dock 690 includes application affordances 692 that can be selected to access different applications.

[0221] In some embodiments, media item 620-2 is not immediately sent to the participants in the messaging conversation. For example, in FIG. 6AN, instead of selecting send affordance 680, the user selects completion affordance 618. In response to detecting input 684 on completion affordance 618 (e.g., a tap gesture on display 601), device 600 displays a representation of media item 685 in message composition field 608 of messaging user interface 603, as shown in FIG. 6AO. In this embodiment (shown in FIGS. 6AO-6AR), instead of displaying application dock 690, device 600 displays text suggestion area 614 along with keyboard display area 612 to allow the user to compose text (e.g., 687) or, optionally, add other message content (e.g., multimedia content) before sending the message. In response to detecting input 686 on send icon 688 (e.g., a tap gesture on display 601), device 600 sends a representation of media item 685 to participants in the message conversation, as shown in Figures 6AQ-6AR.

[0222] In some embodiments, transmitting a media item 685 includes transmitting the media item with depth data encoded therein. Storing the media item with the depth data allows for depth-based effects (e.g., effects based on the position of an object (e.g., a user's face) in the z-direction) to be applied later (e.g., by a recipient). In some embodiments, when a media item is transmitted, its effects are encoded directly into the visual (e.g., RGB) information to improve compatibility with other devices.

[0223] In some embodiments, after sending a representation of a media item 685, device 600 displays a messaging user interface 603 having a text suggestion area 614 displayed above a keyboard display area 612, as shown in FIG. 6AR. In response to receiving an input 694 on an application dock affordance 610 (e.g., a tap gesture on display 601), device 600 replaces text suggestion area 614 with an application dock 690, as shown in FIG. 6AT.

[0224] In some embodiments, the application dock 690 remains displayed in the messaging user interface 603 until the user selects the message composition field 608 or the keyboard area 612 (an action associated with composing the text of a message). For example, as shown in FIG. 6AU, the device 600 detects an input 696 on the keyboard display area 612 (e.g., a tap gesture on the device 601). In response, as shown in FIG. 6AV, the device 600 removes the application dock 690 and displays the text suggestion area 614 above the keyboard display area 612. In some embodiments, if the input 696 is an input on the message composition field (instead of the keyboard display area 612), the device 600 also removes the application dock 690 and displays the text suggestion area 614, as shown in FIG. 6AV.

[0225] FIG. 6AW shows messaging user interface 603 after application dock affordance 610 is selected (e.g., in response to input 694 of FIG. 6AS). Application dock 690 is displayed above keyboard selection region 612 (e.g., replacing text suggestion region 614). In response to detecting input 698 on application affordance 692-1 (e.g., a tap gesture on display 601), device 600 displays application display region 699 in the position previously occupied by text suggestion region 614 and keyboard display region 612. Application display region 699 optionally includes application dock 690, which includes a representation 6100 of the selected application affordance (e.g., application affordance 692-1), as shown in FIG. 6AX.

[0226] In some embodiments, application display area 699 includes graphical objects 6102 that can be selected for use in messaging user interface 603. In some embodiments, the type of graphical objects displayed in application display area 699 depends on the application affordance selected to invoke the display of the application display area. In the embodiment shown in FIG. 6AX, selected application affordance 692-1 corresponds to an avatar application. Thus, graphical objects 6102-1 and 6102-2 displayed in application display area 699 correspond to avatars that can be selected for use in the messaging user interface. If the application affordance selected from application dock 610 corresponds to a sticker application, then graphical objects 6102 correspond to stickers that can be selected for use in the messaging user interface.

[0227] As noted above, the previous embodiments described with respect to Figures 6C-6AX are discussed with respect to device 600 operating in a still image capture mode of operation associated with photo affordance 619-2. However, unless otherwise noted, these embodiments also apply to the operation of device 600 when the device is operating in a video recording mode associated with video affordance 619-1. As such, the following description of Figures 6AY-6BQ, which show many of the previous embodiments applied to device 600 when operating in a video recording mode associated with video affordance 619-1, has been omitted as necessary to avoid redundancy.

[0228] 6AY-6AZ illustrate device 600 switching to video recording mode in response to detecting input 6104 on video affordance 619-1 of camera application user interface 615. Upon switching to video recording mode, device 600 removes camera-specific affordances 617-2, 617-3, and 617-4 and displays timer 6122.

[0229] 6AZ-6BB show device 600 activating effects mode in response to input 6106 on effects affordance 622, and then displaying visual effects option affordance 624 in camera options area 625 while video recording mode is enabled.

[0230] 6BC-6BE show device 600 displaying avatar options menu 628 in response to detecting input 6108 on avatar effects affordance 624-1. Avatar options menu 628 includes avatar options 630 that can be selected to display an avatar corresponding to the subject's face during video mode. When robot avatar option 630-3 is selected, device 600 displays robot avatar 633 on the subject's face while maintaining a view of the subject's body and background 636. Close icon 650 is selected with input 6110 to close avatar options menu 628 and return to camera options area 625 with effects affordance 622 highlighted and visual effects options affordance 624 displayed.

[0231] 6BF-6BH illustrate device 600 generating a recorded video of image data represented in image display area 620. Device 600 detects input 6112 on capture affordance 621 and initiates capture of the image data represented in image display area 620. In some embodiments, device 600 generates a video recording of the image data based on a determination that a video recording mode of device 600 is enabled. In some embodiments, device 600 generates a video recording or optionally a still image capture based on input 6112 detected on capture affordance 621. For example, if input 6112 is detected as a tap gesture, device 600 captures a still image of the image data represented in image display area 620. If input 6112 is detected as a tap-and-hold gesture, device 600 captures a video recording of the image data, as described in more detail above.

[0232] In FIG. 6BG , in response to detecting input 6112 on capture affordance 621, device 600 initiates processing to record image data represented in image display area 620, including the subject, background, and visual effects. Device 600 also modifies camera application user interface 615 by removing camera effects area 625, expanding image display area 620, and displaying stop affordance 6114 and, optionally, image capture affordance 6116 for capturing a still image of the live camera recording. In FIGS. 6BG and 6BH , image display area 620 represents a live camera recording 620-3 of objects within the field of view of camera 602, with the addition of visual effects (e.g., robot avatar 633) enabled according to various selections made as described above. These visual effects, which may include stickers and full-screen effects, are displayed in image display area 620 as part of the live image data and recorded to form part of the resulting media item. As discussed herein, these visual effects can be optionally revealed or hidden in the media item after the media item has been recorded.

[0233] In Figure 6BH, device 600 detects changes in the user's face and modifies robot avatar 633 based on the detected changes in the user's face while recording the live image stream. In response to detecting input 6118 on stop affordance 6114, device 600 stops recording image data represented in image display area 620 and displays the playback interface shown in Figure 6BI.

[0234] In FIG. 6BI, device 600 displays recorded video media item 620-4 in image display area 620 and displays re-capture affordance 679 and recipient identifier 606, which indicates the currently selected recipient of video item 620-4 to which the user should subsequently send the media item, as discussed herein.

[0235] The device 600 also removes the stop affordance 6114 and image capture affordance 6116 and displays a camera options area 625 with a video scrubber 6120, effects affordance 622, edit affordance 678, markup affordance 677, and send affordance 680 for the recorded video media item 620-4. The camera options area 625 also includes a visual effects option affordance 624. The visual effects option affordance 624 can be selected to display a respective options menu that can be used to modify the captured media item 620-4. FIGS. 6BJ-6BN show examples of modifying visual effects in the captured video media item 620-4. As shown in FIG. 6BI, the send affordance 680 is displayed in the same location / area where the capture affordance 6116 was displayed, such that the send affordance 680 replaces the capture affordance 6116 on the display.

[0236] 6BJ-6BN show device 600 displaying avatar options menu 628 in response to detecting input 6124 on avatar effect affordance 624-1. As shown in FIG. 6BK, avatar options menu 628 shows avatar options 630, including selected robot avatar option 630-3. As shown in FIG. 6BL, device 600 detects input 6126 to select rabbit avatar option 630-4 and updates video media item 620-4 to show rabbit avatar 634 positioned on the subject's head while still displaying the subject's body and background 636, as shown in FIG. 6BM. As shown in FIG. 6BN, in response to detecting input 6128 on close icon 650, device 600 closes avatar options menu 628 and displays camera options area 625 of FIG. 6BO. Camera options area 625, including video scrubber 6120, updates from displaying robot avatar 633 of FIG. 6BJ to displaying rabbit avatar 634. In some embodiments, in response to input directed at the video scrubber 6120 (e.g., a swipe input or a drag input involving a contact movement that begins at the location of the video scrubber 6120), the device scrubs through the recorded video (e.g., changing the appearance of the video preview and moving the scrubber or moving the playhead within the scrubber according to the input). In some embodiments, scrubbing through the recorded video shows how effects applied to the video affect it over time (e.g., how stickers and / or avatars move and / or change expression as the user's head / face moves, or change expression within the field of view of one or more cameras). In some embodiments, the video scrubber 6120 includes a selectable affordance for cropping the video (e.g., changing the start or end time of the video scrubber by dragging the edit affordances on either end of the video scrubber).For example, in response to user input directed at video scrubber 8120 of Figure 8AW, the device may scrub through the video using avatars and stickers, but at a rate determined based on changes in the input (e.g., the magnitude and direction of movement of a contact on a touch-sensitive display), as shown in Figures 8AW-8AZ. Interactions with video scrubber 6120 described above (and video scrubber 8120 described below) are optionally applied in a manner similar to interactions with other scrubbers described herein.

[0237] As described above with respect to Figures 6AK-6AO, video media item 620-4 can be immediately sent to participants in a message conversation in response to detecting an input (e.g., input 6130) on send affordance 680, as shown in Figures 6BP and 6BQ. A representation of video media item 620-4 can be displayed in message creation field 608 in response to input of completion affordance 618.

[0238] 7A-7B are flow diagrams illustrating a method for displaying visual effects in a messaging application using an electronic device, according to some embodiments. Method 700 is performed on a device (e.g., 100, 300, 500, 600) that includes a camera, a display device, and one or more input devices. Some operations in method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0239] As described below, method 700 provides an intuitive way to display visual effects in messaging applications. The method reduces the cognitive burden on users to apply visual effects to images sent in messaging applications, thereby creating a more efficient human-machine interface. In battery-operated computing devices, enabling users to more quickly and efficiently display visual effects on images conserves power and increases the time between battery charges.

[0240] An electronic device (e.g., 600) displays (702) via a display device (e.g., 601) a messaging user interface (e.g., 603) including a camera affordance (e.g., 609, a selectable icon associated with functionality for activating a camera application) for a message conversation involving at least a first participant.

[0241] An electronic device (e.g., 600) detects (704) a first input (e.g., 616, a touch gesture on a touchscreen display at a location corresponding to the camera affordance) directed at the camera affordance via one or more input devices.

[0242] In response to detecting a first input (e.g., 616), the electronic device (e.g., 600) displays (706) a camera user interface (e.g., 615). The camera user interface includes (708) a capture affordance (e.g., a selectable icon, 621, associated with functionality for capturing image data using the electronic device's camera).

[0243] In some embodiments, the camera user interface (e.g., 615) includes an effects mode affordance (e.g., 622, a selectable icon associated with a function for activating a mode in which various visual effects are available to modify the image data) associated with a mode in which visual effects are enabled for display on the captured image data (710). Including the effects mode affordance in the camera user interface enables a user to recognize that a particular effect (e.g., a visual effect) can be applied to an image via the camera user interface. Providing additional control of the device improves device usability, making the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and additionally reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, visual effects can be added to a representation of the image data within the camera's field of view. In some embodiments, visual effects can be added to the captured image data. In some embodiments, the visual effects are based on depth data. In some embodiments, the electronic device detects selection of the effects mode affordance (e.g., 623) via one or more input devices. In some embodiments, in response to detecting a selection of the effects mode affordance, the electronic device transitions the electronic device from a first camera mode (e.g., a standard camera mode) to a second camera mode different from the first camera mode (e.g., an effects camera mode, a mode that utilizes various visual effects to modify image data). In some embodiments, while the device is in the second camera mode, a visual indication that the second camera mode is active is displayed (e.g., the effects mode affordance is highlighted).

[0244] In some embodiments, further in response to detecting a selection (e.g., 623) of an effects mode affordance (e.g., 622), the electronic device (e.g., 600) stops displaying one or more camera mode affordances. In some embodiments, further in response to detecting a selection of an effects mode affordance, the electronic device displays a plurality of effects option affordances (e.g., 624, selectable icons each associated with a function for generating a visual effect). In some embodiments, the effects option affordances include a sticker affordance (e.g., 624-2) and / or an avatar affordance (e.g., 624-1) in a location within the camera user interface (e.g., 615) previously occupied by one or more camera mode affordances. In some embodiments, the location at which the effects option affordances are displayed is any location within a particular region (e.g., camera effects region 625) where the camera mode affordance was previously displayed. In some embodiments, the location at which each effects option affordance is displayed is the same location at which the respective camera mode affordance was displayed in that region. In some embodiments, displaying the effects option affordance includes replacing the camera mode affordance with the effects option affordance.

[0245] In some embodiments, an electronic device (e.g., 600) detects a selection (e.g., 654) of a first one of the effect option affordances (e.g., sticker affordance 624-2) via one or more input devices. In some embodiments, in response to detecting the selection of the first one of the effect option affordances, the electronic device stops displaying the multiple effect option affordances (e.g., 624) and displays the multiple selectable graphic icons (e.g., 658, stickers). By stopping the display of the multiple effect option affordances and displaying the multiple selectable graphic icons in response to detecting the selection of the first one of the effect option affordances, a user can quickly and easily recognize that the first one of the effect option affordances is associated with a graphic icon (e.g., sticker) option, thereby improving device usability and making the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs and reducing user errors when operating / interacting with the device), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, displaying the sticker includes displaying an area (e.g., sticker menu 656) above the affordance of the effects option, which area includes multiple sticker options that can be selected for display on the image represented in the camera user interface. In some embodiments, the user selects a sticker (e.g., 658-1) by tapping on the sticker (e.g., 660), and the sticker automatically appears on the image (e.g., in a default position such as the center of the image). In some embodiments, the user selects a sticker by touching and dragging the sticker from the sticker menu onto the image.In some embodiments, further in response to detecting a selection of a first of the effect option affordances while displaying the capture affordance, the electronic device stops displaying the capture affordance.

[0246] In some embodiments, an electronic device (e.g., 600) detects selection (e.g., 626) of a second one of the effect option affordances (e.g., avatar affordance 624-1) via one or more input devices. In some embodiments, in response to detecting selection of the second one of the effect option affordances, the electronic device stops displaying the multiple effect option affordances (e.g., 624) and displays an avatar selection area (e.g., avatar menu 628) having multiple avatar affordances (e.g., 630 displayed in a linear arrangement) (e.g., affordances representing avatars). By stopping displaying the multiple effect option affordances and displaying the avatar selection area in response to detecting selection of the second one of the effect option affordances, a user can quickly and easily recognize that the second one of the effect option affordances is associated with an avatar selection, thereby improving usability of the device and making the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs and reducing user errors when operating / interacting with the device), as well as reducing device power usage and improving battery life by allowing users to use the device more quickly and efficiently. In some embodiments, the avatar affordance corresponds to an avatar that is customizable, non-customizable, or a combination thereof. In some embodiments, displaying the avatar selection area includes displaying an area (e.g., avatar menu 628) above the effects options affordance, the area including multiple avatar affordances that can be selected to display corresponding avatars on an image presented in the camera user interface.

[0247] In some embodiments, an electronic device (e.g., 600) detects a swipe input (e.g., 646, a vertical swipe gesture) on an avatar selection area (e.g., 628) via one or more input devices. In response to detecting the swipe input on the avatar selection area, the electronic device increases the size of the avatar selection area (e.g., 628-1) and displays a plurality of avatar affordances (e.g., 630) arranged in a matrix. By increasing the size of the avatar selection area and displaying a plurality of avatar affordances in a matrix in response to detecting the swipe input on the avatar selection area, the user can (simultaneously) view one or more additional selectable avatars that are not (simultaneously) visible in the avatar selection area. Providing additional control options without cluttering the user interface with additional displayed controls improves device usability and makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs and reducing user errors when operating / interacting with the device), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, increasing the size of the avatar display area includes vertically expanding the avatar display area using avatar affordances displayed in a matrix in the avatar display area to present a full-screen display of the avatar display area.

[0248] In some embodiments, the camera user interface (e.g., 615) further includes a first representation of the image data (e.g., a live camera preview 620-1). Providing the first representation of the image data (e.g., a live camera preview) provides visual feedback of one or more changes (to the image) made by the user before saving / confirming the changes. Providing the user with improved visual feedback improves device usability and makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, further in response to detecting a selection of an effects mode affordance (e.g., 624-1, 624-2), the electronic device (e.g., 600) ceases displaying the first representation of the image data in accordance with a determination that the first representation of the image data corresponds to image data obtained from a second camera (e.g., a rear-facing camera), and displays a second representation of the image data (e.g., a live camera preview), where the second representation of the image data corresponds to image data obtained from the camera (e.g., a front-facing camera). In some embodiments, the representation of the image data corresponding to the front-facing camera includes a representation of a user positioned within a field of view of the front-facing camera.

[0249] In some embodiments, while the electronic device (e.g., 600) is in a second camera mode (e.g., an effects camera mode, a mode that makes available various visual effects for modifying image data), the electronic device receives a request to transition to a first camera mode (e.g., a normal mode, a no-effects mode) (e.g., selection of an active visual effect affordance). In some embodiments, the electronic device transitions the electronic device from the second camera mode to the first camera mode in response to receiving the request to transition to the first camera mode. In some embodiments, the electronic device deactivates the first visual effect (e.g., disables, stops displaying a displayed first visual effect) in accordance with the first visual effect being active (e.g., actively applied to a preview of captured image data or image data for capture).

[0250] In some embodiments, after deactivating the first visual effect, the electronic device (e.g., 600) detects a subsequent selection of an effect mode affordance (e.g., 624-1, 624-2) via one or more input devices. In some embodiments, in response to detecting the subsequent selection of the effect mode affordance, the electronic device reactivates the first visual effect pursuant to a determination that the subsequent selection of the effect mode affordance occurred within a predetermined time after deactivating the first visual effect. By reactivating the first visual effect pursuant to a determination that the subsequent selection of the effect mode affordance occurred within a predetermined time after deactivating the first visual effect, the user can quickly and easily return to the previous visual effect (e.g., without having to reselect / regenerate the effect). Providing additional control options without cluttering the user interface with additional displayed controls improves device usability and makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, after selecting an effect mode affordance that removes a visual effect from an image, if the effect mode affordance is selected again within a predetermined time, the removed visual effect is restored to the image.

[0251] The electronic device (e.g., 600) detects (712) a second input (e.g., 676, a touch gesture on a touchscreen display at a location corresponding to the capture affordance) directed at the capture affordance (e.g., 621) via one or more input devices.

[0252] In response to detecting a second input (714), the electronic device (e.g., 600) captures image data (716) using the camera (e.g., 602). In some embodiments, capturing the image data includes capturing the image data in a first image capture mode (e.g., photo capture mode, still image capture mode) according to a value of a characteristic (e.g., duration of contact) of the second input satisfying a first capture mode criterion (e.g., less than a threshold duration). In some embodiments, capturing the image data includes capturing the image data in a second image capture mode (e.g., video capture mode, continuous capture mode) according to a value of a characteristic of the second input satisfying a second capture mode criterion (e.g., greater than a second threshold duration). In some embodiments, the capture affordance (e.g., 621) is a multi-function capture affordance. In some embodiments, the electronic device captures a photo (e.g., a still image) when a tap is detected on the capture affordance. In some embodiments, the electronic device captures video (e.g., continuous images) when a press-and-hold gesture is detected on the capture affordance.

[0253] In response to detecting (714) a second input (e.g., 676), the electronic device (e.g., 600) stops displaying (718) the capture affordance (e.g., 621).

[0254] In some embodiments, the electronic device (e.g., 600) displays a markup affordance (e.g., 677), an edit affordance (e.g., 678), and a re-capture affordance (e.g., 679) after capturing image data using the camera (e.g., 602). In some embodiments, the electronic device receives a fourth user input via one or more input devices while displaying the markup affordance, the edit affordance, and the re-capture affordance. In some embodiments, the electronic device, in response to detecting the fourth user input, initiates a process for editing the captured image data according to the fourth user input corresponding to the edit affordance. In some embodiments, the process for editing includes displaying one or more affordances for editing the captured image data. In some embodiments, the electronic device, in response to detecting the fourth user input, initiates a process for marking up the captured image data according to the fourth user input corresponding to the markup affordance. In some embodiments, the process for editing includes displaying one or more affordances for marking up the captured image data. In some embodiments, in response to detecting the fourth user input, the electronic device initiates a process to re-capture the captured image data according to the fourth user input corresponding to the re-capture affordance, hi some embodiments, initiating the process to re-capture the captured image data includes capturing new image data and replacing the captured image data with the new image data.

[0255] In response to detecting (714) a second input (e.g., 676), the electronic device (e.g., 600) displays (720) a send affordance (e.g., 680, a selectable icon associated with functionality for sending captured image data to a participant in a conversation or for presenting the captured image data in a composition area prior to subsequent transmission) in a location within the camera user interface (e.g., 615) previously occupied by the capture affordance (e.g., 621). Displaying the send affordance in a location within the camera user interface previously occupied by the capture affordance provides visual feedback that the captured image is ready to be sent to an intended recipient. Providing the user with visual feedback without cluttering the user interface improves device usability and makes the user-device interface more efficient (e.g., by assisting the user in providing appropriate inputs and reducing user errors when operating / interacting with the device), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, the location where the send affordance (e.g., 680) is displayed is anywhere within a particular region where the capture affordance was previously displayed (e.g., camera effect region 625). In some embodiments, the location where the send affordance is displayed is the same location where the capture affordance was displayed within that region. In some embodiments, displaying the send affordance includes replacing the capture affordance with the send affordance.

[0256] In some embodiments, while displaying the send affordance (e.g., 680), the electronic device (e.g., 600) displays (722) a representation of the first participant (e.g., an icon, photo, avatar, or other identifier associated with the first participant). Displaying the representation of the first participant while displaying the send affordance allows the user to quickly and easily recognize the intended recipient, thereby improving device usability and making the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, the representation of the first participant serves as an indication to the user that the captured photo will be sent to the first participant. In some embodiments, the representation of the first participant is not displayed (724) prior to capturing image data using the camera (e.g., 602). In some embodiments, the camera user interface (e.g., 615) includes camera-specific affordances (e.g., 619 corresponding to filters, lighting options, timer options, etc.) that are displayed prior to capturing image data. In some embodiments, displaying a representation of the first participant replaces the displayed camera-specific affordances with the representation of the first participant.

[0257] The electronic device (e.g., 600) detects (726) a third input directed toward the transmit affordance (e.g., a touch gesture on the touchscreen display at a location corresponding to the transmit affordance 680) via one or more input devices.

[0258] In response to detecting the third input, the electronic device (e.g., 600) initiates (728) a process (e.g., immediately transmitting or presenting the image data captured within the creation area before subsequent transmission) and transmits the captured image data to the first participant.

[0259] In some embodiments, the electronic device (e.g., 600) displays a completion affordance (e.g., a selectable icon associated with a function to close the camera user interface and display a messaging user interface) before detecting a third input directed toward the send affordance (e.g., 680). In some embodiments, the electronic device detects selection of the completion affordance via one or more input devices. In some embodiments, the electronic device displays a messaging user interface (e.g., 603) having a message composition area (e.g., 608) in response to detecting selection of the completion affordance. In some embodiments, the electronic device displays a representation of the captured image data in the message composition area in response to detecting selection of the completion affordance. In some embodiments, selection of the completion affordance closes the camera user interface and displays the captured image data in a message composition field of the messaging user interface without sending the captured image data.

[0260] In some embodiments, the electronic device (e.g., 600) is in a first image capture mode (e.g., photo capture mode, video capture mode) before detecting a third user input. In some embodiments, the electronic device maintains the first image capture mode further in response to detecting a third user input. In some embodiments, the electronic device can be configured (e.g., user-configured) to capture image data according to multiple modes (e.g., photo capture mode, video capture mode). In some embodiments, the selection of the image capture mode is persistent even when the electronic device transitions from a first camera mode (e.g., standard camera mode, no-effects camera mode) to a second camera mode (e.g., effects camera mode).

[0261] In some embodiments, initiating the process of sending the captured image data to the first participant includes sending the captured image data to the first participant (e.g., without displaying the messaging user interface 615). In some embodiments, selecting a send affordance (e.g., 680) from the camera user interface immediately sends the captured image data to other participants in the message conversation without displaying an intermediate user interface or requiring further input from the user.

[0262] In some embodiments, initiating the process of sending the captured image data to the first participant includes redisplaying a messaging user interface (e.g., 615), where the messaging user interface further includes a keyboard area (e.g., 612) and an application menu affordance (e.g., 610, a selectable icon associated with a function for displaying the application menu user interface). Redisplaying the messaging user interface as part of initiating the process of sending the captured image data to the first participant provides visual feedback that the captured image data is being sent via the message conversation. Providing the user with improved visual feedback improves device usability, makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and additionally reduces device power usage and improves battery life by allowing the user to use the device more quickly and efficiently. In some embodiments, the application menu affordance is displayed adjacent to a message composition field in the messaging user interface. In some embodiments, the electronic device (e.g., 600) detects selection of the application menu affordance via one or more input devices. In some embodiments, in response to detecting a selection of the application menu affordance, the electronic device displays an application menu area adjacent to (e.g., above) the keyboard area, the application menu area having a plurality of application affordances (e.g., selectable icons each associated with a function for starting an application associated with the respective application affordance). In some embodiments, the application affordances include a sticker affordance and an avatar affordance.

[0263] In some embodiments, the electronic device (e.g., 600) detects a fourth input on a messaging user interface (e.g., 615). In some embodiments, pursuant to determining that the fourth input corresponds to the location of a keyboard area (e.g., 612) or the location of a message composition area (e.g., 608) within the messaging user interface, the electronic device ceases displaying the application menu area and displays a text suggestion area (e.g., an area having a list of suggested words for the user to conveniently select) in the location within the messaging user interface previously occupied by the application menu area.

[0264] In some embodiments, the electronic device (e.g., 600) detects a selection of one of a plurality of application affordances (e.g., a sticker affordance or an avatar affordance) in an application menu region. In some embodiments, in response to detecting a selection of the application affordance (e.g., 610), the electronic device ceases displaying a keyboard region (e.g., 612) and displays an application display region in a location in the messaging user interface (e.g., 615) previously occupied by the keyboard region, the application display region including a plurality of graphical objects (e.g., avatars or stickers) corresponding to the selected application affordance. In some embodiments, the selected application affordance is a sticker affordance, and the graphical object displayed in the application display region is a sticker. In some embodiments, the selected application affordance is an avatar affordance, and the graphical object displayed in the application display region is an avatar (e.g., a customizable avatar and / or a non-customizable avatar).

[0265] It should be noted that the details of the processes described above with respect to method 700 (e.g., FIGS. 7A-7B) are also applicable in an analogous manner to the methods described below. For example, methods 900, 1100, 1300, and 1500 optionally include one or more of the features of the various methods described above with respect to method 700. For example, visual effects such as stickers and virtual avatars are displayed on image data in a camera application user interface, a media user interface, and a user interface for a live video communication session. For the sake of brevity, these details will not be repeated below.

[0266] 8A-8BQ illustrate example user interfaces for displaying visual effects in a camera application, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in FIGS. 9A and 9B.

[0267] In FIG. 8A, device 600 shows home screen 800 and detects input 801 on camera application affordance 802.

[0268] 8B, in response to detecting input 801, device 600 launches a camera application associated with camera application affordance 802 and displays camera application user interface 815. Camera application user interface 815 is similar to camera application user interface 615, described above with respect to the embodiment shown in FIGS. 6A-6BQ.

[0269] The camera application user interface 815 includes an image display area 820 that displays a representation of image data, such as, for example, streamed image data (e.g., a live camera preview, a live camera recording, or a live video communication session) representing objects located within the field of view of a camera (e.g., the rear-facing camera or camera 602), or media items, such as photos or video recordings. In the embodiment shown in FIG. 8B , the image display area 820 shows a live camera preview 820-1′ from the rear-facing camera of device 600.

[0270] The camera application user interface 815 also includes an area above the image display area 820 that contains camera-specific affordances 817. The camera-specific affordances include an affordance 817-1 associated with a camera flash function, an affordance 817-2 associated with a camera mode function, an affordance 817-3 associated with a timer function, and an affordance 817-4 associated with a filter function.

[0271] The camera application user interface 815 also includes a camera options region 825 (similar to the camera options region 625) disposed below the image display region 820. The camera options region 825 includes a camera selector affordance 827 for switching between cameras (e.g., the rear camera and the camera 602) and camera options affordances 819 associated with different capture modes in which the camera can record image data. For example, the video affordance 819-1 is associated with functionality for activating the camera's video recording capture mode, and the photo affordance 819-2 is associated with functionality for activating the camera's still image capture mode. In the embodiments discussed below with respect to FIGS. 8B-8AQ, the device 600 is in the still image capture mode of operation associated with the photo affordance 819-2. However, unless otherwise noted, these embodiments also apply to the video recording mode associated with the video affordance 819-1.

[0272] Camera options region 825 further includes effects affordance 822 for enabling or disabling a mode (visual effects mode, effects mode) of device 600 in which device 600 is enabled or disabled to display visual effects in image display region 820. Effects affordance 822 is similar to effects affordance 622 and therefore has the same functionality as effects affordance 622 unless otherwise specified. Thus, effects affordance 822 is selected to enable the display of visual effects and deselected to disable the display of visual effects.

[0273] Camera options area 825 also includes a capture affordance 821 that functions similarly to capture affordance 621 described above. Capture affordance 821 can be selected to capture image data represented in image display area 820. In some embodiments, device 600 captures the image data in a manner based on the currently enabled capture option (e.g., video recording capture mode or image capture mode). In some embodiments, device 600 captures the image data according to the type of gesture detected on capture affordance 821. For example, if device 600 detects a tap gesture on capture affordance 821, device 600 captures a still image of the image data represented in image display area 820 at the time the tap gesture occurred. If device 600 detects a tap-and-hold gesture on capture affordance 821, device 600 captures a video recording of the image data represented in image display area 820 for the duration of the tap-and-hold gesture. In some embodiments, video recording stops when a finger is lifted from the affordance. In some embodiments, the video recording continues until a subsequent input (e.g., a tap input) is detected at the location corresponding to the affordance. In some embodiments, the captured image (e.g., a still image or a video recording) can be shared with other devices, for example, using a messaging application.

[0274] In FIG. 8C, device 600 detects input 823 on effect affordance 822.

[0275] 8D , in response to detecting input 823, device 600 activates camera 602 (e.g., switches from the rear-facing camera) and updates image display area 820 to display a live camera preview 820-1 from camera 602, showing a representation of object 832 located within the field of view of camera 602 and a background 836 displayed behind object 832. As discussed herein, in some embodiments, image data captured using camera 602 includes depth data that can be used to determine the depth of objects within the field of view of camera 602. In some embodiments, device 600 analyzes objects (e.g., in the image data) based on the detected depths of these objects and uses this determination to apply visual effects as discussed herein. For example, device 600 can classify object 832 as being in the foreground of live camera preview 820-1 and objects located behind the user as being in the background of live camera preview 820-1. These background objects are collectively referred to herein as background 836.

[0276] Device 600 also highlights effects affordance 822, indicating that visual effects are enabled for display, and updates camera options area 825 by replacing camera option affordance 819 with visual effects option affordance 824. The visual effects option affordances include avatar effects affordance 824-1 and sticker effects affordance 824-2. Visual effects option affordance 824 is similar to visual effects option affordance 624 described above. Visual effects option affordance 824 corresponds to different visual effects that can be applied to the image displayed in image display area 820. Selecting one of the visual effects option affordances (e.g., 824-1 or 824-2) causes a menu to be displayed with visual effect options that correspond to the selected visual effects option affordance.

[0277] 8E-8F, device 600 detects input 826 on avatar effect affordance 824-1 and displays avatar options menu 828. Avatar options menu 828 is similar to avatar options menu 628. Avatar options menu 828 includes a scrollable list of avatar options 830 and a selection area 829 for indicating a selected one of the avatar options 830. The displayed avatar options include a customizable female avatar 830-1, a null avatar option 830-2, a robot avatar option 830-3, a rabbit avatar option 830-4, and a second customizable female avatar 830-5. As shown in FIG. 8F, null avatar option 830-2 is selected as the default avatar option 830. This causes the representation of subject 832 displayed in image display area 820 to be shown without an avatar appearing on the subject's face. Avatar options 830 may be pre-configured, such as avatar options 830-3 or 830-4, or may be customizable avatars, such as avatar options 830-1 or 830-5.

[0278] Avatar options 830 have similar functionality to avatar options 630. As such, avatar options 830 correspond to virtual avatar visual effects that are applied to a representation of a subject in image display area 820. Specifically, each avatar option 830, when selected, corresponds to a virtual avatar that is transposed onto the subject's face in the image display area, while other portions of the image in the image display area (e.g., the background or other parts of the user, such as the user's body) remain displayed. A user (e.g., subject 832) positioned within the field of view of camera 602 can control the visual aspects of the virtual avatar by changing the pose (e.g., rotation or orientation) of their face, as described above.

[0279] Avatar options 830 can be scrolled through via gestures on the avatar options menu 828. For example, selecting robot avatar option 830-3, as shown in FIG. 8H, then displays as robot avatar 833 on the subject's face in image display area 820, horizontal gesture 844a shown in FIG. 8G. By continuing to scroll via horizontal gesture 844b, additional avatar options 830 are scrolled onto the displayed avatar options menu 828. In some embodiments, as the user scrolls through avatar options 830, image display area 820 shows an avatar (e.g., robot avatar 833 in FIG. 8J) that includes a blur effect 803 applied to image data depicted behind the avatar.

[0280] 8K, device 600 displays customizable female avatar option 830-5 positioned within selection region 829 just before it is selected (e.g., gesture 844b has not yet ended), and edit affordance 804 displayed below customizable female avatar option 830-5. Device 600 continues to display robot avatar 833 with blur effect 803. In some embodiments, once the customizable avatar option is positioned in selection region 829, device 600 displays edit affordance 804 in selection region 829. Edit affordance 804 can be selected to display an avatar editing user interface, such as that discussed below with respect to FIG. 8AA.

[0281] FIG. 8L is after avatar option 830-5 is selected (e.g., gesture 844b is ended). Device 600 removes robot avatar 833 from image display area 820 (while continuing blur effect 803) and displays an animation of customizable female avatar 835 emerging from avatar option 830-5 in selection area 829 and moving into image display area 820 (e.g., to a center position in image display area 820 or a position corresponding to the subject's face), where avatar 835 is modified based on detected changes to the subject's face. In some embodiments, the animation includes the avatar (e.g., 835) beginning with a static appearance (e.g., similar to a sticker matching the displayed avatar option) and then transitioning to a dynamic state (shown in FIG. 8L) where the avatar (e.g., 835) is modified based on device 600 detecting changes to the user's face. In some embodiments, the transition of the avatar from static to dynamic occurs before the avatar moves out of the selection area. In some embodiments, the transition from static to dynamic for an avatar occurs when the avatar moves from the selection area 829 to the image display area 820 .

[0282] As shown in Figures 8M and 8N, when avatar 835 is applied to the subject's face, device 600 modifies avatar 835 based on detected changes in the subject's face, including changes to the position of the face within the field of view of camera 602.

[0283] In some embodiments, when device 600 no longer detects the subject's face within the field of view of camera 602, device 600 re-applies blur effect 803 (similar to blur effect 644) to the background and displays prompt 838 instructing the user to return their face to the field of view of camera 602. In the embodiment shown in FIG. 8P , device 600 displays prompt 838 including a representation 840 of a selected avatar (e.g., a customizable female avatar) and a message 842 instructing the user to reposition their head within the field of view of camera 602. In some embodiments, displaying prompt 838 includes displaying an animation of the selected avatar (e.g., 835) returning to a center position in image display area 820 and showing slowed movement of the avatar and its features that appear to stop based on a physical model of the avatar's features.

[0284] In some embodiments, when device 600 detects the user's face returning to the field of view of camera 602, device 600 displays avatar 835 that moves from the center position of image display area 820 to the position of the user's face and resumes changing the avatar based on the detected changes in the user's face.

[0285] In some embodiments, the avatar options menu 828 can be expanded with a vertical gesture (e.g., 805) to display an expanded version of the avatar options menu shown in FIGS. 8Q-8T. The expanded version of the avatar options menu displays a larger number of avatar options 830, including different types of avatars (e.g., customizable and non-customizable). In the expanded avatar display menu, a new avatar option (e.g., robot avatar option 830-3) can be selected. The device 600 then displays the corresponding avatar (e.g., robot avatar 833) on the user's face when the expanded version of the avatar options menu returns to its original (e.g., condensed) state, as shown in FIG. 8U.

[0286] In some embodiments, device 600 displays different avatars over the user's head in response to detecting a swipe gesture on image display area 820. For example, in FIG. 8V , device 600 detects a swipe gesture 806 moving in a rightward direction across image display area 820. In response to detecting swipe gesture 806, device 600 scrolls through avatar options 830 in avatar options menu 828. In some embodiments, the magnitude of the scrolling is determined by the magnitude of the swipe gesture. This allows a user to scroll through multiple avatars based on the magnitude of the swipe gesture. Additionally, the direction of the scrolling and the corresponding movement of the avatar on image display area 820 is determined based on the direction of swipe gesture 806. Thus, the scrolling of avatar options and the corresponding movement of the transitioning avatars can be leftward if the swipe is leftward, or rightward if the swipe is rightward (as shown in FIGS. 8V and 8W ).

[0287] As scrolling through avatar options 830 begins, the currently selected avatar option (e.g., robot avatar option 830-3 in FIG. 8V) leaves selection region 829, and as scrolling continues, a new avatar option (e.g., avatar option 830-1 in FIG. 8W) enters selection region 829. As scrolling through avatar options begins, device 600 also displays an animation in image display region 820 of the currently selected avatar (e.g., robot avatar 833) moving away from the user's face and out the right side of image display region 820, while the next avatar (e.g., customizable avatar 831) enters image display region 820 from the left and moves toward the user's face. In some embodiments, device 600 applies a blur effect 803 to image display region 820 while the avatar is being scrolled across image display region 820.

[0288] 8V and 8W, null avatar option 830-2 is not selected to better illustrate the transition from the currently selected avatar to the new avatar that appears in image display area 820. However, it should be understood that null avatar option 830-2 can also be selected with a swipe gesture on image display area 820. In such a case, the currently selected avatar is moved off-screen while the background is blurred, and then the blur effect is removed to reveal the user's face without the avatar.

[0289] 8X and 8Y illustrate selection of a different customizable avatar option (e.g., customizable avatar option 830-5) and a corresponding displayed customizable avatar 835. When customizable avatar option 830-5 is displayed in selection area 829, edit affordance 804 is displayed. Edit affordance 804 can be selected (e.g., via input 807) to display avatar editing user interface 808 shown in FIG. 8AA. Avatar editing user interface 808 provides an interface for editing the avatar (e.g., customizable avatar 835) corresponding to the customizable avatar option (e.g., 830-5) associated with the selected edit affordance 804.

[0290] In some embodiments, the applied visual effects may include a shadow 851 shown on the subject's neck below the applied custom avatar 835, or lighting effects such as light reflections on eyeglasses. As device 600 modifies avatar 835 to reflect the user's real-time actions, device 600 also modifies the lighting effects projected onto avatar 835 and onto the subject, including moving the display position of the reflected shadow, based on the relative positions of the modeled light source and avatar 835.

[0291] 8AA, avatar editing user interface 808 includes a representation of a customizable avatar 835, as well as various avatar feature options 810 (e.g., facial color option 810-1 and facial shape option 810-2) that represent the currently selected avatar feature option and available feature options that can be selected to modify avatar 835. The avatar feature options correspond to values ​​for aspects of the currently selected avatar feature, such as a particular avatar feature, specifically the avatar head feature indicated by highlighted avatar head affordance 809-1. Avatar editing user interface 808 shows selected facial color option 810-1a and selected facial shape option 810-2a, as represented on avatar 835. The displayed avatar feature options can be changed by selecting a different avatar feature. For example, in FIG. 8AB, device 600 updates avatar editing user interface 808 to display different avatar hair feature options (e.g., hair texture option 811-1 and hair color option 811-2) when avatar hair affordance 809-2 is selected.

[0292] Device 600 modifies avatar 835 when different avatar feature options are selected. For example, in Figure 8AB, device 600 detects the selection of straight hair texture option 811-2a and updates avatar 835 to have the selected straight hair texture option. In some embodiments, device 600 modifies the representation of avatar 835 shown in avatar editing user interface 808 in response to the detected changes in the user's face.

[0293] In response to detecting selection of completion affordance 812, device 600 exits avatar editing user interface 808 and returns to camera application user interface 815, showing selected avatar option 830-5 and corresponding avatar 835 based on the hair texture option selected in the avatar editing user interface.

[0294] 8AD-8AF illustrate embodiments in which a newly customized avatar can be created and added to avatar options menu 828. In response to detecting selection of new avatar affordance 813, device 600 displays avatar editing user interface 808 with a representation of a default avatar without the selected avatar feature options, as shown in FIG. 8AE. Device 600 can modify the default avatar based on the user selection of the avatar feature options and save the avatar as a new avatar selectable from avatar options menu 828. Thus, in FIG. 8AF, device 600 detects selection of the cancel affordance and refrains from saving the modified avatar to avatar options menu 828, as shown in FIG. 8AG.

[0295] In response to detecting a selection of the cancel icon 850 in FIG. 8AG, the device 600 closes the avatar options menu 828 and displays the camera options area 825 as shown in FIG. 8AH.

[0296] In response to detecting selection of sticker effect affordance 824-2, device 600 displays sticker options menu 856 having a scrollable list of stickers 858 of FIG. 8AI. Sticker options menu 856 and stickers 858 are similar to sticker options menu 656 and stickers 658 described above. Stickers are static graphical objects that a user can select and apply to an image in image display area 820. In some embodiments, stickers can be selected with a tap gesture, and the corresponding sticker is then displayed in a position on image display area 820. In some embodiments, stickers can be selected with a touch-and-drag gesture, starting on the selected sticker, dragging the sticker to the desired location, and then ending the drag (e.g., lifting the finger) to place the sticker in the desired location. An example of such an embodiment is shown in FIGS. 8AI-8AK, where helmet sticker 858-1 has been selected (e.g., input 853) and dragged to a position above (e.g., away from) custom avatar 835. In Figure 8AK, device 600 detects that subject 832 moves laterally from the position shown in Figure 8AJ. In response, device 600 displays both helmet sticker 858-1 and avatar 835, which move laterally based on the lateral movement (e.g., direction and magnitude) of the subject's representation.

[0297] In FIG. 8AK, device 600 detects input 854 on close affordance 855. In response, device 600 closes sticker options menu 856 and displays camera options area 825 of FIG. 8AL. In FIG. 8AL, device 600 shows subject 832 tilting their head and shifting back toward the center of the screen. In response to detecting this movement of subject 832, device 600 displays helmet sticker 858-1 above customizable avatar 835 of FIG. 8AL, shifting it laterally (from the position shown in FIG. 8AK) based on the subject's head's movement toward the center of the screen. Device 600 displays helmet sticker 858-1 while maintaining its spacing relative to avatar 835. However, as shown in FIG. 8AL, helmet sticker 858-1 does not rotate (e.g., tilt) in response to the subject's head rotation (e.g., tilt), while avatar 835 does rotate (e.g., tilt).

[0298] In FIG. 8AL, device 600 detects selection of capture affordance 821, which causes device 600 to capture an image of live camera preview 820-1 displayed in image display area 820, including visual effects (e.g., customized avatar 835 and helmet sticker 858-1), as well as other image data including subject 832 and background 836.

[0299] 8A-M, device 600 displays camera application user interface 815 showing a captured image (e.g., media item 820-2) in image display area 820 (e.g., media item 820-2 replaces live camera preview 820-1 shown in image display area 820). Media item 820-2 is a representation of live camera preview 820-1 at the time capture affordance 821 was selected. Thus, media item 820-2 includes helmet sticker 858-1 and avatar 835 displayed over the face of subject 832 and background 836.

[0300] Device 600 also replaces the camera-specific affordance (eg, affordance 817 shown in FIG. 8AL) with re-capture affordance 879 and saves affordance 818 for saving captured media item 820-2.

[0301] Device 600 also updates camera options region 825, replacing capture affordance 821 and camera selector affordance 827 with markup affordance 877, edit affordance 878, and share affordance 880. Markup affordance 877 allows the user to mark up media item 820-2. Edit affordance 878 allows the user to edit media item 820-2, such as by cropping the image or adjusting other properties of media item 820-2. Share affordance 880 allows the user to send media item 820-2 to another device, such as a messaging or email application.

[0302] Device 600 displays camera options region 825, which includes visual effects option affordance 824. Visual effects option affordance 824 can be selected to display a respective options menu, which can be used to modify captured media item 820-2 (as well as recorded video media item 820-4, described below). For example, FIGS. 8AN-8AP show device 600 adding a sticker to media item 820-2. Sticker effects affordance 824-2, which causes device 600 to display sticker options menu 856 in FIG. 8AO, also shows selection of rabbit sticker 858-2 placed on subject 832 within media item 820-2, shown in FIG. 8AP. Sticker options menu 856 is closed in FIG. 8AP, revealing camera options region 825 in FIG. 8AQ.

[0303] In FIG. 8AQ, device 600 also displays media item 820-2, which includes rabbit sticker 858-2 placed on subject 832, along with other visual effects (eg, avatar 835 and helmet sticker 858-1).

[0304] Stickers can be added to recorded media items in video format in a manner similar to that described above for media item 820-2 (still image). For example, Figure 8AR shows an embodiment in which video media item 820-4 is a recorded video. Media item 820-4 can be created in a manner similar to that described above for media item 620-4.

[0305] In FIG. 8AR, device 600 displays recorded video media item 820-4 in image display area 820. Device 600 also displays a camera options area 825 with a video scrubber 8120, effects affordance 822, edit affordance 878, markup affordance 877, and share affordance 880 for recorded video media item 820-4. Camera options area 825 also includes visual effects options affordance 824. Visual effects options affordance 824 can be selected to display respective option menus that can be used to modify captured media item 820-4. For example, FIGS. 8AR-8AV show device 600 adding stickers 858-2 and 858-3 to media item 820-4.

[0306] In some embodiments, displayed stickers may have differently modeled behavior in a video media item (or live video stream). For example, some stickers have the appearance of being applied to a display (e.g., 601) and remain static as objects in the image data move. An example of such a sticker is shown by heart sticker 858-3 in FIGS. 8AX-8BC. In some embodiments, if device 600 does not detect the presence of all or part of a subject's representation (e.g., head or face) in the media item, live camera preview, or field of view of camera 602, the sticker has this static behavior when placed on a media item (e.g., 820-4) or live camera preview (e.g., 820-1). For example, if a subject's face is completely off-screen (e.g., not detected in the field of view of camera 602) and a sticker is placed on image display area 820, once the face is detected in the field of view of camera 602 and displayed on image display area 820 along with the previously placed sticker, the sticker remains stationary and does not track any movement of the subject's face.

[0307] Other stickers are applied to the display to give the appearance of moving and following objects in the image (e.g., items in the camera's field of view, including an avatar or representation of the subject). In some embodiments, the stickers are positioned at a distance from the object they are following. An example of such a sticker is displayed by helmet sticker 858-1 in FIGS. 8AK and 8AX-8BC. Helmet sticker 858-1 follows the lateral movement of subject 832, but does not follow the rotational movement of the subject. Another example of such a sticker is displayed by starfish sticker 858-5 in FIGS. 8BE-8BQ. When positioned at a distance from the subject's face, starfish sticker 858-5 follows the movement of the subject's face while maintaining its position relative to the face (e.g., starfish sticker 858-5 follows the forward, backward, and side-to-side movement of the subject's face, but does not follow the rotational movement of the subject's face).

[0308] However, other stickers have the appearance of being applied to an object in the field of view of camera 602 and move to follow the object in the field of view (e.g., the sticker has the appearance of depth matching the object in the image). An example of such a sticker is shown by rabbit sticker 858-2 in FIGS. 8AX-8BC. Rabbit sticker 858-2 moves with the lateral and rotational movement of the subject (specifically, the subject's shoulder). Another example of such a sticker is shown by eyeglasses sticker 858-4 in FIGS. 8BG-8BM. When placed on the subject's face in FIGS. 8BG-8BM, eyeglasses sticker 858-4 follows the lateral movement of the subject's face (e.g., left / right, up / down), the forward / backward movement of the subject's face, and the rotational movement of the subject's face. Glasses sticker 858-4 has the appearance of being physically attached (e.g., glued) to the subject's face.

[0309] In some embodiments, the behavior of a sticker changes based on its position relative to an object within a media item (or a live camera preview or camera field of view). In some embodiments, the behavior of a sticker changes in response to detecting a change in the sticker's position relative to the object. Examples of such stickers are shown in Figures 8BE-8BQ and are described in more detail below.

[0310] 8AW-8BC illustrate video playback and editing of media item 820-4, including avatar 835, helmet sticker 858-1, rabbit sticker 858-2, and heart sticker 858-3. As media item 820-4 advances during playback, heart sticker 858-3 remains stationary, having the appearance of being attached to display 601 and not moving. Helmet sticker 858-1 has the appearance of being attached to the display, but moves (e.g., transforms) as the position of avatar 835 changes. Rabbit sticker 858-2 has the appearance of being attached to the shoulder of subject 832 within media item 820-4. As the shoulder moves, rabbit sticker 858-2 changes position, including orientation and displayed size, to give the appearance of following the subject's shoulder during playback of media item 820-4.

[0311] In some embodiments, stickers or other virtual objects applied to the position of a representation of the field of view of one or more cameras, including a respective object (e.g., a face, hand, or other body part of a device user) being tracked in three dimensions (e.g., via depth information from a depth sensor), are attached to the respective object such that as the respective object moves laterally (e.g., left and right and / or up and down) within the field of view of one or more cameras, the size and / or orientation of the virtual object changes as the distance of the respective object from and / or orientation of the respective object relative to the one or more cameras changes. For example, as shown in FIGS. 8AS-8AT, rabbit sticker 858-2 is placed on a representation of a subject's shoulder. In FIGS. 8AX-8AY, as the shoulder moves above and toward the camera, rabbit sticker 858-2 moves laterally (upward) with the shoulder and expands as the shoulder moves toward the camera. In some embodiments, stickers or other virtual objects applied to positions in the field of view of one or more cameras that are separate (e.g., remote) from a respective object (e.g., a face, hand, or other body part of a device user) being tracked in three dimensions (e.g., via depth information from a depth sensor) move laterally (e.g., left and right and / or up and down) as the respective object moves laterally (e.g., left and right and / or up and down) within the field of view of one or more cameras without attaching to the object, such that the size and / or orientation of the virtual object does not change as the distance of the respective object from the one or more cameras and / or the orientation of the respective object relative to the one or more cameras changes. For example, as shown in FIGS. 8AX and 8AY, helmet sticker 858-1 is positioned separate from a representation of a subject. In FIGS. 8AX and 8AY, as the representation of the subject moves sideways toward the camera, helmet sticker 858-1 moves laterally based on the movement of the user's representation, but does not change orientation or size.

[0312] In Figure 8AY, device 600 detects input 882 (e.g., a swipe or drag gesture) on scrubber 8120 and scrubs media item 820-4 based on the input. As shown in Figure 8AZ, in response to input 882, device 600 flips through media item 820-4 to display the portion of media item 820-4 that was previously displayed in Figure 8AX. Device 600 detects input 884 on edit affordance 878.

[0313] In FIG. 8BA, device 600 enters edit mode (to edit media item 820-4) and displays a selectable affordance 885 to crop the video by dragging affordance 885 (via input 883) to the end of video scrubber 8120 to change the end time of the video scrubber.

[0314] As shown in Figure 8BB, the video scrubber 8120 is trimmed in response to the input 883. The device 600 detects the input 886 on the completion affordance 887, causing the edit mode to end.

[0315] In Figure 8BC, video scrubber 8120 is shown cropped (e.g., shorter in length than before entering edit mode in Figure 8AZ). Device 600 detects input 888 on effects affordance 822 and exits effects mode.

[0316] As depicted in FIG. 8BD, the visual effect displayed on media item 820-4 is hidden from view in response to detecting input 888 that deselects effect affordance 822.

[0317] In some embodiments, a sticker may have a behavior that is determined based on conditions of the sticker's placement (e.g., the sticker's position relative to other objects, the presence (or absence) of an object when the sticker is placed, etc.). For example, in some embodiments, a sticker may have a first type of behavior when placed away from an object or area, and a second type of behavior when placed on an object or area. In some embodiments, if no object is present when the sticker is placed, the sticker may have a third type of behavior. In some embodiments, the behavior of a sticker may change based on changes in the sticker's placement (e.g., relative to an object).

[0318] For example, Figures 8BE-8BQ show exemplary embodiments illustrating various sticker behaviors displayed in live camera preview 820-1 that change based on changes in sticker placement relative to the subject's face. In the embodiments shown in Figures 8BE-8BQ, the sticker behavior is determined based on the sticker's position relative to the subject's face.

[0319] 8BE, device 600 displays a camera application user interface 815 similar in appearance to that shown in FIG. 8D, but with stickers 858 displayed in image display area 820. Image display area 820 shows a live camera preview 820-1 from camera 602, showing a representation of subject 832 located in the field of view of camera 602 and a background 836 displayed behind subject 832. Glasses sticker 858-4, starfish sticker 858-5, and baseball sticker 858-6 are each displayed positioned away from (e.g., not on) the face of subject 832. The stickers shown in FIGS. 8BE-8BQ can be positioned and moved within the image display area according to various embodiments discussed herein.

[0320] When the sticker 858 is placed away from (e.g., not on) the subject's face, the sticker 858 has a first type of behavior: for example, the sticker follows lateral (e.g., side-to-side and up / down), forward (e.g., towards the camera 602), and backward (e.g., away from the camera 602) movements of the subject's face, but does not follow rotational movements of the subject's face (e.g., does not follow the pitch and yaw of the subject's face).

[0321] 8BF, device 600 detects input 889 on glasses sticker 858-4 (e.g., a touch-and-drag gesture on display 601). In response to detecting input 889, device 600 displays glasses sticker 858-4 that moves with the input (e.g., in the direction and magnitude of the movement of input 889), as shown in FIGS.

[0322] In some embodiments, device 600 provides an indication when the sticker moves to a position corresponding to the object it is following. For example, in FIG. 8BG , device 600 indicates that eyeglasses sticker 858-4 is moving from a position away from the subject's face to a position on the subject's face. As eyeglasses sticker 858-4 moves to the subject's face, device 600 displays bracket 890 around the subject's face and, optionally, generates haptic feedback 892 (e.g., tactile output with or without audio output). In some embodiments, bracket 890 provides a visual indication of an area where eyeglasses sticker 858-4 can be placed so that it will appear on the subject's face (resulting in a modified behavior of eyeglasses sticker 858-4). In this manner, bracket 890 (and haptic feedback 892) can be indicated to the user while the user drags the sticker to a placement area (e.g., inside the bracket) where the sticker will be placed on the subject's face. In some embodiments, haptic feedback 892 indicates that glasses sticker 858-4 has moved from an area outside the subject's face (e.g., an area where the sticker has a first type of behavior) to an area on the subject's face (e.g., an area where the sticker has a second type of behavior).

[0323] In some embodiments, when device 600 detects that the position of the sticker (or other visual effect) is moving to a position corresponding to the object the sticker is following, device 600 changes the appearance of the sticker and changes the behavior of the sticker based on the position of the object (in some embodiments, the behavior of the sticker is changed after detecting the end of input 889). Device 600 also changes the appearance and behavior of the sticker in the opposite manner when the sticker moves from a position corresponding to the object the sticker is following to a position away from the object.

[0324] In FIG. 8BH, when device 600 detects the position of eyeglasses sticker 858-4 on the subject's face (e.g., located within bracket 890), device 600 modifies the sticker based on the position (e.g., size and orientation) of the subject's face. For example, eyeglasses sticker 858-4 increases in size (e.g., to proportionally fit the size of the subject's face) and rotates from its static position in FIG. 8BG to match the position of the subject's face in FIG. 8BH. In some embodiments, the sticker is placed on a plane corresponding to the subject's nose and transforms its coordinates as the face moves. In some embodiments, a face mesh (e.g., a depth mask) is used to determine the nose point. In some embodiments, the sticker is placed on a plane but does not fit the shape of the subject's face.

[0325] Device 600 also changes the behavior of the sticker to a second type of behavior (e.g., a behavior different from the first type of behavior). As shown in FIG. 8BH, the second type of behavior can include, in addition to side-to-side and forward-to-backward movement, behavior in which the sticker follows the rotational movement of the subject's face (e.g., follows the pitch and yaw of the face) to maintain its 3D position on the face. The behavior of the other stickers remains unchanged. Thus, baseball sticker 858-6 and starfish sticker 858-5 continue to have the first type of behavior because they are not positioned on the subject's face. Instead, starfish sticker 858-5 and baseball sticker 858-6 continue to maintain their relative distance from the face.

[0326] In some embodiments, bracket 890 persists along with input 889, with input 889 positioned on the subject's face (e.g., within the bracket). Thus, when device 600 detects the end of input 889, device 600 stops displaying bracket 890, as shown in FIG. 8BI.

[0327] 8BI and 8BJ show device 600 rotating glasses sticker 858-4 along with the subject's face, while starfish sticker 858-5 and baseball sticker 858-6 remain stationary (unlike glasses sticker 858-4, they do not follow the rotational movement of the subject's face).

[0328] In Figure 8BK, the subject moves closer to camera 602. Glasses sticker 858-4 moves with the forward movement of the subject's face and maintains its rotational position relative to the subject's face. Similarly, baseball sticker 858-6 and starfish sticker 858-5 move with the forward movement of the subject's face while maintaining their relative spacing to the subject's face. Baseball sticker 858-6 and starfish sticker 858-5 do not rotate with the subject's face.

[0329] In FIG. 8BK, device 600 detects input 893 on capture affordance 821 and, in response, captures an image of live camera preview 820-1.

[0330] 8BL, device 600 displays camera application user interface 815 showing a captured image (e.g., media item 820-5) in image display area 820 (e.g., media item 820-5 replaces live camera preview 820-1 shown in image display area 820). Media item 820-5 is a representation of live camera preview 820-1 at the time capture affordance 821 is selected. The embodiment shown in FIG. 8BL is similar to that shown in FIG. 8AM, but includes media item 820-5 (instead of media item 820-2), which includes glasses sticker 858-4, starfish sticker 858-5, and baseball sticker 858-6 displayed against subject 832's face and background 836.

[0331] As discussed herein, the display of visual effects is similar across different embodiments. For example, unless otherwise specified, visual effects may be displayed and manipulated in a similar manner in a camera application, a messaging application, an avatar editing application, a live video messaging application, or any other application discussed herein. Additionally, visual effects may be displayed and manipulated in a similar manner across different types of image data. For example, unless otherwise specified, visual effects may be displayed and manipulated in a similar manner in a live camera preview, a media item, streamed image data, or any other image data discussed herein. For example, FIGS. 8BL-8BQ illustrate displaying stickers on media item 820-5 in a similar manner as described above for live camera preview 820-1. Specifically, FIGS. 8BL-8BQ illustrate that stickers within a media item (e.g., 820-5) can be rearranged and their respective behaviors changed accordingly in a similar manner as described above for live image preview 820-1.

[0332] In FIG. 8BM, device 600 detects input 894 on eyeglasses sticker 858-4 and displays bracket 890. In FIG. 8BN, input 894 moves eyeglasses sticker 858-4 away from the subject's face. Upon moving the eyeglasses sticker away from the face, device 600 generates haptic feedback 892, stops displaying bracket 890, and changes eyeglasses sticker 858-4 back to its original shape (e.g., a slightly tilted shape from when the eyeglasses sticker was positioned away from the subject's face in FIG. 8BE). Device 600 also changes the behavior of eyeglasses sticker 858-4 back to a first type of behavior (a behavior associated with a sticker positioned away from the object it is tracking) and changes the sticker's appearance based on the changed behavior. Thus, glasses sticker 858-4 is displayed with a larger size in FIGS. 8BN-8BQ (compared to the size of glasses sticker 858-4 in FIGS. 8BE-8BG), maintaining the relative spacing of glasses sticker 858-4 to the position of the subject's face within media item 820-5 based on the forward movement of the subject's face in FIGS. 8BJ-8BK, as well as the size changes of starfish sticker 858-5 and baseball sticker 858-6.

[0333] 8BO-8BQ illustrate the placement of different stickers on a subject's face in media item 820-5 and the different stickers changing based on different sticker behaviors as they are moved over the subject's face. For example, in FIG. 8BO, device 600 detects input 895 on baseball sticker 858-6. When baseball sticker 858-6 is dragged over the subject's face in FIG. 8BP, device 600 displays bracket 890, generates haptic feedback 892 (e.g., tactile output), and repositions baseball sticker 858-6 based on the angle of the subject's face. In other words, the behavior of baseball sticker 858-6 changes to follow the rotational movement of the subject's face, similar to how eyeglasses sticker 858-4 follows the rotational movement of the subject's face when placed on the subject's face.

[0334] In FIG. 8BQ, input 895 ends and device 600 stops displaying bracket 890.

[0335] 9A-9B are flow diagrams illustrating a method for displaying visual effects in a camera application using an electronic device, according to some embodiments. Method 900 is performed on a device (e.g., 100, 300, 500, 600) that includes a camera and a display device. Some operations in method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0336] As described below, method 900 provides an intuitive way to display visual effects in a camera application. Such a method reduces the cognitive burden on a user to apply visual effects to images viewed in the camera application, thereby creating a more efficient human-machine interface. In battery-operated computing devices, enabling users to more quickly and efficiently display visual effects on images conserves power and increases the time between battery charges.

[0337] An electronic device (e.g., 600) displays (902) a camera user interface (e.g., 815) via a display device (e.g., 601). The camera user interface includes (904) a camera display area (e.g., 820) that includes a representation (e.g., 835) of image data captured via the camera (e.g., 602).

[0338] In some embodiments, the image data includes depth data (image data that includes depth aspects of the captured image or video (e.g., depth data independent of RGB data)) (906). In some embodiments, the image data includes at least two components: RGB components that encode visual characteristics of the captured image, and depth data that encodes information about the relative spatial relationships of elements in the captured image (e.g., the depth data encodes that the user is in the foreground, while background elements, such as trees located behind the user, are in the background). In some embodiments, the depth data is a depth map. In some embodiments, a depth map (e.g., a depth map image) includes information (e.g., values) about the distance of objects in a scene from a viewpoint (e.g., a camera). In one depth map embodiment, each depth pixel defines the position in the Z-axis of the viewpoint where its corresponding two-dimensional pixel is located. In some examples, a depth map is composed of pixels, where each pixel is defined by a value (e.g., 0 to 255). For example, a value of "0" represents the pixel located furthest away in a "3D" scene, and a value of "255" represents the pixel closest to the viewpoint (e.g., camera) in the "3D" scene. In other examples, the depth map represents the distance between an object in a scene and the plane of the viewpoint. In some embodiments, the depth map contains information about the relative depth of various features of an object of interest in the viewpoint of a depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears on a user's face). In some embodiments, the depth map contains information that enables the device to determine the contours of the object of interest in the z-direction.

[0339] In some embodiments, the depth data has a first depth component (e.g., a first portion of the depth data encoding the spatial location of the subject within the camera viewing area, a plurality of depth pixels forming a discrete portion of a depth map, such as a foreground or particular object) that includes a representation of the subject within the camera viewing area (e.g., 820). In some embodiments, the depth data has a second depth component (e.g., a second portion of the depth data encoding the spatial location of a background within the camera viewing area, a plurality of depth pixels forming a discrete portion of a depth map, such as a background) that is separate from the first depth component, and the second depth aspect includes a representation of the background in the camera viewing area. In some embodiments, the first depth aspect and the second depth aspect are used to determine a spatial relationship between the subject within the camera viewing area and the background within the camera viewing area. This spatial relationship can be used to distinguish the subject from the background. This distinction can be utilized, for example, to apply different visual effects (e.g., visual effects having a depth component) to the subject and the background. In some embodiments, all regions of the image data that do not correspond to the first depth component (e.g., regions of the image data that are outside the range of the depth camera) are segmented (e.g., excluded) from the depth map.

[0340] In some embodiments, the representation (e.g., 835) of image data captured via a camera (e.g., 602) is a live camera preview (e.g., a stream of image data representing what is in the camera's field of view).

[0341] In some embodiments, while the first camera viewing mode is active, the electronic device (e.g., 600) detects a swipe gesture on the camera viewing area (e.g., 820). In some embodiments, in response to detecting the swipe gesture on the camera viewing area, for example, the electronic device (e.g., 600) changes the appearance of a displayed representation of a selected avatar option in the camera viewing area from a first appearance (e.g., an appearance based on a currently selected avatar option) to a second appearance (e.g., an appearance based on a different avatar option (e.g., a null avatar option, or an avatar option corresponding to a different avatar, including an avatar of a different type (e.g., customizable, non-customizable))), where the second appearance corresponds to a different one of a plurality of avatar options (e.g., a different avatar option included in the avatar selection area). Changing the appearance of the displayed representation of the selected avatar option in response to detecting a swipe gesture on the camera viewing area provides a user with a quick and easy way to change the representation of the selected avatar. Providing additional control options without cluttering the user interface (UI) with additional displayed controls improves usability of the device, makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and also reduces power usage and improves battery life of the device by allowing the user to use the device more quickly and efficiently.

[0342] In some embodiments, if a different one of the plurality of avatar options is a null avatar option, the device (e.g., 600) stops displaying a representation of the avatar over the representation of the subject (e.g., the device stops replacing image data of the user's head with a virtual avatar). In some embodiments, if a different one of the plurality of avatar options is an avatar option for a different avatar character (including a customizable or non-customizable avatar character), the device replaces the selected avatar character with the different avatar character (e.g., the device replaces the representation of the avatar with a representation of the different avatar). In some embodiments, replacing the selected avatar character with the different avatar character includes displaying an animation of the different avatar character moving to the center of the screen. In some embodiments, replacing the selected avatar character with the different avatar character includes displaying an animation of the different avatar character moving to the user's head. In some embodiments, replacing the selected avatar character with the different avatar character includes blurring the background while the selected avatar is being replaced. When / while a selected avatar character is being replaced with a different avatar character, visual feedback is provided that the avatar character is being changed by displaying an animation (e.g., a different avatar character moves to the center of the screen, a different avatar character moves to the user's head and blurs the background). Providing improved visual feedback to the user improves usability of the device, makes the user-device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and additionally reduces device power usage and improves battery life by allowing the user to use the device more quickly and efficiently.In some embodiments, the currently selected avatar option corresponds to a first type of avatar (e.g., a customizable avatar) and the different avatar option corresponds to a second type of avatar (e.g., a non-customizable avatar).

[0343] In some embodiments, changing the appearance of the displayed representation of the selected avatar option in the camera viewing area (e.g., 820) from a first appearance to a second appearance includes moving a first version of the representation of the selected avatar option having the first appearance off the display. In some embodiments, changing the appearance of the displayed representation of the selected avatar option in the camera viewing area from the first appearance to the second appearance includes substantially moving a second version of the representation of the selected avatar option having the second appearance to the center of the display. Moving the first version of the representation of the selected avatar option off the display and moving the second version of the representation of the selected avatar option to the center of the display provides visual feedback that the first version has been replaced with the second version. Providing improved visual feedback to the user improves device usability and makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently.

[0344] In some embodiments, changing the appearance of the displayed representation of the selected avatar option from a first appearance to a second appearance includes moving a first version of the representation of the selected avatar option having the first appearance off the display. In some embodiments, changing the appearance of the displayed representation of the selected avatar option from the first appearance to the second appearance includes moving a second version of the representation of the selected avatar option having the second appearance into substantially the position of the representation of the subject displayed in the camera viewing area (e.g., 820). Moving the first version of the representation of the selected avatar option off the display and moving the second version of the representation of the selected avatar option into substantially the position of the representation of the subject displayed in the camera viewing area provides visual feedback that the first version has been replaced with the second version. Providing improved visual feedback to the user improves device usability and makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), as well as reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently.

[0345] In some embodiments, changing the appearance of the displayed representation of the selected avatar option from a first appearance to a second appearance includes changing the visual appearance of the background displayed in the camera viewing area (e.g., 820) (e.g., blurring the background, desaturating the background).

[0346] The camera user interface (908) also includes a first affordance (e.g., an affordance corresponding to a virtual avatar) associated with a first camera display mode (e.g., a mode in which image data of the user's head is replaced with a virtual avatar).

[0347] In some embodiments, the camera user interface (e.g., 815) further includes a sticker affordance (e.g., 824-2, an affordance corresponding to a function for enabling display of stickers) associated with a sticker display mode (e.g., a mode in which stickers can be applied to image data). In some embodiments, while displaying the image data (and, optionally, a representation of a selected avatar option) in the camera viewing area (e.g., 820), the electronic device (e.g., 600) detects a gesture (e.g., FIG. 8AH) directed toward the sticker affordance. In some embodiments, in response to detecting the gesture directed toward the sticker affordance, the electronic device activates the sticker display mode, wherein activating the sticker display mode includes displaying a sticker selection area (e.g., 856) including a plurality of sticker options (e.g., sticker 858), detecting a selection of one of the plurality of sticker options (e.g., sticker 858-2) in the sticker selection area, and, in response to detecting the selection, displaying a representation of the selected sticker option (e.g., sticker 858-2) over the image data in the camera viewing area. In some embodiments, the selected sticker option displayed over the image data has the appearance of being placed on the display screen (e.g., 601) (e.g., similar to the appearance of a user placing a physical sticker on the screen of a device) and does not move or interact with objects represented in the image data. In some embodiments, the selected sticker option displayed over the image data has the appearance of being placed on the display screen (e.g., similar to the appearance of a user placing a physical sticker on the screen of a device), but moves based on the movement of objects represented in the image data. For example, the sticker appears to be placed on the display and interacts with objects represented in the image data (e.g., people), but the movement of the sticker is limited to movement along the x and y axes.In other words, the sticker appears to be applied to the device screen, but can be moved across the screen by the object represented in the image data (e.g., a person in the image data appears to touch the sticker or drag the sticker across the screen). In some embodiments, a selected sticker option displayed on the image data has the appearance of being inserted into the image data as an interactive object forming part of the image data. Providing the appearance of being inserted into the image data as an interactive object forming part of the image data provides visual feedback that the selected sticker can act as an interactive object in the image. Providing improved visual feedback to the user improves device usability, making the user-device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and additionally reducing device power usage and improving battery life by allowing the user to use the device more quickly and efficiently. In such embodiments, the sticker appears to move based on interaction with the object represented in the image data. For example, the sticker may appear to be sitting on a person's shoulder. As the person moves, the sticker moves with the person's shoulder, maintaining the appearance of sitting on the person's shoulder. This includes movement along the x-axis and y-axis, as well as movement along the z-axis.

[0348] In some embodiments, while displaying a representation of a selected sticker option (e.g., 858-1, 858-2, 858-3, 858-4, 858-5, 858-6) on image data in a camera viewing area (e.g., 820), a device (e.g., 600) detects lateral movement of a subject (e.g., 832) within the field of view of one or more cameras. In response to detecting lateral movement of the subject within the field of view of one or more cameras, the device moves a representation of the selected sticker option laterally (e.g., regardless of the sticker's relationship to the subject) in accordance with the movement of the subject within the field of view of one or more cameras (e.g., see helmet sticker 858-1 in FIGS. 8AV-8AY). Moving the representation of the selected sticker option laterally in accordance with the lateral movement of the subject within the field of view of one or more cameras provides visual feedback that the selected sticker can act as an interactive object within the image. Providing improved visual feedback to the user improves the usability of the device, makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and additionally reduces the device's power usage and improves battery life by allowing the user to use the device more quickly and efficiently.

[0349] In some embodiments, while displaying a representation of a selected sticker option (e.g., 858-1, 858-2, 858-3, 858-4, 858-5, 858-6) on the image data in the camera viewing area (e.g., 820), the device (e.g., 600) detects a rotation (e.g., rotation about an axis perpendicular to the display, e.g., the subject turning its head) of a subject (e.g., 832) in the field of view of one or more cameras (e.g., 602). In response to detecting a rotation of a subject in the field of view of one or more cameras, the device performs one or more of the following steps: Following a determination that the representation of the selected sticker option has (was positioned in) a first relationship to the subject (e.g., the sticker was initially (or currently) positioned at a location on the display corresponding to the subject, e.g., the sticker is positioned on a representation of the subject's face or other designated area (e.g., bracket (e.g., 890))), the device rotates the representation of the selected sticker option in accordance with the magnitude and orientation of the subject's rotation (e.g., the sticker rotates and twists to follow the pitch and yaw of the subject's face) (e.g., see eyeglasses sticker 858-4 in Figures 8BH-8BM). Following a determination that the representation of the selected sticker option does not have (e.g., was not positioned at) the first relationship to the subject (e.g., the sticker was initially (or currently) positioned at a location on the display away from the subject, e.g., the sticker was placed outside the subject's facial representation or other designated area), the device ceases rotating the representation of the selected sticker option according to the magnitude and direction of the subject's rotation (e.g., see starfish sticker 858-5 and baseball sticker 858-6 in Figures 8BE-8BN). Selectively rotating the representation of the selected sticker option based on whether the representation of the selected sticker option has the first relationship to the subject provides visual feedback about the behavior of the selected sticker option and indicates that the selected sticker option can function as an interactive object in the image.Providing improved visual feedback to the user improves the usability of the device, makes the user device interface more efficient (e.g., by assisting the user in providing appropriate inputs when operating / interacting with the device and reducing user errors), and additionally reduces the device's power usage and improves battery life by allowing the user to use the device more quickly and efficiently.

[0350] In some embodiments, while displaying a representation of a selected sticker option (e.g., 858-1, 858-2, 858-3, 858-4, 858-5, 858-6) on image data in a camera viewing area (e.g., 820), a device (e.g., 600) detects movement of a subject (e.g., 832) toward (or away from) one or more cameras (e.g., 602). In response to detecting movement of the subject toward (or away from) one or more cameras, the device performs one or more of the following procedures: Following a determination that the representation of the selected sticker option has (e.g., was placed in) a first relationship to the subject (e.g., the sticker was initially (or currently) placed in a position on the display corresponding to the subject, e.g., the sticker was placed when a representation of the subject's face was present (e.g., detected) within the field of view of a camera), the device enlarges (or reduces) the representation of the selected sticker option according to the magnitude of the subject's movement toward (or away from) one or more cameras. For example, rabbit sticker 858-2 expands as a representation of the subject's shoulder moves toward the camera, as shown in FIGS. 8AX and 8AY. In another example, starfish sticker 858-5 and baseball sticker 858-6 expand as the subject moves toward the camera in FIGS. 8BJ-8BK. Following a determination that the representation of the selected sticker option does not have (e.g., was not placed) a first relationship to the subject (e.g., the sticker was initially (or currently) placed at a position on the display away from the subject; e.g., the sticker was placed when a representation of the subject's face was not present (e.g., not detected) within the field of view of the camera), the device stops expanding the representation of the selected sticker option according to the magnitude o...

Claims

1. An electronic device having a display device and one or more input devices, displaying a representation of the pre-captured image data via said display device; displaying a representation of an object within the representation of the pre-captured image data; and displaying a representation of a first portion of a background of the subject in the representation of the pre-captured image data; displaying a representation of a first virtual avatar over a first portion of the representation of the subject in the representation of the pre-captured image data; displaying a set of one or more virtual avatar options via said display device while displaying a representation of said pre-captured image data; detecting, via the one or more input devices, a selection of each avatar option in the set of one or more virtual avatar options, each avatar option corresponding to a second virtual avatar different from the first virtual avatar; In response to detecting a selection of each of said avatar options, ceasing to display the representation of the first virtual avatar over the first portion of the representation of the subject in the representation of the pre-captured image data while maintaining display of the first portion of the background of the subject in the representation of the pre-captured image data; and displaying a representation of the second virtual avatar over the first portion of the representation of the subject in the representation of the pre-captured image data at a position in the representation of the pre-captured image data previously occupied by the first virtual avatar, without displaying the representation of the second virtual avatar over the first portion of the background of the subject in the representation of the pre-captured image data.

2. The method of claim 1 , wherein the pre-captured image data is pre-captured video data.

3. detecting a change in a pose of the subject in the pre-captured video data while displaying a representation of the first virtual avatar over a first portion of the representation of the subject in the representation of the pre-captured image data; and modifying the representation of the first virtual avatar in response to detecting a change in pose of the subject in the pre-captured video data.

4. detecting a change in a pose of the subject in the pre-captured video data while displaying the representation of the second virtual avatar over a first portion of the representation of the subject in the representation of the pre-captured image data; and modifying the representation of the second virtual avatar in response to detecting a change in pose of the subject in the pre-captured video data.

5. Detecting changes in the subject's pose in the pre-captured video data includes detecting changes in the subject's pose; modifying the representation of the first virtual avatar includes modifying the representation of the first virtual avatar based on each change in the subject's pose; The method of claim 4 , wherein modifying the representation of the second virtual avatar comprises modifying the representation of the second virtual avatar based on each change in the subject's pose.

6. detecting a change in a facial expression of the subject in the pre-captured video data while displaying the representation of the first virtual avatar or the representation of the second virtual avatar over a first portion of the representation of the subject in the representation of the pre-captured image data; 3. The method of claim 2, further comprising: modifying a facial expression of the representation of the first virtual avatar or a facial expression of the representation of the second virtual avatar in response to detecting a change in a facial expression of the subject in the pre-captured video data.

7. Displaying the representation of the first virtual avatar or the representation of the second virtual avatar over a first portion of the representation of the subject in the representation of the pre-captured image data includes: displaying the representation of the first virtual avatar or the representation of the second virtual avatar within the first region of the representation of the subject within the representation of the pre-captured image data in accordance with a determination that a position of the subject's head corresponds to a first region of the representation of the subject within the representation of the pre-captured image data; and displaying the representation of the first virtual avatar or the representation of the second virtual avatar within the second region of the representation of the subject in the representation of the pre-captured image data in accordance with a determination that a position of the subject's head corresponds to a second region different from the first region of the representation of the subject in the representation of the pre-captured image data.

8. the pre-captured image data includes depth data of the object; the representation of the first virtual avatar is positioned at a simulated depth relative to the representation of the subject in the representation of the pre-captured image data determined based on the depth data of the subject; 8. The method of claim 1, wherein the representation of the second virtual avatar is positioned at a simulated depth relative to the representation of the subject in the representation of the pre-captured image data determined based on the depth data of the subject.

9. In response to detecting a selection of each of the avatar options, 10. The method of claim 1, further comprising: displaying a second portion of the background of the subject in the representation of the pre-captured image data that was previously obscured by the representation of the first virtual avatar.

10. displaying the representation of pre-captured image data includes displaying a representation of a first portion of the object within the representation of the pre-captured image data; Depending on the selection of each avatar option, The method of any one of claims 1 to 9, further comprising maintaining a representation of a first portion of the background of the subject in the representation of the pre-captured image data.

11. capturing, via one or more cameras, a live feed of the subject within a field of view of the one or more cameras prior to displaying the representation of pre-captured image data; While capturing a live feed of said subject, displaying a live feed representation of the subject via the display device; The method of any one of claims 1 to 10, further comprising: displaying the representation of the first virtual avatar over at least a portion of the live-feed representation of the subject.

12. detecting, via the one or more input devices, a request to initiate transmission of the pre-captured image data while displaying the representation of the second virtual avatar over the first portion of the representation of the subject in the representation of the pre-captured image data; and displaying, via the display device, a representation of the pre-captured image data in a message composition field of a messaging user interface in response to detecting a request to initiate transmission of the pre-captured image data; The method of any preceding claim, wherein the representation of the pre-captured image data in the message composition field includes a second representation of the second virtual avatar.

13. A computer program that causes a computer to execute the method according to any one of claims 1 to 12.

14. A display device; one or more input devices; a memory for storing the computer program of claim 13; and one or more processors capable of executing the computer programs stored in the memory.

15. An electronic device comprising means for carrying out the method according to any one of claims 1 to 12.

Images