User interface for controlling media capture settings

Abstract

A user interface for controlling media capture settings is described, comprising a user interface for controlling capture of media with associated depth information (e.g., for applying a simulated depth-of-field effect), controlling media capture settings with multiple input types, controlling zoom settings for media capture (e.g., for applying a simulated depth-of-field effect), and controlling the zoom settings for media capture (e.g., for applying a simulated depth-of-field effect). Auxiliary previews are conditionally displayed at different zoom levels), using internal and external media stores to capture and view media and control indicators for level capture guidance.

Description

[0001] This application is a divisional application of patent application No. 202480030665.9, filed on April 26, 2024, entitled "User Interface for Controlling Media Capture Settings". Cross-references to related applications

[0002] This application claims U.S. Patent Application No. 18 / 373,158, filed September 26, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”; U.S. Patent Application No. 18 / 373,163, filed September 26, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”; U.S. Patent Application No. 18 / 373,168, filed September 26, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”; U.S. Patent Application No. 18 / 373,171, filed September 26, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”; and U.S. Patent Application No. 18 / 373,171, filed September 11, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”. The U.S. Provisional Patent Application No. 63 / 537,800, entitled “CAPTURE SETTINGS”, filed June 2, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”, and filed May 5, 2023, entitled “USER INTERFACES FOR CONTROLLING MEDIA CAPTURE SETTINGS”, all of which are hereby cited in their entirety. Technical Field

[0003] This disclosure relates generally to computer user interfaces, and more specifically to techniques for controlling media capture settings. Background Technology

[0004] Since the advent of virtual cameras (camera applications such as those on smartphones and other personal electronic devices), the range of functionalities available to users for capturing, storing, and editing photo and video media has expanded significantly. For example, virtual cameras offer numerous options for focus settings, zoom settings, lens settings, lighting settings, color settings, media size, media format, visual effects, and other aspects of media capture and management. Summary of the Invention

[0005] However, some technologies used to control media capture settings with electronic devices are often cumbersome and inefficient. For example, some existing technologies for controlling media capture settings use complex and time-consuming user interfaces that may include numerous inputs that need to be accessed and used. Existing technologies require more time and effort than actually needed to find, control, and manage media capture settings, thus wasting user time and device energy. This latter consideration is particularly important in battery-powered devices.

[0006] Therefore, this technology provides electronic devices with faster and more efficient methods and interfaces for controlling media capture settings. Such methods and interfaces optionally complement or replace other methods for controlling media capture settings. These methods and interfaces reduce the cognitive burden on the user and result in a more efficient human-machine interface. They reduce the processing power required by the device to display the user interface, receive input through the user interface, and respond to user input. For battery-powered computing devices, these methods and interfaces save power and increase the time interval between battery charging sessions.

[0007] According to some embodiments, a method is described. The method is executed at a computer system communicating with a display generation component, one or more cameras, and one or more sensors, and includes: when displaying a camera preview via the display generation component, obtaining information about a physical environment corresponding to the field of view of the one or more cameras based on information from the one or more sensors of the computer system; and in response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: based on the computer system determining that a depth capture criterion is met based on the information about the physical environment corresponding to the field of view of the one or more cameras, displaying a first optional user interface object via the display generation component, the first optional user interface object initiating a process for performing a first depth-based media capture function when selected; and abandoning the display of the first optional user interface object based on the determination that the depth capture criterion is not met.

[0008] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component, one or more cameras, and one or more sensors. The one or more programs include instructions for: obtaining information about the physical environment corresponding to the field of view of the one or more cameras based on information from one or more sensors of the computer system when displaying a camera preview via the display generation component; and in response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: displaying a first optional user interface object via the display generation component based on a determination by the computer system, based on the information about the physical environment corresponding to the field of view of the one or more cameras, that a depth capture criterion is met, the first optional user interface object initiating a process for performing a first depth-based media capture function when selected; and abandoning the display of the first optional user interface object based on a determination that the depth capture criterion is not met.

[0009] According to some embodiments, a transient computer-readable storage medium is described. This transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component, one or more cameras, and one or more sensors. The one or more programs include instructions for: obtaining information about the physical environment corresponding to the field of view of the one or more cameras based on information from one or more sensors of the computer system when displaying a camera preview via the display generation component; and in response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: displaying a first optional user interface object via the display generation component based on a determination by the computer system, based on the information about the physical environment corresponding to the field of view of the one or more cameras, that a depth capture criterion is met, the first optional user interface object initiating a process for performing a first depth-based media capture function when selected; and abandoning the display of the first optional user interface object based on a determination that the depth capture criterion is not met.

[0010] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component, one or more cameras, and one or more sensors. The computer system includes one or more processors and memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: obtaining information about the physical environment corresponding to the field of view of the one or more cameras based on information from the one or more sensors of the computer system when displaying a camera preview via the display generation component; and in response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: displaying a first optional user interface object via the display generation component based on a determination by the computer system, based on the information about the physical environment corresponding to the field of view of the one or more cameras, that a depth capture criterion is met, initiating a process for performing a first depth-based media capture function when the first optional user interface object is selected; and abandoning the display of the first optional user interface object based on a determination that the depth capture criterion is not met.

[0011] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component, one or more cameras, and one or more sensors. The computer system includes: means for obtaining information about a physical environment corresponding to the field of view of the one or more cameras based on information from the one or more sensors of the computer system when displaying a camera preview via the display generation component; and, in response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: means for displaying a first optional user interface object via the display generation component based on the information from the computer system regarding the physical environment corresponding to the field of view of the one or more cameras, indicating that a depth capture criterion is met, the first optional user interface object, when selected, initiating a process for performing a first depth-based media capture function; and means for abandoning the display of the first optional user interface object based on the determination that the depth capture criterion is not met.

[0012] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component, one or more cameras, and one or more sensors. The one or more programs include instructions for: obtaining information about the physical environment corresponding to the field of view of the one or more cameras based on information from one or more sensors of the computer system when displaying a camera preview via the display generation component; and in response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: displaying a first optional user interface object via the display generation component based on a determination by the computer system, based on the information about the physical environment corresponding to the field of view of the one or more cameras, that a depth capture criterion is met, the first optional user interface object initiating a process for performing a first depth-based media capture function when selected; and abandoning the display of the first optional user interface object based on a determination that the depth capture criterion is not met.

[0013] According to some implementations, a method is described. The method is executed at a computer system communicating with a display generation component and a camera, and the method includes: displaying a camera user interface via the display generation component, wherein the camera user interface includes an optional user interface object corresponding to a current state of a first media capture setting, wherein the current state of the first media capture setting is a first state, and includes a plurality of states that can be used for the first media capture setting; detecting a first user input corresponding to the optional user interface object; in response to detecting the first user input: switching the first media capture setting to a second state among the plurality of states, different from the first state, based on determining that the first user input is a first type of input; and displaying an optional user interface object corresponding to a third state among the plurality of states of the first media capture setting, wherein selecting the optional user interface object corresponding to the third state causes the first media capture setting to switch to the third state, wherein the third state is different from both the first and second states.

[0014] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and a camera. The one or more programs include instructions for: displaying a camera user interface via the display generation component, wherein the camera user interface includes an optional user interface object corresponding to a current state of a first media capture setting, wherein the current state of the first media capture setting is a first state, and a plurality of states including the first state are available for use with the first media capture setting; detecting a first user input corresponding to the optional user interface object; in response to detecting the first user input: switching the first media capture setting to a second state among the plurality of states, different from the first state, based on determining that the first user input is a first type of input; and displaying an optional user interface object corresponding to a third state among the plurality of states of the first media capture setting, wherein selecting the optional user interface object corresponding to the third state causes the first media capture setting to switch to the third state, wherein the third state is different from both the first and second states.

[0015] According to some embodiments, a transient computer-readable storage medium is described. This transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and a camera. The one or more programs include instructions for: displaying a camera user interface via the display generation component, wherein the camera user interface includes an optional user interface object corresponding to a current state of a first media capture setting, wherein the current state of the first media capture setting is a first state, and a plurality of states including the first state are available for use with the first media capture setting; detecting a first user input corresponding to the optional user interface object; in response to detecting the first user input: switching the first media capture setting to a second state among the plurality of states, different from the first state, based on determining that the first user input is a first type of input; and displaying an optional user interface object corresponding to a third state among the plurality of states of the first media capture setting, wherein selecting the optional user interface object corresponding to the third state causes the first media capture setting to switch to the third state, wherein the third state is different from both the first and second states.

[0016] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and a camera. The computer system includes one or more processors and memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: displaying a camera user interface via the display generation component, wherein the camera user interface includes an optional user interface object corresponding to a current state of a first media capture setting, wherein the current state of the first media capture setting is a first state, and a plurality of states including the first state are available for the first media capture setting; detecting first user input corresponding to the optional user interface object; in response to detecting the first user input: switching the first media capture setting to a second state among the plurality of states, different from the first state, based on determining that the first user input is a first type of input; and displaying an optional user interface object corresponding to a third state among the plurality of states of the first media capture setting, wherein selecting the optional user interface object corresponding to the third state causes the first media capture setting to switch to the third state, wherein the third state is different from both the first and second states.

[0017] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and a camera. The computer system includes: components for displaying a camera user interface via the display generation component, wherein the camera user interface includes an optional user interface object corresponding to a current state of a first media capture setting, wherein the current state of the first media capture setting is a first state, and a plurality of states including the first state are available for use with the first media capture setting; components for detecting first user input corresponding to the optional user interface object; in response to detecting the first user input: components for switching the first media capture setting to a second state among a plurality of states, different from the first state, based on determining that the first user input is a first type of input; and components for displaying an optional user interface object corresponding to a third state among a plurality of states of the first media capture setting based on determining that the first user input is a second type of input, wherein selecting the optional user interface object corresponding to the third state switches the first media capture setting to the third state, wherein the third state is different from both the first and second states.

[0018] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to execute by one or more processors of a computer system communicating with a display generation component and a camera. The one or more programs include instructions for: displaying a camera user interface via the display generation component, wherein the camera user interface includes an optional user interface object corresponding to a current state of a first media capture setting, wherein the current state of the first media capture setting is a first state, and a plurality of states including the first state are available for the first media capture setting; detecting a first user input corresponding to the optional user interface object; in response to detecting the first user input: switching the first media capture setting to a second state among the plurality of states, different from the first state, based on determining that the first user input is a first type of input; and displaying an optional user interface object corresponding to a third state among the plurality of states of the first media capture setting, wherein selecting the optional user interface object corresponding to the third state causes the first media capture setting to switch to the third state, wherein the third state is different from both the first and second states.

[0019] According to some embodiments, a method is described. The method is performed at a computer system communicating with a display generation component and one or more cameras, and the method includes: when displaying a camera preview at a first zoom level via the display generation component, detecting a first input corresponding to a request to zoom the camera preview, the camera preview including a first representation of the field of view of one or more cameras, the field of view of the one or more cameras including a first portion of the environment; in response to detecting the first input: displaying, based on a request to zoom to a second zoom level above a corresponding zoom threshold, a camera preview at the second zoom level; and an auxiliary representation of the field of view of one or more cameras, the auxiliary representation being smaller than the camera preview and including a representation of a larger portion of the environment than included in the camera preview at the second zoom level; and displaying, based on a request to zoom to a third zoom level below a corresponding zoom threshold, a camera preview at the third zoom level including a second representation of the field of view of one or more cameras, the field of view of the one or more cameras including a second portion of the environment, without displaying the auxiliary representation of the field of view of the one or more cameras.

[0020] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generating component and one or more cameras. The one or more programs include instructions for: upon displaying a camera preview at a first zoom level via the display generating component, detecting a first input corresponding to a request to zoom the camera preview, the camera preview including a first representation of the field of view of one or more cameras, the field of view of the one or more cameras including a first portion of the environment; in response to detecting the first input: displaying, based on a request to zoom to a second zoom level above a corresponding zoom threshold, a camera preview at the second zoom level; and an auxiliary representation of the field of view of one or more cameras, the auxiliary representation being smaller than the camera preview and including a representation of a larger portion of the environment than included in the camera preview at the second zoom level; and displaying, based on a request to zoom to a third zoom level below a corresponding zoom threshold, a camera preview at the third zoom level including a second representation of the field of view of one or more cameras, the field of view of the one or more cameras including a second portion of the environment, without displaying the auxiliary representation of the field of view of the one or more cameras.

[0021] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generating component and one or more cameras. The one or more programs include instructions for: upon displaying a camera preview at a first zoom level via the display generating component, detecting a first input corresponding to a request to zoom the camera preview, the camera preview including a first representation of the field of view of one or more cameras, the field of view of the one or more cameras including a first portion of the environment; in response to detecting the first input: displaying, based on a request to zoom to a second zoom level above a corresponding zoom threshold, a camera preview at the second zoom level; and an auxiliary representation of the field of view of one or more cameras, the auxiliary representation being smaller than the camera preview and including a representation of a larger portion of the environment than included in the camera preview at the second zoom level; and displaying, based on a request to zoom to a third zoom level below a corresponding zoom threshold, a camera preview at the third zoom level including a second representation of the field of view of one or more cameras, the field of view of the one or more cameras including a second portion of the environment, without displaying the auxiliary representation of the field of view of the one or more cameras.

[0022] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more cameras. The computer system includes one or more processors and memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: when displaying a camera preview at a first zoom level via the display generation component, detecting a first input corresponding to a request to zoom the camera preview, the camera preview including a first representation of the field of view of the one or more cameras, the field of view of the one or more cameras including a first portion of the environment; in response to detecting the first input: displaying, based on a request to zoom to a second zoom level above a corresponding zoom threshold, a camera preview at the second zoom level; and an auxiliary representation of the field of view of the one or more cameras, the auxiliary representation being smaller than the camera preview and including a representation of a larger portion of the environment than included in the camera preview at the second zoom level; and displaying, based on a request to zoom to a third zoom level below a corresponding zoom threshold, a camera preview at the third zoom level including a second representation of the field of view of the one or more cameras, the field of view of the one or more cameras including a second portion of the environment, without displaying the auxiliary representation of the field of view of the one or more cameras.

[0023] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more cameras. The computer system includes: components for detecting a first input corresponding to a request to zoom the camera preview when the camera preview is displayed via the display generation component at a first zoom level, the camera preview including a first representation of the field of view of the one or more cameras, the field of view of the one or more cameras including a first portion of the environment; in response to detecting the first input: components for displaying the camera preview at the second zoom level based on a request to zoom to a second zoom level above a corresponding zoom threshold; and an auxiliary representation of the field of view of the one or more cameras, the auxiliary representation being smaller than the camera preview and including a representation of the environment larger than that included in the camera preview at the second zoom level; and components for displaying the camera preview at a third zoom level based on a request to zoom to a third zoom level below a corresponding zoom threshold, wherein the camera preview at the third zoom level includes a second representation of the field of view of the one or more cameras, the field of view of the one or more cameras including a second portion of the environment, without displaying the auxiliary representation of the field of view of the one or more cameras.

[0024] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to execute by one or more processors of a computer system communicating with a display generation component and one or more cameras. The one or more programs include instructions for: upon displaying a camera preview at a first zoom level via the display generation component, detecting a first input corresponding to a request to zoom the camera preview, the camera preview including a first representation of the field of view of one or more cameras, the field of view of the one or more cameras including a first portion of the environment; in response to detecting the first input: displaying, based on a request to zoom to a second zoom level above a corresponding zoom threshold, a camera preview at the second zoom level; and an auxiliary representation of the field of view of one or more cameras, the auxiliary representation being smaller than the camera preview and including a representation of a larger portion of the environment than included in the camera preview at the second zoom level; and displaying, based on a request to zoom to a third zoom level below a corresponding zoom threshold, a camera preview at the third zoom level including a second representation of the field of view of one or more cameras, the field of view of the one or more cameras including a second portion of the environment, without displaying the auxiliary representation of the field of view of the one or more cameras.

[0025] According to some embodiments, a method is described. The method is executed at a computer system communicating with a display generation component and a camera, and the method includes: detecting a request to view recently captured media while displaying a camera user interface via the display generation component; and: in response to detecting the request to view recently captured media: displaying a representation of a first set of one or more captured media items based on determining that one or more sets of external storage criteria are met, wherein the first set of one or more captured media items is stored on an external storage device communicating with the computer system and has not yet been added to a media library associated with the computer system; and displaying a representation of a second set of one or more captured media items, different from the first set of one or more captured media items, based on determining that the set of one or more external storage criteria are not met, wherein the second set of one or more captured media items is included in a media library associated with the computer system.

[0026] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and a camera. The one or more programs include instructions for: detecting a request to view recently captured media when a camera user interface is displayed via the display generation component; and: in response to detecting the request to view recently captured media: displaying a representation of a first set of one or more captured media items based on determining that one or more sets of external storage criteria are met, wherein the first set of one or more captured media items is stored on an external storage device communicating with the computer system and has not yet been added to a media library associated with the computer system; and displaying a representation of a second set of one or more captured media items, different from the first set of one or more captured media items, based on determining that the set of one or more external storage criteria are not met, wherein the second set of one or more captured media items is included in a media library associated with the computer system.

[0027] According to some embodiments, a transient computer-readable storage medium is described. This transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and a camera. The one or more programs include instructions for: detecting a request to view recently captured media when a camera user interface is displayed via the display generation component; and: in response to detecting the request to view recently captured media: displaying a representation of a first set of one or more captured media items based on determining that one or more sets of external storage criteria are met, wherein the first set of one or more captured media items is stored on an external storage device in communication with the computer system and has not yet been added to a media library associated with the computer system; and displaying a representation of a second set of one or more captured media items, different from the first set of one or more captured media items, based on determining that the set of one or more external storage criteria are not met, wherein the second set of one or more captured media items is included in a media library associated with the computer system.

[0028] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and a camera. The computer system includes one or more processors and memory configured to execute one or more programs by the one or more processors. The one or more programs include instructions for: detecting a request to view recently captured media when a camera user interface is displayed via the display generation component; and: in response to detecting the request to view recently captured media: displaying a representation of a first set of one or more captured media items based on determining that a set of one or more external storage criteria is met, wherein the first set of one or more captured media items is stored on an external storage device communicating with the computer system and has not yet been added to a media library associated with the computer system; and displaying a representation of a second set of one or more captured media items different from the first set of one or more captured media items based on determining that the set of one or more external storage criteria is not met, wherein the second set of one or more captured media items is included in a media library associated with the computer system.

[0029] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and a camera. The computer system includes components for: detecting a request to view recently captured media when a camera user interface is displayed via the display generation component; and: in response to detecting the request to view recently captured media: displaying a representation of a first set of one or more captured media items based on determining that one or more sets of external storage criteria are met, wherein the first set of one or more captured media items is stored on an external storage device communicating with the computer system and has not yet been added to a media library associated with the computer system; and displaying a representation of a second set of one or more captured media items, different from the first set of one or more captured media items, based on determining that the set of one or more external storage criteria are not met, wherein the second set of one or more captured media items is included in a media library associated with the computer system.

[0030] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to execute by one or more processors of a computer system communicating with a display generation component and a camera. The one or more programs include instructions for: detecting a request to view recently captured media when a camera user interface is displayed via the display generation component; and: in response to detecting the request to view recently captured media: displaying a representation of a first set of one or more captured media items, wherein the first set of one or more captured media items is stored on an external storage device communicating with the computer system and has not yet been added to a media library associated with the computer system, based on determination that the one or more external storage criteria are not met; and displaying a representation of a second set of one or more captured media items, different from the first set of one or more captured media items, wherein the second set of one or more captured media items is included in a media library associated with the computer system, based on determination that the one or more external storage criteria are not met.

[0031] According to some embodiments, a method is described. The method is performed at a computer system communicating with a display generation component and one or more cameras, and the method includes: when displaying a camera user interface via the display generation component a representation of a portion of the field of view of one or more cameras: based on determining that the portion of the field of view of the one or more cameras satisfies a set of content criteria, simultaneously displaying via the display generation component the representation of the portion of the field of view of the one or more cameras along with a level indicator, the level indicator indicating the orientation of the field of view of the one or more cameras relative to a corresponding orientation, wherein the set of content criteria includes a first criterion based on whether first type of content is detected in the portion of the field of view of the one or more cameras; and based on determining that the portion of the field of view of the one or more cameras does not satisfy the set of content criteria, displaying the representation of the portion of the field of view of the one or more cameras without displaying the level indicator.

[0032] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more cameras. The one or more programs include instructions for: when displaying a camera user interface via the display generation component a representation of a portion of the field of view of one or more cameras: based on determining that the portion of the field of view of the one or more cameras satisfies a set of content criteria, simultaneously displaying via the display generation component the representation of the portion of the field of view of the one or more cameras along with a level indicator, the level indicator indicating the orientation of the field of view of the one or more cameras relative to a corresponding orientation, wherein the set of content criteria includes a first criterion based on whether content of a first type is detected in the portion of the field of view of the one or more cameras; and based on determining that the portion of the field of view of the one or more cameras does not satisfy the set of content criteria, displaying the representation of the portion of the field of view of the one or more cameras without displaying the level indicator.

[0033] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more cameras. The one or more programs include instructions for: when displaying a camera user interface via the display generation component a representation of a portion of the field of view of one or more cameras: based on determining that the portion of the field of view of the one or more cameras satisfies a set of content criteria, simultaneously displaying via the display generation component the representation of the portion of the field of view of the one or more cameras along with a level indicator, the level indicator indicating the orientation of the field of view of the one or more cameras relative to a corresponding orientation, wherein the set of content criteria includes a first criterion based on whether first type of content is detected in the portion of the field of view of the one or more cameras; and based on determining that the portion of the field of view of the one or more cameras does not satisfy the set of content criteria, displaying the representation of the portion of the field of view of the one or more cameras without displaying the level indicator.

[0034] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component, one or more cameras, and one or more sensors. The computer system includes one or more processors and memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: when displaying a camera user interface via the display generation component a representation of a portion of the field of view of the one or more cameras: based on determining that the portion of the field of view of the one or more cameras satisfies a set of content criteria, simultaneously displaying the representation of the portion of the field of view of the one or more cameras along with a level indicator, the level indicator indicating the orientation of the field of view of the one or more cameras relative to a corresponding orientation, wherein the set of content criteria includes a first criterion based on whether first type of content is detected in the portion of the field of view of the one or more cameras; and based on determining that the portion of the field of view of the one or more cameras does not satisfy the set of content criteria, displaying the representation of the portion of the field of view of the one or more cameras without displaying the level indicator.

[0035] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more cameras. The computer system includes components for performing the following operations when a camera user interface, comprising a representation of a portion of the field of view of the one or more cameras, is displayed via the display generation component: based on determining that the portion of the field of view of the one or more cameras satisfies a set of content criteria, simultaneously displaying the representation of the portion of the field of view of the one or more cameras along with a level indicator, the level indicator indicating the orientation of the field of view of the one or more cameras relative to a corresponding orientation, wherein the set of content criteria includes a first criterion based on whether first type of content is detected in the portion of the field of view of the one or more cameras; and based on determining that the portion of the field of view of the one or more cameras does not satisfy the set of content criteria, displaying the representation of the portion of the field of view of the one or more cameras without displaying the level indicator.

[0036] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to execute by one or more processors of a computer system communicating with a display generation component and one or more cameras. The one or more programs include instructions for: when displaying a camera user interface via the display generation component a representation of a portion of the field of view of the one or more cameras: based on determining that the portion of the field of view of the one or more cameras satisfies a set of content criteria, simultaneously displaying the representation of the portion of the field of view of the one or more cameras along with a level indicator, the level indicator indicating the orientation of the field of view of the one or more cameras relative to a corresponding orientation, wherein the set of content criteria includes a first criterion based on whether first type of content is detected in the portion of the field of view of the one or more cameras; and based on determining that the portion of the field of view of the one or more cameras does not satisfy the set of content criteria, displaying the representation of the portion of the field of view of the one or more cameras without displaying the level indicator.

[0037] According to some embodiments, a method is described. The method includes: at a computer system communicating with a display generation component and one or more input devices: while simultaneously displaying a representation of a media item and a control element for adjusting a simulated depth-of-field effect of the media item via the display generation component: detecting input via one or more input devices; and in response to detecting the input: selecting, based on determining that the input includes a selection of a corresponding portion of the representation of the media item, a corresponding position of the representation of the media item corresponding to the corresponding portion of the representation of the media item selected by the input as the focal position of the simulated depth-of-field effect of the media item; and adjusting the control element and changing the magnitude of the simulated depth-of-field effect of the media item based on determining that the input includes a selection of the control element.

[0038] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting input via one or more input devices when simultaneously displaying a representation of a media item and a control element for adjusting a simulated depth-of-field effect of the media item via the display generation component; and in response to detecting the input: selecting a corresponding position of the representation of the media item corresponding to the corresponding portion of the representation of the media item selected by the input as the focal position of the simulated depth-of-field effect of the media item; and adjusting the control element and changing the magnitude of the simulated depth-of-field effect of the media item based on the determination that the input includes a selection of the control element.

[0039] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting input via one or more input devices when simultaneously displaying a representation of a media item and a control element for adjusting a simulated depth-of-field effect of the media item via the display generation component; and in response to detecting the input: selecting a corresponding position of the representation of the media item corresponding to the corresponding portion of the representation of the media item selected by the input as the focal position of the simulated depth-of-field effect of the media item; and adjusting the control element and changing the magnitude of the simulated depth-of-field effect of the media item based on the determination that the input includes a selection of the control element.

[0040] According to some embodiments, a computer system configured to communicate with a display generation component and one or more input devices is described. The computer system includes one or more processors and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: detecting input via one or more input devices when simultaneously displaying a representation of a media item and a control element for adjusting a simulated depth-of-field effect of the media item via the display generation component; and in response to detecting the input: selecting a corresponding position of the representation of the media item corresponding to the corresponding portion of the representation of the media item selected by the input as the focal position of the simulated depth-of-field effect of the media item; and adjusting the control element and changing the magnitude of the simulated depth-of-field effect of the media item based on the determination that the input includes a selection of the control element.

[0041] According to some embodiments, a computer system configured to communicate with a display generation component and one or more input devices is described. The computer system includes components for performing the following operations when simultaneously displaying a representation of a media item and a control element for adjusting a simulated depth-of-field effect of the media item via the display generation component: detecting input via one or more input devices; and in response to detecting the input: selecting a corresponding position of the representation of the media item corresponding to the corresponding portion of the representation of the media item selected by the input as the focal position of the simulated depth-of-field effect of the media item; and adjusting the control element and changing the magnitude of the simulated depth-of-field effect of the media item based on the determination that the input includes a selection of the control element.

[0042] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting input via one or more input devices when simultaneously displaying a representation of a media item and a control element for adjusting a simulated depth-of-field effect of the media item via the display generation component; and in response to detecting the input: selecting a corresponding position of the representation of the media item corresponding to the corresponding portion of the representation of the media item selected by the input as the focal position of the simulated depth-of-field effect of the media item; and adjusting the control element and changing the magnitude of the simulated depth-of-field effect of the media item based on the determination that the input includes a selection of the control element.

[0043] According to some embodiments, a method is described. The method includes: at a computer system communicating with a display generation component and one or more input devices: detecting, via the one or more input devices, a request to display controls for editing a representation of a media item; and in response to detecting the request to display controls for editing a representation of a media item, simultaneously displaying via the display generation component: a first control element for editing features of a set of one or more images that can be used for display in an image sequence when the representation of the media item is displayed via the display generation component; and a second control element for editing a simulated depth-of-field effect of the representation of the media item.

[0044] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: detecting a request via one or more input devices to display controls for editing a representation of a media item; and, in response to detecting the request to display controls for editing a representation of a media item, simultaneously displaying via the display generation component: a first control element for editing features of one or more images that can be used for display in an image sequence when the representation of the media item is displayed via the display generation component; and a second control element for editing a simulated depth-of-field effect of the representation of the media item.

[0045] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for: detecting a request via one or more input devices to display controls for editing a representation of a media item; and, in response to detecting the request to display controls for editing a representation of a media item, simultaneously displaying via the display generation component: a first control element for editing features of one or more images that can be used to display in an image sequence when the representation of the media item is displayed via the display generation component; and a second control element for editing a simulated depth-of-field effect of the representation of the media item.

[0046] According to some embodiments, a computer system configured to communicate with a display generation component and one or more input devices is described. The computer system includes one or more processors and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: detecting a request via the one or more input devices to display controls for editing a representation of a media item; and, in response to detecting the request to display controls for editing a representation of a media item, simultaneously displaying via the display generation component: a first control element for editing features of one or more images that can be used to display in an image sequence when the representation of the media item is displayed via the display generation component; and a second control element for editing a simulated depth-of-field effect of the representation of the media item.

[0047] According to some embodiments, a computer system configured to communicate with a display generation component and one or more input devices is described. The computer system includes: components for detecting a request via one or more input devices to display controls for editing a representation of a media item; and components for simultaneously displaying via the display generation component, in response to detecting the request to display controls for editing the representation of the media item: a first control element for editing features of one or more images that can be used for display in an image sequence when the representation of the media item is displayed via the display generation component; and a second control element for editing a simulated depth-of-field effect of the representation of the media item.

[0048] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for: detecting a request via one or more input devices to display controls for editing a representation of a media item; and, in response to detecting the request to display controls for editing a representation of a media item, simultaneously displaying via the display generation component: a first control element for editing features of one or more images that can be used to display in an image sequence when the representation of the media item is displayed via the display generation component; and a second control element for editing a simulated depth-of-field effect of the representation of the media item.

[0049] According to some embodiments, a method is described. The method includes: at a computer system in communication with a display generation component and one or more input devices: while displaying a representation of a media item including a simulated depth-of-field effect via the display generation component: detecting input directed to the representation of the media item including the simulated depth-of-field effect via one or more input devices; and in response to detecting the input directed to the representation of the media item, displaying a plurality of images corresponding to the media item via the display generation component, including displaying the plurality of images sequentially over time.

[0050] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for: when displaying a representation of a media item including a simulated depth-of-field effect via the display generation component: detecting input directed to the representation of the media item including the simulated depth-of-field effect via one or more input devices; and in response to detecting the input directed to the representation of the media item, displaying a plurality of images corresponding to the media item via the display generation component, including displaying the plurality of images sequentially over time.

[0051] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for: when displaying a representation of a media item including a simulated depth-of-field effect via the display generation component: detecting input directed to the representation of the media item including the simulated depth-of-field effect via one or more input devices; and in response to detecting the input directed to the representation of the media item, displaying a plurality of images corresponding to the media item via the display generation component, including displaying the plurality of images sequentially over time.

[0052] According to some embodiments, a computer system configured to communicate with a display generation component and one or more input devices is described. The computer system includes one or more processors and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: when displaying a representation of a media item including a simulated depth-of-field effect via the display generation component; detecting input directed to the representation of the media item including the simulated depth-of-field effect via the one or more input devices; and in response to detecting the input directed to the representation of the media item, displaying a plurality of images corresponding to the media item via the display generation component, including displaying the plurality of images sequentially over time.

[0053] According to some embodiments, a computer system configured to communicate with a display generation component and one or more input devices is described. The computer system includes: components for performing the following operations when displaying a representation of a media item including a simulated depth-of-field effect via the display generation component: detecting input directed to the representation of the media item including the simulated depth-of-field effect via one or more input devices; and, in response to detecting the input directed to the representation of the media item, displaying a plurality of images corresponding to the media item via the display generation component, including displaying the plurality of images sequentially over time.

[0054] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for: when displaying a representation of a media item including a simulated depth-of-field effect via the display generation component: detecting input directed to the representation of the media item including the simulated depth-of-field effect via one or more input devices; and in response to detecting the input directed to the representation of the media item, displaying a plurality of images corresponding to the media item via the display generation component, including displaying the plurality of images sequentially over time.

[0055] According to some embodiments, a method is described. The method includes: performing the following operations at a computer system having one or more cameras, wherein the computer system communicates with a display generation component and one or more input devices: displaying a camera user interface via the display generation component, the camera user interface including one or more optional controls for managing zoom levels to capture media; detecting a first input, including movement, pointing to a corresponding optional control; and, in response to detecting the first input pointing to the corresponding optional control, navigating through a plurality of candidate zoom levels according to movement from a first zoom level to a second zoom level, including capturing one or more zoom levels while navigating through the plurality of candidate zoom levels; and: capturing a third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has been selected as the capture zoom level; and abandoning the capture of the third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has not yet been selected as the capture zoom level.

[0056] According to some embodiments, a non-transitory computer-readable storage medium is described. This non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component, one or more cameras, and one or more input devices. The one or more programs include instructions for: displaying a camera user interface via the display generation component, the camera user interface including one or more optional controls for managing zoom levels to capture media; detecting a first input, including movement, pointing to a corresponding optional control; and, in response to detecting the first input pointing to the corresponding optional control, navigating from a first zoom level to a second zoom level according to movement through a plurality of candidate zoom levels, including capturing one or more zoom levels while navigating through the plurality of candidate zoom levels; and: capturing a third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has been selected as the capture zoom level; and abandoning the capture of the third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has not yet been selected as the capture zoom level.

[0057] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component, one or more cameras, and one or more input devices. The one or more programs include instructions for: displaying a camera user interface via the display generation component, the camera user interface including one or more optional controls for managing zoom levels to capture media; detecting a first input, including movement, pointing to a corresponding optional control; and, in response to detecting the first input pointing to the corresponding optional control, navigating from a first zoom level to a second zoom level according to movement through a plurality of candidate zoom levels, including capturing one or more zoom levels while navigating through the plurality of candidate zoom levels; and: capturing a third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has been selected as the capture zoom level; and abandoning the capture of the third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has not yet been selected as the capture zoom level.

[0058] According to some embodiments, a computer system configured to communicate with a display generation component, one or more cameras, and one or more input devices is described. The computer system includes one or more processors and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: displaying a camera user interface via the display generation component, the camera user interface including one or more optional controls for managing zoom levels to capture media; detecting a first input, including movement, pointing to a corresponding optional control; and, in response to detecting the first input pointing to the corresponding optional control, navigating from a first zoom level to a second zoom level according to movement through a plurality of candidate zoom levels, including capturing one or more zoom levels while navigating through the plurality of candidate zoom levels; and: capturing a third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has been selected as the capture zoom level; and abandoning the capture of the third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has not yet been selected as the capture zoom level.

[0059] According to some embodiments, a computer system configured to communicate with a display generation component, one or more cameras, and one or more input devices is described. The computer system includes: components for displaying a camera user interface via the display generation component, the camera user interface including one or more optional controls for managing zoom levels to capture media; components for detecting a first input, including movement, pointing to a corresponding optional control; and components for: in response to detecting the first input pointing to the corresponding optional control, navigating through a plurality of candidate zoom levels according to movement from a first zoom level to a second zoom level, including capturing one or more zoom levels while navigating through the plurality of candidate zoom levels; and: capturing a third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has been selected as the capture zoom level; and abandoning the capture of the third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has not yet been selected as the capture zoom level.

[0060] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to execute by one or more processors of a computer system in communication with a display generation component, one or more cameras, and one or more input devices. The one or more programs include instructions for: displaying a camera user interface via the display generation component, the camera user interface including one or more optional controls for managing zoom levels to capture media; detecting a first input, including movement, pointing to a corresponding optional control; and, in response to detecting the first input pointing to the corresponding optional control, navigating from a first zoom level to a second zoom level according to movement through a plurality of candidate zoom levels, including capturing one or more zoom levels while navigating through the plurality of candidate zoom levels; and: capturing a third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has been selected as the capture zoom level; and abandoning the capture of the third zoom level while navigating through the plurality of candidate zoom levels based on determining that a third zoom level between the first and second zoom levels has not yet been selected as the capture zoom level.

[0061] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0062] Therefore, faster and more efficient methods and interfaces are provided for controlling media capture settings on such devices, thereby improving their effectiveness, efficiency, and user satisfaction. These methods and interfaces can complement or replace other methods used to control media capture settings. Attached Figure Description

[0063] To better understand the various described embodiments, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals in all the drawings indicate corresponding parts.

[0064] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some implementation schemes.

[0065] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0066] Figure 2 Examples of portable multi-functional devices with touchscreens according to some implementation schemes are shown.

[0067] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0068] Figure 4A An exemplary user interface for a menu applied to a portable multi-functional device, according to some implementation schemes, is illustrated.

[0069] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is illustrated according to some embodiments.

[0070] Figure 5A Examples of personal electronic devices according to some implementation schemes are shown.

[0071] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0072] Figures 5C to 5D Exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor according to some embodiments are illustrated.

[0073] Figures 5E to 5H Exemplary components and user interfaces of a personal electronic device according to some implementation schemes are illustrated.

[0074] Figures 6A to 6U Example techniques and user interfaces for controlling the capture of media with associated depth information, according to some implementation schemes, are illustrated.

[0075] Figure 7 It is a flowchart of a method for controlling the capture of media with associated depth information, according to some implementation schemes.

[0076] Figures 8A to 8T Example technologies and user interfaces for controlling media capture settings are illustrated according to some implementation schemes.

[0077] Figure 9 This is a flowchart of a method for controlling media capture settings according to some implementation schemes.

[0078] Figures 10A to 10I Example techniques and user interfaces for controlling scaling settings for media capture, according to some implementation schemes, are illustrated.

[0079] Figure 11 This is a flowchart of a method for controlling scaling settings for media capture, according to some implementation schemes.

[0080] Figures 12A to 12T Examples of techniques and user interfaces for capturing and viewing media using internal and external media storage devices, according to some implementation schemes, are illustrated.

[0081] Figure 13 This is a flowchart of a method for capturing and viewing media using internal and external media storage devices, according to some implementation schemes.

[0082] Figures 14A to 14Q Example techniques and user interfaces for conditionally displaying level indicators based on media capture content are illustrated according to some implementation schemes.

[0083] Figure 15 This is a flowchart of a method for conditionally displaying level indicators based on media capture content, according to some implementation schemes.

[0084] Figures 16A to 16M Examples of techniques and user interfaces for capturing, viewing, and / or editing media items are illustrated according to some implementation schemes.

[0085] Figure 17 It is a flowchart of a method for capturing, viewing and / or editing media items according to some implementation schemes.

[0086] Figures 18A to 18T Example technologies and user interfaces for displaying, editing, and / or controlling settings for media items are illustrated according to some implementation schemes.

[0087] Figure 19 It is a flowchart of a method for displaying, editing, and / or controlling settings for media items, according to some implementation schemes.

[0088] Figures 20A to 20TExample technologies and user interfaces for displaying media items are illustrated according to some implementation schemes.

[0089] Figure 21 This is a flowchart of a method for displaying media items according to some implementation schemes.

[0090] Figures 22A to 22AL Example techniques and user interfaces for controlling camera zoom levels are illustrated according to some implementation schemes.

[0091] Figure 23 This is a flowchart of a method for controlling the zoom level of a camera, according to some implementation schemes. Detailed Implementation

[0092] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0093] Electronic devices are needed that provide efficient methods and interfaces for controlling media capture settings. For example, automatically providing a depth capture indicator when depth capture criteria are met; providing quick access to depth effect controls when relevant; and hiding these controls when they are unlikely to be used. For example, providing control modes that switch between a limited set of states in response to one type of input and provide access to additional states in response to another type of input, providing quick and intuitive access to media controls without cluttering the user interface. For example, displaying a secondary, zoomed-out camera preview when the main camera preview is magnified beyond a certain threshold helps users create and capture media content from a broader environmental perspective. For example, displaying different media based on whether external storage criteria are met provides quick access to relevant media. For example, conditionally displaying level indicators based on the presence of specific content, automatically providing capture guidance when relevant, and avoiding cluttering the user interface when display conditions are not met. Such technologies can reduce the cognitive burden on users when capturing and managing media, thereby increasing productivity. Such technologies reduce the processing power of devices used to display the user interface, receive input through the user interface, and respond to user input. In addition, such technologies can reduce processor power and battery power that would otherwise be wasted on redundant user input.

[0094] under Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5H A description of an exemplary device for performing management event notifications is provided. Figures 6A to 6U An exemplary user interface is shown for controlling the capture of media with associated depth information. Figure 7This is a flowchart illustrating a method for controlling the capture of media with associated depth information according to some implementation schemes. Figures 6A to 6U The user interface in the document is used to illustrate the processes described below, including Figure 7 The process in. Figures 8A to 8T An exemplary user interface for controlling media capture settings is shown. Figure 9 This is a flowchart illustrating a method for setting up control media capture according to some implementation schemes. Figures 8A to 8T The user interface in the document is used to illustrate the processes described below, including Figure 9 The process in. Figures 10A to 10I An exemplary user interface for controlling scaling settings in media capture is shown. Figure 11 This is a flowchart illustrating a method for controlling the capture of media with associated depth information according to some implementation schemes. Figures 10A to 10I The user interface in the document is used to illustrate the processes described below, including Figure 11 The process in. Figures 12A to 12T An exemplary user interface is shown for capturing and viewing media using internal and external media storage devices. Figure 13 This is a flowchart illustrating methods for capturing and viewing media using internal and external media storage devices according to some implementation schemes. Figures 12A to 12T The user interface in the document is used to illustrate the processes described below, including Figure 13 The process in. Figures 14A to 14Q An exemplary user interface is shown for conditionally displaying level indicators based on media capture content. Figure 15 This is a flowchart illustrating a method for conditionally displaying level indicators based on media capture content, according to some implementation schemes. Figures 14A to 14Q The user interface in the document is used to illustrate the processes described below, including Figure 15 The process in. Figures 16A to 16M An exemplary user interface for capturing, viewing, and / or editing media items is shown. Figure 17 This is a flowchart illustrating methods for capturing, viewing, and / or editing media items according to some implementation schemes. Figures 16A to 16M The user interface in the document is used to illustrate the processes described below, including Figure 17 The process in. Figures 18A to 18T An exemplary user interface for displaying, editing, and / or controlling settings for media items is shown. Figure 19 This is a flowchart illustrating methods for displaying, editing, and / or controlling media item settings according to some implementation schemes. Figures 18A to 18T The user interface in the document is used to illustrate the processes described below, including Figure 19 The process in. Figures 20A to 20T An exemplary user interface for displaying media items is shown. Figure 21This is a flowchart illustrating a method for displaying media items according to some implementation schemes. Figures 20A to 20T The user interface in the document is used to illustrate the processes described below, including Figure 21 The process in. Figures 22A to 22AK An exemplary user interface for controlling the zoom level of a camera is shown. Figure 23 This is a flowchart illustrating a method for controlling the zoom level of a camera according to some implementation schemes. Figures 22A to 22AK The user interface in the document is used to illustrate the processes described below, including Figure 23 The process in.

[0095] The processes described below enhance device operability and make the user-device interface more efficient through various technologies (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), including providing improved visual feedback to users, reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional display controls, performing operations without requiring further user input when a set of conditions has been met, and / or additional technologies. These technologies also reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0096] Furthermore, in a method described herein where one or more steps depend on the satisfaction of one or more conditions, it should be understood that the described method can be repeated in multiple repetitions such that, during the repetitions, all conditions determining the steps in the method are satisfied in different repetitions of the method. For example, if the method requires performing a first step (if the conditions are satisfied) and a second step (if the conditions are not satisfied), those skilled in the art will know that the stated steps are repeated until both conditions are satisfied and not satisfied (in no particular order). Thus, a method described as having one or more steps depending on the satisfaction of one or more conditions can be rewritten as a method that repeats until each condition described in the method is satisfied. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing discretionary operations based on the satisfaction of the corresponding one or more conditions, and thus to determine whether possible conditions have been satisfied without explicitly repeating the steps of the method until all conditions determining the steps in the method are satisfied. Those skilled in the art will also understand that, similar to a method having discretionary steps, a system or computer-readable storage medium can repeat the steps of the method multiple times as needed to ensure that all discretionary steps have been performed.

[0097] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. In some embodiments, these terms are used to distinguish one element from another. For example, a first touch may be referred to as a second touch, and similarly, a second touch may be referred to as a first touch, without departing from the scope of the various described embodiments. In some embodiments, a first touch and a second touch are two separate references to the same touch. In some embodiments, both a first touch and a second touch are touches, but they are not the same touch.

[0098] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context expressly indicates otherwise. It will also be understood that, as used herein, the term “and / or” refers to and covers any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0099] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in response to," or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to determining..." or "in response to detecting [the stated condition or event]."

[0100] This document describes implementations of electronic devices, user interfaces for such devices, and associated processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functionalities such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, the iPhone from Apple Inc., Cupertino, California. ® Devices, iPod Touch ® Devices and iPads ®Device. Optionally, other portable electronic devices may be used, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the electronic device is a computer system that communicates with a display generating component (e.g., via wireless or wired communication). The display generating component is configured to provide visual output, such as display via a CRT display, via an LED display, or via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. As used herein, “display” content includes displaying content (e.g., video data rendered or decoded by display controller 156) by sending data (e.g., image data or video data) to an integrated or external display generating component via a wired or wireless connection to visually generate content.

[0101] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0102] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0103] Various applications running on this device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or within the respective applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0104] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1AThis is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic outputs on device 100 (e.g., generating haptic outputs on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0105] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values, including at least four different values ​​and, in particular, hundreds of different values ​​(e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or change of the contact area detected on the touch-sensitive surface, the capacitance and / or change of the touch-sensitive surface adjacent to the contact, and / or the resistance and / or change of the touch-sensitive surface adjacent to the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of the contact as an attribute of user input allows users to access additional device functionality that would otherwise be inaccessible to the user on a smaller device with limited physical space, which is used (e.g., on a touch-sensitive display) to display an indication and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0106] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with the touch-sensitive surface, which has been physically pressed (e.g., displaced) by the user's movement, does not move. As another example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or perceived by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of the user (e.g., "release click", "press click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception described by a typical (or common) user.

[0107] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0108] Memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0109] Peripheral interface 118 can be used to couple input and output peripherals of the device to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs (such as computer programs (e.g., including instructions)) and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0110] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, subscriber identity module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks (such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs))) and other devices. RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as via short-range communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)) and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0111] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and sends the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and sends the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or sent to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both outputs (e.g., a single-ear or dual-ear headset) and inputs (e.g., a microphone).

[0112] I / O subsystem 106 couples input / output peripherals (such as touchscreen 112 and other input control devices 116) on device 100 to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / transmit electrical signals to the other input control device 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dials, slide switches, joysticks, click dials, etc. In some embodiments, input controller 160 is optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing device such as mouse. One or more buttons (e.g., ... Figure 2 Optionally, 208) includes an increase / decrease button for volume control of speaker 111 and / or microphone 113. The one or more buttons optionally include a push-button (e.g., Figure 2(Ref. 206 in the original text). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication, via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking user gestures (e.g., hand gestures and / or air gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system. In some implementations, air gestures are gestures detected without the user touching an input element that is part of the device (or independently of an input element that is part of the device) and based on the detected movement of a part of the user's body through the air (including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of the user's fingers relative to another of the user's fingers or a part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tapping gesture that includes the hand moving a predetermined amount and / or speed in a predetermined pose, or a shaking gesture that includes a predetermined speed or amount of rotation of a part of the user's body)).

[0113] A quick press of a push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of a push button (e.g., 206) optionally powers the device 100 on or off. The functionality of one or more of these buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0114] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0115] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0116] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed thereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as in the iPhone from Apple Inc. (Cupertino, California). ® and iPod Touch ® The technology used.

[0117] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles a multi-touch-sensitive touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0118] The touch-sensitive display in some embodiments of the touchscreen 112 is described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, “Mode-Based Graphical User Interfaces For Touch Sensitive Input”. (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, “Virtual Input Device Placement On A Touch Screen User Interface”; (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, “Operation Of A Computer With A Touch Screen Interface”; (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, “Activating Virtual Keys Of ATouch-Screen Virtual Keyboard”; and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, “Multi-Functional Hand-Held Device”. The full text of all these applications is incorporated herein by reference.

[0119] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally use any suitable object or accessory such as a stylus, finger, etc., to interact with touchscreen 112. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positioning or commands for performing the user-desired actions.

[0120] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. Optionally, the touchpad is a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0121] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0122] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 coupled to an optical sensor controller 158 in I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also referred to as a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of device 100, facing away from a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the positioning of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensors within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0123] The device 100 optionally also includes one or more depth camera sensors 175. Figure 1A A depth camera sensor is shown coupled to a depth camera controller 169 in I / O subsystem 106. Depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with imaging module 143 (also referred to as camera module), depth camera sensor 175 is optionally used to determine depth maps of different portions of an image captured by imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information are optionally acquired for video conferencing while a user views other video conferencing participants on a touchscreen display, and selfies with depth map data are captured. In some embodiments, depth camera sensor 175 is located at the rear of the device, or both the rear and front of device 100. In some embodiments, the positioning of depth camera sensor 175 can be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.

[0124] In some implementations, the depth map (e.g., a depth map image) contains information (e.g., values) relating to the distance of objects in the scene from the viewpoint (e.g., a camera, optical sensor, depth camera sensor). In one implementation of the depth map, each depth pixel defines the location of its corresponding two-dimensional pixel on the Z-axis of the viewpoint. In some implementations, the depth map is composed of pixels, where each pixel is defined by a value (e.g., 0 to 255). For example, a "0" value represents the pixel furthest from the viewpoint (e.g., a camera, optical sensor, depth camera sensor) in the "3D" scene, and a "255" value represents the pixel closest to the viewpoint in the "3D" scene. In other implementations, the depth map represents the distance between objects in the scene and the plane of the viewpoint. In some implementations, the depth map includes information about the relative depth of various features of the object of interest within the field of view of the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of a user's face). In some implementations, the depth map includes information that enables the device to determine the contour of the object of interest along the z-direction.

[0125] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor is shown coupled to a strength sensor controller 159 in I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0126] The device 100 optionally also includes one or more proximity sensors 166. Figure 1AA proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally be configured as described in the following U.S. patent applications: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call), the proximity sensor is turned off and the touchscreen 112 is disabled.

[0127] The device 100 may optionally also include one or more tactile output generators 167. Figure 1A A haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components for converting electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0128] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally be configured as described in the following U.S. Patent Publications: 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and 20060017692, entitled "Methods And Apparatuses For Operating A Portable DeviceBased On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.

[0129] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the figure. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, indicating what applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from various sensors and input control devices 116 of the device; and position information relating to the device's position and / or orientation.

[0130] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0131] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, Wireless LAN, etc.). In some embodiments, the external port is for use with an iPod. ® (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.

[0132] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (amplitude), velocity (amplitude and direction), and / or acceleration (change in amplitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0133] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by the user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0134] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes: detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes: detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0135] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0136] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from applications, etc., to specify the graphics to be displayed, and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data to output to the display controller 156.

[0137] The haptic feedback module 133 includes various software components for generating instructions which are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0138] The text input module 134, optionally a component of the graphics module 132, provides a soft keyboard for entering text in various applications, such as contacts 137, email 140, IM 141, browser 147, and any other application that requires text input.

[0139] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to telephone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0140] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: • Contacts module 137 (sometimes called address book or contact list); • Telephone module 138; • Video conferencing module 139; • Email client module 140; • Instant Messaging (IM) module 141; • Fitness support module 142; • Camera module 143 for still images and / or video images; • Image management module 144; • Video player module; • Music player module; • Browser module 147; • Calendar module 148; • Widget module 149, which optionally includes one or more of the following: weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5 and other widgets obtained by the user, as well as user-created widgets 149-6. • Widget creator module 150 for creating user-created widgets 149-6; • Search module 151; • Video and music player module 152, which combines the video player module and the music player module; • Memo module 153; • Map module 154; and / or • Online video module 155.

[0141] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0142] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0143] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify input telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As noted above, wireless communication optionally employs any of a variety of communication standards, protocols, and technologies.

[0144] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0145] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0146] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for performing the following operations: entering a character sequence corresponding to an instant message, modifying previously entered characters, sending a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing received instant messages. In some embodiments, the sent and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages delivered using SMS or MMS) and internet-based messages (e.g., messages delivered using XMPP, SIMPLE, or IMPS).

[0147] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for performing the following operations: creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0148] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0149] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for performing operations such as arranging, modifying (e.g., editing) or otherwise manipulating, marking, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0150] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for performing the following operations: browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.

[0151] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0152] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally a micro-application downloaded and used by a user (e.g., weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a user-created micro-application (e.g., user-created widget 149-6). In some embodiments, the widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! widgets).

[0153] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 is optionally used by the user to create widgets (e.g., turning user-specified portions of a webpage into widgets).

[0154] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for performing the following operations: searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0155] Combining touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0156] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the memo module 153 includes executable instructions for creating and managing memos, to-do items, etc., according to user instructions.

[0157] Combining RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135 and browser module 147, map module 154 is optionally used to receive, display, modify and store maps and data associated with the maps (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0158] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following operations: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Additional descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0159] Each of the modules and applications described above corresponds to an executable set of instructions for performing one or more functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 optionally stores a subset of the above-described modules and data structures. Additionally, memory 102 optionally stores additional modules and data structures not described above.

[0160] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons and dial pads, etc.) on device 100 is optionally reduced.

[0161] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, main desktop menu, or root menu. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0162] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any of the aforementioned applications 137 to 151, 155, 380 to 390).

[0163] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0164] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0165] Event monitor 171 receives event information from peripheral interface 118. The event information includes information about sub-events, such as a user touch on touch-sensitive display 112 as part of a multi-touch gesture. Peripheral interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0166] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 sends event information. In other implementations, peripheral device interface 118 sends event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0167] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0168] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0169] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined, at least in part, based on the hit view of the initial touch that initiates the touch-based gesture.

[0170] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0171] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0172] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0173] In some implementations, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another implementation, event classifier 170 is a separate module or part of another module (such as contact / motion module 130) stored in memory 102.

[0174] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0175] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0176] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device orientation).

[0177] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in events (e.g., 187-1 and / or 187-2) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch-sensitive display 112, and lifting the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0178] In some implementations, event definition 186 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0179] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0180] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0181] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0182] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from delivering (and deferred delivering) the sub-events to the corresponding hit view. In some implementations, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag acquires the flag and performs a predefined process.

[0183] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined process.

[0184] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the positioning of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and transmits that display information to graphics module 132 for display on a touch-sensitive display.

[0185] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0186] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0187] Figure 2A portable multifunction device 100 with a touchscreen 112 is illustrated according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0188] Device 100 optionally also includes one or more physical buttons, such as a "main desktop" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0189] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headphone jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to: power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0190] Figure 3This is a block diagram of an exemplary multi-functional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the reference above). Figure 1A The described tactile output generator 167), sensor 359 (e.g., optical sensor, accelerometer, proximity sensor, touch sensor and / or contact intensity sensor (similar to the one described above)) Figure 1A The described contact strength sensor 165). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores information related to portable multifunction device 100. Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0191] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each module described above corresponds to a set of instructions for performing the functions described above. The modules or computer programs described above (e.g., instruction sets or including instructions) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules, and therefore various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures described above. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0192] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.

[0193] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is illustrated. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes elements or a subset or superset thereof: • One or more signal strength indicators 402 for one or more wireless communications (e.g., cellular signals and Wi-Fi signals); • Time 404; • Bluetooth indicator 405; • Battery status indicator 406; • Tray 408 with icons for commonly used applications, such as: The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemails; The email client module 140 has an icon 418 labeled "Mail" which optionally includes an indicator 410 for the number of unread emails; The browser module 147 has an icon 420 labeled "Browser"; and o The video and music player module 152 (also known as the iPod (Apple Inc. trademark) module 152) is labeled with an icon 422 marked "iPod"; and • Icons of other applications, such as: The icon 424 of oIM module 141 marked as "message"; The calendar module 148 has an icon 426 labeled "Calendar"; The image management module 144 has an icon 428 labeled "Photo". The icon 430 of the camera module 143 is labeled "camera"; o The icon 432 of the online video module 155, which is labeled "Online Video"; The icon 434 labeled "Stock Market" in the stock market widget 149-2; The icon 436 labeled "map" in the map module 154; The weather widget 149-1 has icon 438 labeled "weather"; The alarm clock widget 149-4 has an icon 440 labeled "clock"; The icon 442 of the fitness support module 142 is labeled "fitness support"; The icon 444 labeled "Memo" in the memo module 153; and o The icon 446, labeled "Settings," is used to set up an application or module that provides access to settings for the device 100 and its various applications 136.

[0194] It should be pointed out that, Figure 4A The illustrated icon labels are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0195] Figure 4B An example is illustrated having a touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)). Figure 3 A tablet device or touchpad 355) device (e.g., Figure 3 An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.

[0196] While some examples of input on a touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations the device detects input on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450) corresponding to the main axis on the display (e.g., Figure 4BThe main shaft of 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to a specific location on the display (e.g., in...). Figure 4B In the diagram, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, when the touch-sensitive surface (e.g., ...) Figure 4B 451 in the middle) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on that touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0197] Additionally, while the examples below are given primarily with reference to finger input (e.g., finger touch, single-finger tap gesture, finger swipe gesture), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the touch). As another example, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0198] Figure 5A An exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include devices 100 and 300 (e.g., Figures 1A to 4B The device 500 may include some or all of the features described herein. In some embodiments, the device 500 has a touch-sensitive display 504, referred to below as a touchscreen 504. Alternatively, or in addition to the touchscreen 504, the device 500 may also have a display and a touch-sensitive surface. Similar to the cases of devices 100 and 300, in some embodiments, the touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). The one or more intensity sensors of the touchscreen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of the device 500 may respond to touches based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface operations on the device 500.

[0199] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying UserInterface Objects Corresponding to an Application,” published as WIPO Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between TouchInput to Display Output Relationships,” published as WIPO Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0200] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508, if included, can be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0201] Figure 5B An exemplary personal electronic device 500 is depicted. In some embodiments, device 500 may include information about... Figure 1A , Figure 1B and Figure 3Some or all of the components described. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive component 522 and optionally have intensity sensor 524 (e.g., contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 may optionally be a rotatable input device or a pressable and rotatable input device. In some examples, input mechanism 508 may optionally be a button.

[0202] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.

[0203] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause the computer processors to perform, for example, the techniques described below, including methods 700, 900, 1100, 1300, 1500, 1700, 1900, 2100, and 2300. Figure 7 , Figure 9 , Figure 11 , Figure 13 , Figure 15 , Figure 17 , Figure 19 , Figure 21 and Figure 23 A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.

[0204] As used herein, the term "power indication" refers optionally to the power indication in devices 100, 300, and / or 500 ( Figure 1A , Figure 3 and Figures 5A to 5B A user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.

[0205] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of direct interaction with user interface elements on a touchscreen display) that enable direct interaction with user interface elements on the touchscreen display. Figure 1A The touch-sensitive display system 112 or Figure 4A In some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user's expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0206] As used in the specification and claims, the term "characteristic strength" of a contact refers to a characteristic of the contact based on one or more strengths of the contact. In some embodiments, the characteristic strength is based on multiple strength samples. The characteristic strength is optionally based on a predefined number of strength samples or a set of strength samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to lift off, before or after contact begins to move, before contact ends, before or after contact strength increases and / or before or after contact strength decreases). The characteristic strength of the contact is optionally based on one or more of the following: the maximum value of the contact strength, the mean value of the contact strength, the average value of the contact strength, the value at the top 10% of the contact strength, the half maximum value of the contact strength, or the 90% maximum value of the contact strength, etc. In some embodiments, the duration of the contact is used when determining the characteristic strength (e.g., when the characteristic strength is the average value of the contact strength over time). In some implementations, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether a user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a characteristic intensity not exceeding the first threshold results in a first action, contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a characteristic intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the characteristic intensity and one or more thresholds is used to determine whether one or more actions should be performed (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0207] Figure 5C An example is shown using multiple intensity sensors 524A-524D to detect multiple contacts 552A-552E on a touch-sensitive display 504. Figure 5C Additionally, an intensity graph is included, showing the current intensity measurements of intensity sensors 524A-524D relative to intensity units. In this example, the intensity measurements of intensity sensors 524A and 524D are both 9 intensity units, and the intensity measurements of intensity sensors 524B and 524C are both 7 intensity units. In some embodiments, the cumulative intensity is the sum of the intensity measurements of the multiple intensity sensors 524A-524D, which is 32 intensity units in this example. In some embodiments, each contact is assigned a corresponding intensity, i.e., a portion of the cumulative intensity. Figure 5DAn example is given of assigning cumulative intensity to contacts 552A-552E based on their distance from the center of force 554. In this example, each of contacts 552A, 552B, and 552E is assigned a contact intensity of 8 intensity units of the cumulative intensity, and each of contacts 552C and 552D is assigned a contact intensity of 4 intensity units of the cumulative intensity. More generally, in some specific implementations, each contact j is assigned a corresponding intensity Ij according to a predefined mathematical function Ij = A·(Dj / ΣDi), which is a portion of the cumulative intensity A, where Dj is the distance of the corresponding contact j from the center of force, and ΣDi is the sum of the distances of all corresponding contacts (e.g., i=1 to the last) from the center of force. Reference can be performed using electronic devices similar to or equivalent to devices 100, 300, or 500. Figures 5C to 5D The described operation. In some embodiments, the characteristic intensity of the contact is based on one or more intensities of the contact. In some embodiments, an intensity sensor is used to determine the intensity of a single characteristic (e.g., the single characteristic intensity of a single contact). It should be noted that the intensity map is not part of the displayed user interface, but is included within... Figures 5C to 5D This is intended to assist readers.

[0208] In some implementations, a portion of the gesture is identified for determining the characteristic intensity. For example, the touch-sensitive surface optionally receives a series of swipe contacts that transition from a starting position to an ending position, where the contact intensity increases. In this example, the characteristic intensity of the contact at the ending position is optionally based only on a portion of the series of swipe contacts, rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some implementations, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: unweighted moving average smoothing algorithm, triangular smoothing algorithm, median filtering smoothing algorithm, and / or exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the intensity of the swipe contact to achieve the purpose of determining the characteristic intensity.

[0209] Optionally, the contact intensity on a touch-sensitive surface is characterized relative to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device performs an operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device performs an operation different from the operation typically associated with clicking a button on a physical mouse or touchpad. In some embodiments, when a contact with a characteristic intensity lower than the light press intensity threshold (e.g., and higher than the nominal contact detection intensity threshold, where contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector based on the movement of the contact on the touch-sensitive surface without performing the operation associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent across different groups of user interface figures.

[0210] An increase in contact intensity from below a light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in contact intensity from below a deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in contact intensity from below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in contact intensity from above a contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a contact being lifted off the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0211] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the intensity of the contact (or multiple contacts) increases to above a press input intensity threshold. In some embodiments, the corresponding operation is performed in response to detecting that the intensity of the corresponding contact increases to above a press input intensity threshold (e.g., a "downward stroke" of the corresponding press input). In some embodiments, the press input includes the intensity of the corresponding contact increasing to above a press input intensity threshold and the intensity of the contact subsequently decreasing to below the press input intensity threshold, and the corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the press input threshold (e.g., an "upward stroke" of the corresponding press input).

[0212] Figures 5E to 5HAn example of gesture detection is given, where the gesture includes the intensity of contact 562 ranging from less than... Figure 5E The light press intensity threshold (e.g., "IT") L The intensity of ) increases to higher than Figure 5H The deep press intensity threshold (e.g., "IT") D The intensity of the press input corresponds to the strength of the press. On the user interface 570, which includes application icons 572A-572D displayed in the predefined area 574, when a cursor 576 is displayed above the application icon 572B corresponding to application 2, a gesture performed using contact 562 is detected on the touch-sensitive surface 560. In some embodiments, the gesture is detected on the touch-sensitive display 504. The intensity sensor detects the intensity of the contact on the touch-sensitive surface 560. The device determines the intensity of the contact 562 within a deep press intensity threshold (e.g., "IT"). D The intensity reaches its peak at 562. Contact 562 is maintained on the touch-sensitive surface 560. In response to a detected gesture, and based on the intensity increasing to a deep press intensity threshold during the gesture (e.g., "IT"), the intensity is increased to a peak value. D The above contact 562 displays the scale representation 578A-578C (e.g., thumbnails) of the document recently opened for application 2, such as... Figures 5F to 5H As shown. In some embodiments, the intensity is the characteristic intensity of the contact compared to one or more intensity thresholds. It should be noted that the intensity map for contact 562 is not part of the displayed user interface, but is included in... Figures 5E to 5H This is intended to assist readers.

[0213] In some implementations, the display of icons 578A-578C includes animation. For example, icon 578A might initially appear near application icon 572B, such as... Figure 5F As shown, as the animation progresses, icon 578A moves upwards, and icon 578B appears near application icon 572B, as shown. Figure 5G As shown in the diagram. Then, representation 578A moves upward, 578B moves upward toward representation 578A, and representation 578C is displayed near application icon 572B, as shown. Figure 5H As shown in the diagram. This indicates that 578A-578C form an array above icon 572B. In some implementations, the animation progresses according to the intensity of contact 562, such as... Figures 5F to 5G As shown, where 578A-578C appear and increase with the intensity of contact 562 towards the deeper pressing strength threshold (e.g., "IT"). D The animation progresses upwards as it increases. In some implementations, the intensity of the animation progression is based on the strength of the contact characteristics. Reference can be performed using an electronic device similar to or equivalent to device 100, 300, or 500. Figures 5E to 5H The described operation.

[0214] In some implementations, the device employs intensity hysteresis to avoid unintended inputs sometimes referred to as "jitter," wherein the device defines or selects a hysteresis intensity threshold that has a predefined relationship with a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the press input intensity threshold). Therefore, in some implementations, a press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of that contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and an operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., the "upstroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the intensity of the contact increases from an intensity equal to or below a hysteresis intensity threshold to an intensity equal to or above a press input intensity threshold and optionally the intensity of the contact subsequently decreases to an intensity equal to or below the hysteresis intensity, and corresponding operations are performed in response to the detection of a press input (e.g., depending on the environment, the intensity of the contact increases or decreases).

[0215] For ease of explanation, optionally, the description of an operation triggered in response to a press input associated with a press input strength threshold or in response to a gesture including a press input is provided in response to detecting any of the following conditions: the contact strength increases to above the press input strength threshold, the contact strength increases from below a hysteresis strength threshold to above the press input strength threshold, the contact strength decreases to below the press input strength threshold, and / or the contact strength decreases to below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in the example where the operation is described as being performed in response to detecting a decrease in contact strength to below the press input strength threshold, the operation is optionally performed in response to detecting a decrease in contact strength to below a hysteresis strength threshold corresponding to and less than the press input strength threshold.

[0216] As used herein, "installed application" refers to a software application that has been downloaded to an electronic device (e.g., device 100, 300, and / or 500) and is ready to be launched on the device (e.g., become open). In some implementations, the downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the computer system's operating system.

[0217] As used herein, the terms "open application" or "running application" refer to a software application that maintains state information (e.g., as part of device / global internal state 157 and / or application internal state 192). An open or running application is optionally any of the following types of applications: • The active application, which is currently displayed on the screen of the device that is using the application; • Background applications (or background processes) that are not currently displayed but whose one or more processes are being handled by one or more processors; and • Apps that are paused or hibernating, which are not running but have state information stored in memory (both volatile and non-volatile) that can be used to resume the execution of the app.

[0218] As used herein, the term "closed application" refers to a software application that does not retain state information (e.g., the state information of a closed application is not stored in the device's memory). Therefore, closing an application includes: stopping and / or removing the application's process and removing the application's state information from the device's memory. Generally, opening a second application while the first application is running does not close the first application. When the second application is displayed and the first application stops displaying, the first application becomes a background application.

[0219] Now let’s turn our attention to the implementation of user interfaces (“UIs”) on electronic devices, such as portable multifunction devices 100, 300 or 500, and the associated processes.

[0220] Figures 6A to 6U Exemplary user interfaces for controlling the capture of media with associated depth information, according to some embodiments, are illustrated. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 7 The process in.

[0221] Figures 6A to 6B Examples are shown from the back (e.g., Figure 6A ) and from the front (e.g., Figure 6B The computer system 600 (e.g., a mobile phone device) is used for observation. The computer system 600 includes a display 608 and multiple cameras, including a first camera 604A visible from its front side (e.g., a mobile phone device). Figure 6B As shown), the second camera 604B, the third camera 604C, and the fourth camera 604D (as shown) are visible on the rear side of the device. Figure 6A(As shown). In some embodiments, multiple cameras have (e.g., in a fixed and / or variable / configurable manner) different optical specifications, such as different focal lengths (e.g., 24mm, 35mm, and / or 55mm), different aperture sizes (e.g., f / 1.8, f / 2.4, and / or f / 8), and / or different fields of view (e.g., 46°, 84°, and / or 120°). Computer system 600 includes multiple input devices, including hardware buttons 606 and touch-sensitive surfaces of display 608. In some embodiments, computer system 600 includes one or more sensors, such as light sensors and / or depth sensors. In some embodiments, computer system 600 includes one or more hardware input ports, such as Mini USB, Micro USB, and / or USB-C ports. In some embodiments, the methods of using computer system 600 described herein are implemented using (e.g., in conjunction with computer system 600) one or more user devices (e.g., mobile phones, tablet computers, laptop computers, and / or wearable electronic devices (e.g., smartwatches)), remote devices (e.g., servers and / or networked devices), and / or peripheral devices (e.g., external storage drives, microphones, speakers, and / or hardware input devices). In some embodiments, computer system 600 includes one or more features of device 100, 300, or 500 (e.g., the first camera 604A is an optical sensor 164).

[0222] exist Figure 6B At this point, computer system 600 displays camera user interface 610 (e.g., a user interface for media capture, for example, using a camera application) via display 608. Camera user interface 610 includes camera preview 612, which indicates the portion of the field of view of at least one of the first camera 604A, second camera 604B, third camera 604C, and fourth camera 604D that will currently be included (e.g., captured) in media capture. Figure 6A As illustrated, the current field of view of the camera includes a portion of the physical environment captured in the camera preview 612, including the dog and cat in the mid-ground and a portion of the fence and sky in the background. In some embodiments, the computer system 600 detects additional information about the physical environment. For example, using one or more depth sensors and / or by comparing the fields of view of the first camera 604A, the second camera 604B, and / or the third camera 604C, the computer system 600 can obtain depth information about the physical environment, such as the distance of the dog and cat from the computer system 600.

[0223] The camera user interface 610 also includes a flash power indicator 614, a media format power indicator 616, a multi-frame photo power indicator 618, a zoom power indicator 620, a capture mode menu 622, a shutter power indicator 624, and a captured media icon 626. As indicated by the zoom power indicator 620, the zoom level is set to 0.5x magnification (e.g., ultra-wide zoom setting), and as indicated by the capture mode menu 622, the current capture mode is photo capture mode.

[0224] exist Figure 6B At this point, computer system 600 detects input 627 on zoom display 620 requesting an increase in zoom level (e.g., a tap input via a touch-sensitive surface of display 608). In response to input 627, in Figure 6C At this point, computer system 600 sets the zoom level to 2x magnification, thereby physically magnifying camera preview 612 so that the dog and cat appear in the foreground of camera preview 612. At the 2x magnification level, computer system 600 detects the cat and dog as subjects of media capture and indicates detection with subject indicators 628A and 628B, thereby framing the faces of the cat and dog (respectively). Because the current zoom level has increased to more than 1x magnification, and the cat and dog (e.g., subjects of media capture) are determined (e.g., based on acquired depth information) to be within a specific distance range (e.g., 2 to 8 feet from computer system 600), computer system 600 displays depth indicator 630. Depth indicator 630 indicates that computer system 600 is capturing depth information (e.g., using one or more sensors) for media capture.

[0225] exist Figure 6C At this point, the computer system detects input 632 (e.g., a tap input via a touch-sensitive surface of display 608) where a "Portrait" capture mode (e.g., a photo capture mode where content-aware simulated visual effects (e.g., depth effect) are applied to the captured media) is selected from capture mode menu 622. In response to input 632, in Figure 6D At this point, computer system 600 displays a camera preview 612 with simulated depth-of-field effects (e.g., simulated bokeh) based on the acquired depth information about the physical environment, so that the physical environment appears less sharp (e.g., more blurred) the farther away from the focal plane, and the sharper (e.g., more distinct) the closer to the focal plane. Figure 6D As illustrated, when a simulated depth-of-field effect is applied to camera preview 612, the dog appears sharp, while parts of the environment in front of and behind the dog (e.g., away from the focal plane) (including the cat and the fence) are blurred (e.g., in...). Figure 6D(In the center and elsewhere, crosshairs are used to represent the blur effect). Although the crosshairs representing the blur effect in these figures appear uniform, it should be understood that the parts of the environment further away from the focal plane are more blurred than the parts of the environment closer to the focal plane (e.g., in...). Figure 6D In this scenario, the fence will appear more blurred than the cat. In some implementations, the simulated depth-of-field effect mimics the depth of field of a specific aperture setting (e.g., an aperture value that defines the ratio between the focal length of a photographic lens and the aperture diameter, such as f / 1.4, f / 2.8, or f / 8), and the computer system 600 displays an aperture value indication 633, which can be selected to adjust the aperture setting in portrait capture mode (e.g., similar to an aperture setting adjustment described in more detail below). As the distance from the focal plane increases, the degree of focus reduction (e.g., the degree of blurriness) is proportionally greater for lower aperture values ​​(e.g., f / 1.4) than for higher aperture values ​​(e.g., f / 8) (e.g., the lower the aperture value, the more significant the effect of distance on blur). Additionally, in portrait capture mode, the computer system displays a lighting effects menu 634 (e.g., a menu for selecting simulated lighting effects to apply).

[0226] exist Figure 6D At this point, computer system 600 detects input 636 (e.g., a tap input via a touch-sensitive surface of display 608) indicating that a photo capture mode has been selected from capture mode menu 622. In response to input 636, in Figure 6E At this location, the computer system displays error 600 as follows: Figure 6B The described camera user interface 610 includes a camera preview 612 that displays a simulated depth-of-field effect without it being applied, and a depth indicator 630.

[0227] exist Figure 6E At this point, computer system 600 detects input 638 to the selection depth indicator 630. For example, input 638 includes short taps and / or clicks via the touch-sensitive surface of display 608, as described below. Figures 8A to 8T and Figure 9 Further details. In response to input 638, in Figure 6F At this point, while still in photo capture mode, computer system 600 displays a camera preview 612 with a simulated depth-of-field effect. For example... Figure 6FAs illustrated, when a simulated depth-of-field effect is applied to camera preview 612, the dog and cat appear sharp, while parts of the environment in front of and / or behind the dog and cat (e.g., further from the focal plane) are blurred. For example, the simulated depth-of-field effect applied to camera preview 612 simulates an aperture setting of f / 2.8. Additionally, in response to input 638, computer system 600 updates the appearance of depth indicator 630, for example, by changing the color, shadows, and / or visual emphasis of depth indicator 630 (e.g., in...). Figure 6F (The center and other areas are indicated by shadows) to indicate that the depth indicator 630 has been selected.

[0228] exist Figure 6F At this point, computer system 600 detects input 640 of selection depth indicator 630. For example, input 640 includes a long press input via the touch-sensitive surface of display 608, as described below. Figures 8A to 8T and Figure 9 Further details. In response to input 640, in Figure 6G At this location, computer system 600 displays aperture setting slider 644. In some embodiments, computer system 600 displays (e.g., animatedly) the aperture setting slider 644 expanding from depth indicator 630. Figure 6G As illustrated, when the aperture setting slider 644 is displayed, the computer system 600 stops displaying the zoom indication display 620. The aperture setting slider 644 includes an aperture setting indicator 646, which indicates that the current analog aperture setting is f / 2.8. In some embodiments, the computer system 600 stops displaying the aperture setting slider 644 after detecting a finger lift-off (lift-off) event of input 640 after a threshold time period without detecting input on the aperture setting slider 644. In some embodiments, the computer system 600 stops displaying the aperture setting slider 644 in response to detecting input at a location other than the position of the aperture setting slider 644.

[0229] exist Figure 6G At this point, computer system 600 detects input 642 (e.g., dragging and / or gesture input via a touch-sensitive surface of display 608) to adjust the aperture setting via aperture setting slider 644. In some embodiments, in response to dragging input 642 to the left, computer system 600 increases the aperture coefficient value of the aperture setting. For example, as... Figure 6HAs illustrated, computer system 600 increases the current simulated aperture setting to f / 3.5, as indicated by aperture setting indicator 646. A simulated depth-of-field effect is applied to camera preview 612 with the simulated aperture setting of f / 3.5. Computer system 600 displays camera preview 612 where the dog, cat, and some background are sharp, while the sky and fence remain blurred. In some implementations, in response to dragging input 642 to the right, computer system 600 decreases the aperture value of the aperture setting. For example, as... Figure 6I As illustrated, computer system 600 reduces the current simulated aperture setting to f / 1.4, as indicated by aperture setting indicator 646. Applying a simulated depth-of-field effect to camera preview 612 with a simulated aperture setting of f / 1.8, computer system 600 displays camera preview 612 where the dog is sharp, but the cat, fence, and other parts of the background and foreground are blurred. In some embodiments, aperture setting slider 644 represents multiple aperture coefficient values. For example, the scale markings of aperture setting slider 644 may represent discrete aperture coefficient values ​​over a range of aperture sizes (e.g., steps from f / 1.4 to f / 32). In some embodiments, computer system 600 determines the adjusted aperture setting (e.g., aperture coefficient value) based on the distance traveled by input 642 (e.g., net distance and / or normalized distance). For example, you can drag the aperture setting from f / 2.4 to f / 3.5 by dragging to the left at the distance corresponding to the distance between the f / 2.4 and f / 3.5 scale marks, and you can drag the aperture setting from f / 2.4 to f / 1.8 by dragging to the right at the distance corresponding to the distance between the f / 2.4 and f / 1.8 scale marks.

[0230] like Figure 6J As illustrated, once input 642 is released (e.g., lifted from the touch-sensitive surface of display 608), computer system 600 stops displaying aperture setting slider 644 and displays depth indicator 630 with aperture setting indicator 648, which indicates the adjusted aperture setting value (e.g., f / 1.4). In some embodiments, aperture setting indicator 648 is displayed within depth indicator 630. While depth indicator 630 remains selected, computer system 600 continues to display with aperture setting indicator 648, as shown in the figure. Figure 6I The camera preview 612 describes the simulated depth of field (e.g., an adjusted aperture value simulating f / 1.4).

[0231] exist Figure 6JAt this point, computer system 600 detects input 650 on shutter power indicator 624 (e.g., a tap input via a touch-sensitive surface of display 608; in some embodiments, input 650 may include a press of hardware button 606). In response to detecting input 650, computer system 600 initiates media capture using the currently selected capture settings, including a simulated depth-of-field effect using a simulated aperture setting of f / 1.4. Figure 6K As illustrated, after capturing media, computer system 600 updates captured media icon 626 to show a thumbnail of the captured media. The simulated depth-of-field effect applied to the captured media is visible in the thumbnail shown in captured media icon 626.

[0232] exist Figure 6K At this point, computer system 600 detects input 652 to the selection depth indicator 630 (e.g., a short tap input via the touch-sensitive surface of display 608, as described with respect to input 638). In response to detecting input 652, in Figure 6L At this point, computer system 600 stops displaying camera preview 612 with simulated depth-of-field effect. However, because the current zoom level remains above 1x magnification and the cat and dog are still within a certain distance range (e.g., as about...), Figure 6C (As described), so the computer system 600 continues to display the depth indicator 630, indicating that the computer system 600 is still capturing depth information. Additionally, in response to input 652, the computer system 600 updates the appearance of the depth indicator 630, for example, by changing the color, shadow, and / or visual emphasis of the depth indicator 630 (e.g., restoring the appearance of the depth indicator 630). Figure 6F The described appearance changes indicate that the depth indicator 630 has been deselected.

[0233] exist Figure 6L At this point, computer system 600 detects input 654 on shutter power indicator 624 (e.g., a tap input via a touch-sensitive surface of display 608; in some embodiments, input 650 may include a press of hardware button 606). In response to detecting input 654, computer system 600 initiates media capture using the currently selected capture settings. Figure 6M As illustrated, after capturing media, computer system 600 updates the captured media icon 626 to display a thumbnail of the captured media. Because the depth indicator 630 is deselected and the simulated depth-of-field effect is not applied to camera preview 612 when media capture is performed, the thumbnail of the captured media does not appear with a simulated depth-of-field effect.

[0234] exist Figure 6MAt this point, computer system 600 detects input 656 captured on media icon 626 (e.g., a tap input via the touch-sensitive surface of display 608). In response to detecting input 656, in Figure 6N At this point, computer system 600 displays a media user interface 657, which includes the captured media 658 (e.g., in response to...). Figure 6L The computer system 600 displays a media user interface 657 that includes an aperture value indicator 662, while the depth indicator 630 (e.g., indicating that depth information is being captured) is displayed in the camera user interface 610 during media capture. Since the depth indicator 630 is deselected and simulated depth-of-field effects are not applied to the camera preview 612 during media capture, the captured media 658 is displayed without simulated depth-of-field effects (e.g., both in a larger view of the captured media 658 and in a thumbnail of the captured media 658 in the captured media volume 660), and the aperture value indicator 662 is displayed with a deselected appearance (e.g., as per the description of the captured media 658). Figure 6L The appearance of the depth indicator 630 is described in the text.

[0235] exist Figure 6N At this point, computer system 600 detects input 664 of the selected aperture coefficient indicator 662. For example, input 664 includes short taps and / or clicks via the touch-sensitive surface of display 608, as described below regarding... Figures 8A to 8T and Figure 9 Further details. In response to input 664, in Figure 6O At this point, computer system 600 displays captured media 658 with simulated depth-of-field effects. For example... Figure 6O As illustrated, the simulated depth-of-field effect is applied to the captured media 658 with a simulated aperture setting of f / 1.4, after deselecting the depth indicator 630 (e.g., in...). Figure 6K Before selecting the simulated aperture value (e.g., at), choose the simulated aperture value. Figure 6H The depth-of-field effect is applied to the thumbnail of captured media 658 in captured media volume 660, making the dog appear sharp while the cat, background, and foreground are blurred. Additionally, in response to input 664, computer system 600 updates the appearance of aperture coefficient display 662, for example, by changing the color, shadows, and / or visual emphasis of depth indicator 630 (e.g., in...). Figure 6F (The center and other areas are indicated by shadows) to indicate that the depth indicator 630 has been selected.

[0236] exist Figure 6OAt this point, the computer system detects and selects input 664 of the aperture coefficient indicator 662. For example, input 664 includes a long press input via the touch-sensitive surface of display 608, as described below regarding... Figures 8A to 8T and Figure 9 Further details. In response to input 664, in Figure 6P At this point, computer system 600 displays aperture setting menu 666, which indicates that the current simulated aperture setting is f / 1.4. Figure 6P At this point, computer system 600 detects input 668 (e.g., dragging and / or gesture input via the touch-sensitive surface of display 608) to adjust the aperture setting via aperture setting menu 666 (e.g., as per the context of...). Figures 6G to 6I (As described).

[0237] like Figure 6Q As illustrated, once input 668 is released (e.g., lifted from the touch-sensitive surface of display 608), computer system 600 stops displaying aperture setting menu 666 and displays aperture coefficient indicator 662 with aperture setting indicator 670, which indicates the adjusted aperture setting value (e.g., f / 2.8). In some embodiments, computer system 600 stops displaying aperture setting indicator 670 after a predetermined period of time without input. In response to the release of input 668, computer system 600 updates the simulated depth-of-field effect applied to captured media 658 to reflect the adjusted simulated aperture setting of f / 2.8. Figure 6Q As illustrated, when a simulated depth-of-field effect is applied to the captured media 658, the dog and cat appear sharp, while parts of the environment in front of and / or behind the dog and cat (e.g., further away from the focal plane) are blurred.

[0238] exist Figure 6R At this point, computer system 600 displays a camera user interface 610, including a camera preview 612. For example... Figure 6R As illustrated, the current field of view of the camera, including a portion captured in camera preview 612, comprises a part of the physical environment of the flower, which occupies a large area of ​​camera preview 612. Although the current zoom level is at least 1x magnification, because a specific subject (e.g., a person, animal, and / or other predetermined type of content) is not determined to be within a specific distance range (e.g., 2 to 8 feet from computer system 600), computer system 600 does not display depth indicator 630, indicating that depth information has not been captured. Figure 6R At this point, computer system 600 detects input 672 on shutter power indicator 624 (e.g., a tap input via a touch-sensitive surface of display 608; in some embodiments, input 650 may include a press of hardware button 606) and, in response, initiates media capture.

[0239] like Figure 6S As illustrated, computer system 600 displays media user interface 657, including captured media 676 (e.g., in response to...). Figure 6R The computer system 600 displays the aperture parameter display 662 (the captured media at input 672) and the multi-frame photo capture capability display 674. Because the depth indicator 630 is not displayed in the camera user interface 610 during media capture (e.g., indicating that depth information was not captured), the computer system 600 does not display the aperture parameter capability display 662. Therefore, unlike the captured media 658, the computer system 600 does not provide the option to apply a simulated depth-of-field effect to the captured media 676 after capture.

[0240] like Figure 6T As illustrated, the current field of view of the camera, including a portion captured in camera preview 612, includes a part of the physical environment of the flower. (As per...) Figure 6R As described, although the current zoom level is at least 1x magnification, the computer system 600 does not display the depth indicator 630, indicating that depth information has not been captured, because a specific subject (e.g., a person and / or an animal) is not identified as being within a specific distance range (e.g., 2 to 8 feet from the computer system 600).

[0241] exist Figure 6T At this point, computer system 600 detects input 678 pointing to the location of the flower in camera preview 612 (e.g., a tap input via a touch-sensitive surface of display 608). Although camera preview 612 still does not include a specific subject (e.g., a person, animal, and / or other predetermined type of content), in response to input 678, in Figure 6U At this point, computer system 600 detects the flower as the subject of media capture and indicates the detection with subject indicator 680. Additionally, computer system 600 displays depth indicator 630, indicating that depth information is being captured. Therefore, after detecting input 678, it is possible to... Figures 6C to 6Q The description is used to apply and control simulated depth-of-field effects (e.g., applying and modifying simulated depth-of-field effects in camera preview 612 and / or in media user interface 657 of media captured while displaying depth indicator 630).

[0242] Figure 7This is a flowchart illustrating a method for controlling the capture of media with associated depth information using a computer system, according to some embodiments. Method 700 is performed at a computer system (e.g., 100, 300, 500, and / or 600) that communicates with display generation components (e.g., 608) (e.g., a display controller; a touch-sensitive display system; a display (e.g., integrated and / or connected), a 3D display, a transparent display, a projector, and / or a head-up display), one or more cameras (e.g., 604A, 604B, 604C, and / or 604D) (in some embodiments, the computer system includes one or more cameras, such as rear-view (e.g., user-facing) cameras and front-view (e.g., environment-facing) cameras and / or multiple front-view cameras (e.g., with different lenses, such as standard cameras, telephoto cameras, and / or wide-angle cameras)), and one or more sensors (in some embodiments, the computer system includes one or more depth sensors). Some operations in method 700 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0243] As described below, method 700 provides an intuitive way to control the capture of media with associated depth information. This method reduces the cognitive burden on users controlling the capture of media with associated depth information, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to control the capture of media with associated depth information faster and more efficiently saves power and increases the time interval between battery charging.

[0244] When a computer system (e.g., 600) displays (702) a camera preview (e.g., 612) via a display generation component (e.g., 608), it obtains (704) information about the physical environment corresponding to the field of view of one or more cameras (e.g., a live preview of at least a portion of the camera's field of view; in some embodiments, a camera preview in which depth information (e.g., information about the distance between the camera and one or more subjects or features in the camera's field of view) is not currently displayed and / or captured in the preview; in some embodiments, a camera preview in which depth-based functionality (e.g., simulating depth-of-field effects) is not currently used).

[0245] In response to obtaining information about the physical environment corresponding to the field of view of one or more cameras (706) and determining, based on information about the physical environment corresponding to the field of view of one or more cameras (e.g., made without user intervention) by the computer system (e.g., automatically determined and / or determined without explicit user input requesting the display of a first user-selectable user interface object), that depth capture criteria (e.g., portrait criteria; in some embodiments, depth capture criteria include the current zoom setting being 1x or higher; in some embodiments, depth capture criteria include detecting a specific subject (e.g., a person and / or a pet) within a specific distance from the camera (e.g., between 2 and 8 feet, more than 3 feet, and / or less than 10 feet)) is met (e.g., as... Figure 6C As illustrated), the computer system displays (708) via a display generation component (e.g., 608) (in some embodiments, an initial display; in some embodiments, the display of the first optional user interface object is maintained if the first optional user interface object is displayed and the depth capture criterion continues to be met). The first optional user interface object (e.g., 630) (e.g., a depth indicator / energy representation, such as an aperture factor icon and / or another indicator, displayed as part of the camera UI and / or overlaid on the camera preview), which, when selected (e.g., via user input pointing to the first optional user interface object, such as a tap gesture or an air gesture; in some embodiments, in response to detecting user input selecting the first optional user interface object, the computer system (e.g., using a first type of input, such as a tap and / or a short input) switches the depth energy representation between states) initiates a process for performing a first depth-based media capture function (e.g., as shown in the diagram). Figure 6F (Examples include, for example, displaying and / or stopping the display of a simulated depth-of-field preview, capturing media with simulated depth of field applied, and / or controlling simulated depth-of-field settings; in some embodiments, in response to detecting user input that selects a first optional user interface object, the computer system initiates a process for performing a first depth-based media capture function).

[0246] In response to obtaining information about the physical environment corresponding to the field of view of one or more cameras (706) and determining (in some embodiments, without user intervention) that the depth capture criterion is not met, the computer system abandons (710) displaying a first optional user interface object (e.g., such as...). Figure 6B , Figure 6R and Figure 6T(As illustrated) (e.g., maintaining a camera preview without displaying information about the distance between the camera and one or more subjects or features in the camera's field of view, or stopping the display of a first optional user interface object if it is displayed and the depth capture criterion is no longer met). A depth capture capability representation is provided that allows users to initiate depth-based media capture functions (e.g., capturing depth information and / or using depth information for media capture) when the current media capture preview meets the depth capture criterion, providing users with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps users synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities with unintended settings (e.g., not capturing depth information when the user expects depth information / effects), which enhances system operability and makes the user-system interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when interacting with the operating system / system), which in turn reduces power consumption and extends system battery life by enabling users to use the system more quickly and efficiently. Additionally, the depth capture capability indicator provides the user with real-time visual feedback on the status of the computer system (e.g., whether depth information is being captured and / or whether depth-based media capture functionality is available). For example, the initial display of the depth capture capability indicator indicates that depth information can be captured and / or will be captured, and allows the user quick access to relevant functionality for media capture if needed.

[0247] In some embodiments, a first optional user interface object (e.g., 630) indicates the capture status of depth information (e.g., information related to the distance between one or more cameras and subjects in the physical environment and / or the distance between different subjects in the physical environment; in some embodiments, a depth map) of the physical environment corresponding to the field of view of one or more cameras (e.g., displaying the first optional user interface object to indicate that depth information is currently being captured) (e.g., captured by one or more cameras and / or one or more sensors; in some embodiments, a depth indicator to indicate that depth information is currently being captured; in some embodiments, a depth indicator to indicate that depth information will be captured). In some embodiments, the method further includes: capturing (e.g., caching and / or storing) depth information (e.g., information related to the distance between one or more cameras and subjects in the physical environment and / or the distance between different subjects in the physical environment; in some embodiments, a depth map) of the physical environment corresponding to the field of view of one or more cameras while displaying the first optional user interface object (e.g., displaying the first optional user interface object to indicate that depth information is currently being captured). Capturing depth information while displaying a depth capture capability indication provides the user with real-time visual feedback on the status of the computer system (e.g., whether depth information is being captured). This also helps users synthesize media capture events and reduces the risk of missing (e.g., due to users trying to confirm whether depth information is being captured) or capturing in unexpected settings (e.g., not capturing depth information when users expect depth information / effects).

[0248] In some implementations, depth information is based on information from one or more depth sensors (e.g., structured light sensors, time-of-flight sensors (e.g., light detection and ranging and / or ultrasonic sensors) and / or stereo camera sensors).

[0249] In some implementations, depth information is based on a comparison of first image data obtained from a first camera among one or more cameras with second image data obtained from a second camera among one or more cameras (e.g., 604B, 604C, and / or 604D). (e.g., the two cameras are facing substantially the same direction but are separated from each other by a known distance, such that depth information can be derived from the difference between the first image data and the second image data (e.g., the closer the subject is to the two cameras, the greater the difference between the image data corresponding to the subject in the first image data and the second image data)).

[0250] In some implementations, the computer system detects input (e.g., 638, 640, and / or 652) selecting a first optional user interface object (e.g., 630); and in response to detecting input selecting the first optional user interface object, the computer system initiates a process for performing a first depth-based media capture function, wherein the process for performing the first depth-based media capture function includes switching between an enabled state and a disabled state between simulated depth-of-field capture modes (e.g., a mode that captures and / or uses depth information while capturing media (e.g., a live preview to apply depth effects)) (e.g., such as...). Figures 6E to 6F and Figures 6K to 6L (As illustrated) (e.g., enabling depth capture mode based on determining that depth capture mode is disabled when a selection of a first user-selectable user interface object is detected, and disabling depth capture mode based on determining that depth capture mode is enabled when a selection of a first user-selectable user interface object is detected; in some embodiments, enabling depth capture mode includes initializing the depth capture mode settings (e.g., aperture factor settings) to a first value; in some embodiments, the first value is the last used value; in some embodiments, the first value is a default value). Using a first selectable user interface object to switch depth capture mode provides improved control over media capture settings without cluttering the media capture interface with unnecessary controls. This also helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings (e.g., not capturing depth information when the user expects depth information / effects), which enhances system operability and makes the user-system interface more efficient.

[0251] In some implementations, in response to detecting input selecting a first optional user interface object (e.g., 638, 640, and / or 652) and determining that the simulated depth-of-field capture mode is enabled, the computer system displays a simulated depth-of-field effect (e.g., such as depth information about the physical environment corresponding to the field of view of one or more cameras; information related to the distance between one or more cameras and subjects in the physical environment and / or the distance between different subjects in the physical environment; in some implementations, a depth map) based on depth information about the physical environment corresponding to the field of view of one or more cameras. Figures 6F to 6KThe camera preview (e.g., 612) (e.g., a live capture preview) (e.g., applying blur and / or other depth-based visual effects to image data captured using one or more cameras; in some embodiments, the simulated depth-of-field effect is applied only when a specific subject (e.g., a person and / or animal of a specific size in the camera preview) is detected (e.g., blurring the portion of the camera preview that corresponds to an element further away from one or more cameras in the physical environment compared to the portion of the camera preview that corresponds to an element closer to one or more cameras in the physical environment). In some embodiments, in response to detecting input selecting a first optional user interface object (e.g., 638, 640, and / or 652) and based on determining that the simulated depth-of-field capture mode is disabled (e.g., before initially selecting the first optional user interface object or after deselecting the first optional user interface object (e.g., in response to detecting the selection of the first optional user interface object when the depth capture mode is already enabled)), the computer system displays a camera preview without simulated depth-of-field effects (e.g., as shown). Figure 6L (As illustrated). Applying simulated depth-of-field effects to the live camera preview when depth capture mode is enabled provides users with real-time visual feedback on the state of the computer system. For example, the simulated depth-of-field effect indicates to the user that depth information is being captured and allows the user to preview how the captured media will look with the simulated depth-of-field effect applied. This also helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings, which enhances system operability and makes the user-system interface more efficient.

[0252] In some implementations, the process of initiating the execution of the first depth-based media capture function includes: based on determining that the input for selecting the first optional user interface object is an input of a first input type (e.g., 638, 640, and / or 652) (e.g., a short press and / or a tap), switching between an enabled state and a disabled state to simulate a depth-of-field capture mode (e.g., as shown in the image). Figures 6E to 6F and Figures 6K to 6L(As illustrated). In some embodiments, the process of initiating the execution of a first depth-based media capture function includes: displaying a second optional user interface object (e.g., 640) (e.g., long / hold press and / or tap) via a display generation component based on determining that the input selecting the first optional user interface object is a second type of input different from the first type (e.g., 640) (e.g., long / hold press and / or tap). This second optional user interface object, when selected (e.g., via input 642) (e.g., via user input pointing to the second optional user interface object, such as a tap gesture or air gesture), controls (e.g., allows the user to set) the settings (and / or select its value) of an analog depth-of-field capture mode (e.g., as shown in the image). Figures 6G to 6I (As illustrated) (in some embodiments, the simulated aperture value simulates the depth-of-field effect; in some embodiments, a slider can be dragged (e.g., left and right and / or up and down) to select among multiple aperture values; in some embodiments, an optional menu and / or tray includes an optional aperture value for each of the multiple aperture values; in some embodiments, in response to detecting user input selecting a second optional user interface object, the computer system changes the settings of the simulated depth-of-field capture mode based on the user input (e.g., the selected aperture value and / or the degree to which the slider is dragged). Using a first optional user interface object to switch the depth capture mode and access additional options for the depth capture mode provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps the user synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities with unintended settings, which enhances system operability and makes the user-system interface more efficient.

[0253] In some implementations, when displaying a camera preview (e.g., 612), the computer system displays at least one optional user interface object (e.g., 614, 616, 618, 620, and / or 622) different from the first optional user interface object via a display generation component. When this at least one optional user interface object is selected (e.g., via user input pointing to the at least one optional user interface object different from the first optional user interface object, such as a tap gesture or air gesture), a process is initiated to perform a media capture function different from the first depth-based media capture function. (e.g., the different media capture function includes controlling flash settings (e.g., setting the flash to on, off, and / or automatic), controlling multi-frame capture settings (e.g., turning live photo and / or burst mode on or off). The system can control exposure settings (e.g., enabling longer exposure times (e.g., low-light settings) and / or setting maximum exposure times), control zoom settings (e.g., zoom in or zoom out), select a media capture mode (e.g., choosing between modes for capturing standard photos, standard videos, portrait mode photos, high frame rate videos, and / or panoramic photos), and / or select one or more cameras (in some embodiments, user-facing or environment-facing cameras; in some embodiments, selection between different environment-facing cameras); in some embodiments, in response to detecting user input selecting at least one optional user interface object, the computer system initiates a process for performing a media capture function different from the first depth-based media capture function; in some embodiments, as per the description... Figures 8A to 8T and Figure 9 (At least one optional user interface object as described).

[0254] In some implementations, when displaying a camera preview, the computer system displays a third optional user interface object (e.g., 624) (e.g., a shutter power indicator) different from the first optional user interface object via a display generation component. This third optional user interface object, when selected (e.g., via inputs 650, 654, and / or 672) (e.g., via user input pointing to the third optional user interface object, such as a tap gesture or air gesture), initiates the capture of media (e.g., as shown in the image). Figures 6J to 6M and Figures 6R to 6S (As illustrated) (e.g., using one or more cameras to capture photographic media and / or capture video media, and storing the captured media (in some embodiments, along with any associated metadata, such as depth information) to) a media library (in some embodiments, a media library stored on the internal storage of a computer system; in some embodiments, a media library stored on a remote storage device (e.g., a cloud library); in some embodiments, a media library stored on an external storage device; for example, as described in... Figures 12A to 12T and Figure 13(as described); in some implementations, in response to detecting user input that selects a third optional user interface object, the computer system initiates a process of capturing photographic media and / or video media.

[0255] In some implementations, the computer system captures the first media (e.g., such as...) Figure 6K , Figure 6M and Figure 6S (As illustrated) (e.g., using one or more cameras to capture photographic media and / or capture video media, and storing the captured media (in some embodiments, together with any associated metadata, such as depth information) to a media library (in some embodiments, a media library stored on the internal storage of a computer system; in some embodiments, a media library stored on a remote storage device (e.g., a cloud library); in some embodiments, a media library stored on an external storage device; for example, as described in...) Figures 12A to 12T and Figure 13 (As described). In some embodiments, after capturing the first media (in some embodiments, automatically after capturing the first media; in some embodiments, in response to user input requesting to view the first media (in some embodiments, selection of the photo slot; in some embodiments, input navigating to the photo volume)), the computer system displays a representation of the first media via a display generation component (e.g., 658 and / or 676 and / or included in 626) (e.g., a thumbnail of the first media, still frames of the first media (e.g., for video media and / or multi-frame photo capture) and / or the media itself; in some embodiments, the representation of the first media is displayed while displaying a camera preview (e.g., displaying a thumbnail of the captured media in the photo slot); in some embodiments, the display of the camera preview is stopped and the representation of the first media is displayed in a different UI (e.g., a collection of recently captured media and / or the user's media library)). Displaying the captured media provides the user with real-time visual feedback on the status of the computer system and helps the user synthesize media capture events. For example, the user can use the current media capture settings to check the appearance of the captured media and, if necessary, adjust the media capture settings for additional media capture.

[0256] In some implementations, the representation of the first media includes: based on determining when the capture of the first media is initiated (e.g., as...). Figure 6J(Example) A first optional user interface object is displayed in a first state (e.g., selected state, enabled state, and / or on state; in some embodiments, the depth representation is placed in the first state in response to user input that selects the depth representation while it is in a second state), displaying a simulated depth-of-field effect (e.g., as illustrated by...) based on depth information about the physical environment corresponding to the field of view of one or more cameras (e.g., information related to the distance between one or more cameras and a subject in the physical environment and / or the distance between different subjects in the physical environment; in some embodiments, a depth map). Figures 6K to 6L The representation of the first media (illustrated by media icon 626 in the image) (e.g., applying blur and / or other depth-based visual effects to the media, still image, and / or thumbnail) (e.g., blurring the portion of the camera preview corresponding to elements further away from one or more cameras in the physical environment compared to the portion of the camera preview corresponding to elements closer to one or more cameras in the physical environment). In some embodiments, displaying the representation of the first media includes: determining, when initiating the capture of the first media (e.g., as shown in the image), the representation of the first media. Figure 6L (As illustrated) The first optional user interface object is not displayed in the first state (in some embodiments, when the first optional user interface object is displayed in a (second) deselected / disabled state, for example, in response to user input that selects the depth display when the depth display is in the first state; in some embodiments, when the first optional interface object is not displayed), a representation of the first medium that does not have a simulated depth-of-field effect is displayed (e.g., such as...). Figures 6M to 6N (Illustrations of captured media icons 626 and 658 are shown in the image). Displaying a representation of the captured media with or without simulated depth effects, based on whether the depth capture setting is selected or deselected, provides the user with real-time visual feedback on the state of the computer system and helps the user synthesize media capture events. For example, the user can use the current depth capture settings to check the appearance of the captured media and, if necessary, adjust the depth capture settings for additional media capture.

[0257] In some implementations, while displaying the representation of the first media (e.g., 658) (in some implementations, in the camera roll and / or media library UI), the computer system displays a fourth optional user interface object (e.g., 662) (e.g., a depth and / or portrait mode display), which, when selected (e.g., via input 664) (e.g., via user input pointing to the fourth optional user interface object, such as a tap gesture or an air gesture), controls whether the representation of the first media displays a simulated depth-of-field effect (e.g., as shown in the image). Figures 6N to 6Q(Examples provided) (e.g., applying blur and / or other depth-based visual effects to the media) (In some embodiments, the simulated depth-of-field effect is based on depth information about the physical environment corresponding to the field of view of one or more cameras (e.g., information related to the distance between one or more cameras and subjects in the physical environment and / or the distance between different subjects in the physical environment; in some embodiments, a depth map) (e.g., a portion of the camera preview corresponding to one or more background elements further away from one or more cameras in the physical environment is more blurred than a portion of the camera preview corresponding to one or more subjects that have been manually or automatically selected as sharp in the physical environment, and / or a portion of the camera preview corresponding to one or more foreground elements closer to one or more cameras in the physical environment is more blurred than a portion of the camera preview corresponding to one or more subjects that have been manually or automatically selected as sharp in the physical environment); In some embodiments, in response to detecting user input that selects a fourth optional user interface object: based on determining that the fourth optional user interface object is displayed in a deselected / disabled state when the user input is selected, the computer system displays media with simulated depth-of-field effects; and based on determining that the fourth optional user interface object is displayed in a selected / enabled state when the user input is selected, the computer system displays media without simulated depth-of-field effects).

[0258] In some implementations, a first optional user interface object is displayed when it is determined that capture of the first media is initiated (e.g., such as...). Figure 6J and Figure 6L As illustrated, depth information about the physical environment corresponding to the field of view of one or more cameras (e.g., information relating to the distances between one or more cameras and subjects in the physical environment and / or the distances between different subjects in the physical environment; in some embodiments, a depth map) (e.g., a more blurred portion of the camera preview corresponding to elements further away from one or more cameras in the physical environment compared to the portion of the camera preview corresponding to elements closer to one or more cameras in the physical environment) can be used to display a representation of a first medium having a simulated depth-of-field effect (e.g., as shown in the example) Figures 6N to 6Q (As illustrated) (e.g., applying blur and / or other depth-based visual effects to the media). In some implementations, based on the determination that a first optional user interface object is not displayed when initiating capture of the first media (e.g., as shown in the example). Figure 6R As illustrated, depth information cannot be used to display a representation of a first medium that simulates depth of field (e.g., as shown in the example). Figure 6S(As illustrated). Providing the user with the option to apply a simulated depth-of-field effect after media capture offers improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings, enhancing system operability and making the user-system interface more efficient.

[0259] In some embodiments, the computer system detects a first user input (e.g., 678) pointing to a first area of ​​the camera preview (in some embodiments, a tap on the camera preview) (in some embodiments, when abandoning the display of a first optional user interface object (e.g., when the depth capture criterion is not met)). In some embodiments, in response to detecting the first user input and based on the determination that the first area of ​​the camera preview includes a corresponding subject in the physical environment (e.g., an identifiable person, animal, and / or object; in some embodiments, detected based on information about the physical environment corresponding to the field of view of one or more cameras (e.g., camera data and / or depth information)) (in some embodiments, when abandoning the display of the first optional user interface object (e.g., when the depth capture criterion is not met), the computer system captures (e.g., caches and / or stores) depth information about the physical environment corresponding to the field of view of one or more cameras (e.g., information related to the distance between one or more cameras and subjects in the physical environment and / or the distance between different subjects in the physical environment; in some embodiments, a depth map) (in some embodiments, depth information that can be used to present media items with synthetic depth-of-field effects) (e.g., such as Figure 6U (As illustrated) (e.g., when a depth indication is not displayed and / or the depth capture criteria are not met, the user can alternatively enable depth information capture by tapping a subject in the camera preview; in some implementations, depth information capture is abandoned if a first area of ​​the camera preview is determined not to include a representation of the corresponding subject, and if displaying a first optional user interface object is abandoned (e.g., when the depth capture criteria are not met). Capturing depth information in response to detecting a tap on a subject in the camera preview provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps the user synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities with unintended settings (e.g., not capturing depth information when the user expects depth information / effects), which enhances system operability and makes the user-system interface more efficient.

[0260] In some implementations, depending on whether a portrait capture mode (e.g., a mode in which the captured media item is presented with a synthetic depth-of-field effect) is enabled (in some implementations, regardless of whether a depth capture capability representation is displayed and / or depth capture criteria are met), the computer system captures (e.g., caches and / or stores) depth information about the physical environment corresponding to the field of view of one or more cameras (e.g., information related to the distances between one or more cameras and subjects in the physical environment and / or the distances between different subjects in the physical environment; in some implementations, a depth map) (e.g., such as...). Figure 6D (As illustrated) (e.g., regardless of whether a subject is detected, whether a subject has been selected based on user input, and / or whether the user has adjusted any portrait mode settings) (e.g., always capturing depth information in portrait capture mode). In some implementations, media items captured in portrait capture mode include depth information (e.g., which can be used to render one or more depth effects (e.g., synthesized depth of field)). Capturing depth information in portrait capture mode provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps the user synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities with unintended settings (e.g., not capturing depth information when the user expects depth information / effects), which enhances system operability and makes the user-system interface more efficient.

[0261] In some implementations, when (in some implementations, depending on determination) the camera preview includes a representation of the corresponding subject (e.g., a person, animal, and / or object; in some implementations, detected based on information about the physical environment corresponding to the field of view of one or more cameras (e.g., camera data and / or depth information)) and depending on determination, a portrait capture mode is enabled (e.g., using a mode selection enablement representation (e.g., for selecting between standard photo, standard video, portrait, panorama, and high frame rate video modes)), the computer system, based on depth information about the physical environment corresponding to the field of view of one or more cameras (e.g., information related to the distance between one or more cameras and subjects in the physical environment and / or the distance between different subjects in the physical environment; in some implementations, a depth map) (e.g., such as...) Figure 6D(As illustrated) Displays a camera preview (e.g., a live capture preview) with simulated depth-of-field effects (e.g., applying blur and / or other depth-based visual effects to image data captured using one or more cameras) (e.g., a more blurred portion of the camera preview corresponding to elements further away from one or more cameras in the physical environment compared to the portion of the camera preview corresponding to elements closer to one or more cameras in the physical environment). In some embodiments, when the camera preview includes a representation of the corresponding subject, and based on determining that a portrait capture mode is not enabled (e.g., the mode is disabled and / or another mode (e.g., standard photo mode) is enabled) and a first optional user interface object (e.g., a depth indication) is not enabled (in some embodiments, the first optional user interface object is displayed but not selected; in some embodiments, the first optional user interface object is not displayed), the computer system displays a camera preview (e.g., a live capture preview) without simulated depth-of-field effects (e.g., as illustrated). Figure 6C , Figure 6E , Figure 6L and Figure 6M (As illustrated). When a subject is detected and the camera is capturing in portrait mode, a live preview of the simulated depth effect is displayed, providing the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings (e.g., failing to capture depth information when the user expects depth information / effects). This enhances the operability of the system and makes the user-system interface more efficient.

[0262] In some implementations, depth capture criteria include criteria that must be met when the scaling settings are set to at least a threshold magnification (e.g., ...). Figure 6C and Figure 6U (As illustrated) (e.g., a depth indication is displayed when the camera is zoomed to or above a threshold zoom level (e.g., 0.75x zoom, 1x zoom, and / or 2x zoom, but not when the camera is zoomed below the threshold zoom level); in some implementations, the depth indication is displayed at any zoom level (e.g., depth capture criteria do not include criteria satisfied by meeting the threshold zoom level)). Displaying a depth capture indication when zoomed above a certain threshold provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls, for example, when additional display controls are unlikely to be useful and / or less desired by the user given the current composition of the media capture.

[0263] In some implementations, depth capture criteria include a first criterion that is met when a representation of a corresponding subject (e.g., a person, animal, and / or other specific subject) is detected in a camera preview, and a second criterion that is met when the distance from one or more cameras to the corresponding subject in the physical environment falls within a distance range (e.g., the subject is 2 feet to 8 feet, 0 feet to 10 feet, and / or 5 feet to 15 feet away). Figure 6C (As illustrated) (In some implementations, the depth capture criterion includes a third criterion that is met when the lighting quality of the camera preview (e.g., brightness, contrast, and / or the currently selected flash setting) meets a quality criterion (e.g., the subject is or will be adequately illuminated in the media capture). A depth capture capability indication is displayed when zoomed in above a certain threshold, providing the user with improved media capture control options without cluttering the media capture user interface with additional display controls, for example, when additional display controls are unlikely to be useful and / or are not expected by the user given the current composition of the media capture.

[0264] In some implementations, when a first optional user interface object (e.g., 630) is displayed, the computer system displays a zoomable interface object (e.g., 620) (e.g., a button, slider, and / or a menu / dish with multiple optional options), which, when selected (e.g., via input 627) (e.g., via user input pointing at the zoomable interface object, such as a tap gesture or air gesture), controls the zoom level of the camera preview (e.g., as shown in the image). Figures 6B to 6C (As illustrated) (In some embodiments, the depth indicator and zoom control are displayed close to each other in the same area of ​​the camera UI (e.g., both at the bottom of the camera preview, with the depth indicator in the lower left corner and the zoom control at the bottom center, and / or both on the same side of the camera preview, with the depth indicator in the corner and the zoom control in the center); in some embodiments, in response to detecting input selecting a zoom interface object, the computer system changes the zoom level of the camera preview based on that input (e.g., based on the degree to which the slider or wheel indicator is dragged and / or based on a specific option selected using that input). Displaying both the zoom control and the depth indicator helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings, which enhances system operability and makes the user-system interface more efficient. For example, users can check both the depth capture settings and the zoom settings without extensively searching for different controls.

[0265] In some implementations, the computer system detects input of a corresponding type (e.g., 640) directed to a first optional user interface object (e.g., 630) (e.g., long input / hold input). In some implementations, in response to detecting input of a corresponding type directed to the first optional user interface object, the computer system stops displaying the zoom interface object (e.g., 620) and displays a fifth optional user interface object (e.g., 644) (e.g., an extended options enablement for depth capture operations, such as a menu and / or slider), which controls (e.g., allows the user to set) media capture settings associated with the first optional user interface object (and / or selects its values) (e.g., such as...). Figures 6G to 6I (As illustrated) (e.g., aperture values ​​used to apply simulated depth-of-field effects; in some embodiments, a slider can be dragged (e.g., left and right and / or up and down) to select among multiple aperture values; in some embodiments, optional menus and / or trays include optional displays for each of the multiple aperture values ​​that can be selected using tap and / or click input). Hiding the zoom controls and providing additional media capture settings offers the user improved media capture control options without cluttering the media capture user interface with additional display controls.

[0266] In some implementations, the first optional user interface object (e.g., 630), when displayed in an enabled state (e.g., when selected), includes an indication (e.g., 648) of the current value of the depth-based media capture setting (e.g., the currently selected aperture size (aperture factor) value). In some implementations, in response to detecting a change in the current value of the depth-based media capture setting, the indication of the current value of the depth-based media capture setting is updated (e.g., as shown in the image). Figures 6J to 6K and Figures 8M to 8O (As illustrated). In some implementations, the indication of the current value of the depth-based media capture setting has a first value based on the determination of the current value of the depth-based media capture setting (e.g., the indication of the current value of the depth-based media capture setting includes a first number indicating an analog depth stop setting), and the indication of the current value of the depth-based media capture setting has a second value different from the first value based on the determination of the current value of the depth-based media capture setting (e.g., the indication of the current value of the depth-based media capture setting includes a second number indicating an analog depth stop setting, or does not include a number indicating that a default value of the depth-based media capture setting is being used). Displaying and updating the depth display using the indication of the currently selected value of the depth-based media capture setting provides the user with real-time visual feedback on the status of the computer system and helps the user synthesize media capture events.

[0267] It should be noted that the above text regarding method 700 (for example, Figure 7 The details of the process described herein also apply in a similar manner to the methods described below. For example, method 700 optionally includes one or more characteristics of the various methods described below with reference to methods 900, 1100, 1300, 1700, 1900, 2100, and / or 2300. For example, when capturing media with (or without) associated depth information according to method 700, both the capture of the depth information and other media capture settings can be controlled as described with respect to method 900. For example, the camera user interface in conjunction with the depth indication of method 700 can also be combined with the zoom user interface described with respect to method 1100 and the external storage user interface described with respect to method 1300. For the sake of brevity, these details will not be repeated below.

[0268] Figures 8A to 8T Exemplary user interfaces for controlling media capture settings are illustrated according to some implementation schemes. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 9 The process in.

[0269] exist Figure 8A At this point, computer system 600 displays camera settings user interface 802 via display 608. This camera settings user interface is used to select and configure controls for a set of camera settings to be provided in the camera user interface (e.g., camera user interface 610). Figure 8A As illustrated, this set of camera settings includes media format (e.g., "media type") settings (e.g., resolution and / or compression codec for media capture), depth capture settings (e.g., for simulating depth-of-field effects, such as regarding...) Figures 6A to 6U and Figure 7 (Detailed description), camera flash settings and low-light (e.g., "night mode") capture settings (e.g., for exposure length).

[0270] The camera settings user interface 802 includes switching display indicators 804A, 804B, 804C, and 804D, corresponding to media format settings, depth capture settings, camera flash settings, and low-light capture settings, respectively. When selected, the switching display indicators 804A, 804B, 804C, and 804D control whether the control display indicator for the corresponding camera setting is included in the camera user interface 610. Figure 8A As illustrated, the switching indicator shows that each of 804A, 804B, 804C, and 804D is set to the "on" state. Therefore, for now, refer to... Figure 8DThe computer system 600 displays a camera user interface 610 having a media format power indicator 616, a depth indicator 630, a flash power indicator 614, and a night mode power indicator 816. In some embodiments, even when the power indicator 804B is set to "on," the depth indicator 630 is only displayed when depth capture criteria are met (e.g., a zoom level of at least 1x and / or a specific subject (e.g., a person or animal) is detected within a specific distance range), as described above regarding... Figures 6A to 6U and Figure 7 As described. See again. Figure 8A The camera settings user interface 802 includes default display indicators 806A and 806B, corresponding to media type settings and aperture settings, respectively. Default display indicator 806A indicates the currently selected default media format setting (e.g., "HEIC 12") for 12 megapixel resolution and High Efficiency Image Decoding (HEIC) compression. Default display indicator 806B indicates the currently selected default aperture value (e.g., "2.4") for f / 2.4.

[0271] exist Figure 8A At this point, computer system 600 detects input 808 of the default display 806A (e.g., a tap input via the touch-sensitive surface of display 608). In response to detecting input 808, in Figure 8B At this point, computer system 600 displays a default settings user interface 810 for media format settings. The default settings user interface 810 includes a default settings menu 812 that lists default settings options for media format settings, including 12-megapixel HEIC (e.g., "HEIC12"), 48-megapixel HEIC (e.g., "HEIC 48"), 12-megapixel raw decoded (e.g., "RAW 12"), and 48-megapixel raw decoded (e.g., "RAW 48"). Computer system 600 displays a selection indicator 814A indicating that 12-megapixel HEIC is currently selected as the default media format setting.

[0272] exist Figure 8B At this point, computer system 600 detects input 813 selecting the 48-megapixel HEIC option for media format settings (e.g., a tap input via the touch-sensitive surface of display 608). In response to detecting input 813, in Figure 8C At this point, computer system 600 displays selection indicator 814B, indicating that 48 megapixels HEIC has been selected as the additional default value for media format settings.

[0273] exist Figure 8DAt this location, computer system 600 displays camera user interface 610, which, in addition to media format indicator 616, depth indicator 630, flash indicator 614, and night mode indicator 816, also includes camera preview 612, zoom indicator 620, capture mode menu 622, shutter indicator 624, and captured media icon 626 (e.g., as mentioned above regarding...). Figures 6A to 6U As described. Figure 8D As illustrated, the current field of view of the camera, as shown in camera preview 612, includes a cactus in the foreground, a person in the midground, and landscape features and sky in the background. Computer system 600 detects the person as the subject of media capture and indicates this detection with a subject indicator 817. As discussed above, in some embodiments, computer system 600 displays a depth indicator 630 because the switching power indicator 804B is set to "on" and because depth capture criteria are met (e.g., because the zoom level is at least 1x and a person is detected within a distance of 2 to 8 feet from computer system 600).

[0274] exist Figure 8D At this point, the camera flash setting is currently off (e.g., disabled), so the flash will not be used if media capture is initiated. Therefore, computer system 600 displays a flash power indicator 614 in the "off" state, showing the line through the flash power indicator 614 and no longer visually emphasizing it. The low-light capture setting is currently off (e.g., disabled), so standard shorter exposure times (e.g., 0.05 seconds, 0.1 seconds, and / or 0.5 seconds) will be used when capturing media. Therefore, computer system 600 displays a night mode power indicator 816 in the "off" state, showing the line through the night mode power indicator 816 and no longer visually emphasizing it. The depth capture setting is currently deselected, so computer system 600 does not display a camera preview 612 with simulated depth-of-field effects (although in some embodiments, computer system 600 still captures depth information, as indicated by the display of the depth indicator 630 (e.g., as mentioned above regarding...). Figures 6A to 6U and Figure 7 As described above). Therefore, the computer system 600 displays the depth indicator 630 in a deselected state, visually de-emphasizing the depth indicator 630 (e.g., as described above regarding...). Figure 6L (As described). The media format setting is currently set to 12 megapixel HEIC (e.g., one of the previously selected default media format options), indicated by the text "HEIC 12" included in the media format enablement display 616.

[0275] exist Figure 8DAt this location, computer system 600 detects input 818 pointing towards the position of night mode display 816. Input 818 is a first input type. For example, the first input type includes a "short" tap and / or click input via the touch-sensitive surface of display 608, wherein a finger press event is detected for a finite amount of time (e.g., 0.05 seconds, 0.1 seconds, and / or 0.5 seconds), followed by a finger lift-off event. In response to the detection of input 818 (e.g., a first type of input), in Figure 8E At this point, computer system 600 changes the low-light capture setting to an on (e.g., enabled) state, specifically an on state in which media capture will be performed with a maximum exposure time of 1 second. In some embodiments, when the low-light capture setting is on, computer system 600 dynamically adjusts the exposure time for media capture to the maximum exposure time, for example, based on detected ambient light and / or the determined brightness of camera preview 612. In response to the detection of input 818 (e.g., and the low-light capture setting being changed to an on state with a maximum exposure time of 1 second), computer system 600 updates the appearance of night mode indicator 816, for example, by removing lines through night mode indicator 816 and / or visually emphasizing night mode indicator 816 to indicate that low-light capture mode is enabled, and includes a textual indication of the maximum exposure time (e.g., "1S").

[0276] In some implementations, in response to the detection of additional input of the first input type, the computer system 600 cycles through a set of predetermined states for low-light capture settings. For example, as illustrated in sidebar 822, this set of predetermined states may include an on state with a maximum exposure time of 1 second, an on state with a maximum exposure time of 3 seconds, and / or an off (e.g., disabled) state, so that a user can enable, disable, and / or adjust the maximum exposure time using a short tap input on the night mode indicator 816.

[0277] exist Figure 8E At this location, computer system 600 detects input 820 pointing to the position of night mode display 816. Input 820 is a second input type, different from the first input type. For example, the first input type includes a "long" press input via the touch-sensitive surface of display 608, wherein no finger lift-off (lift-off) event is detected for at least a threshold time period after the finger press event is detected (e.g., the input is held for at least 0.5 seconds, 1 second, and / or 1.5 seconds).

[0278] In response to the detection of input 820 (e.g., second type of input), in Figure 8FAt this location, computer system 600 displays exposure slider 824. Exposure slider 824 indicates the current maximum exposure time of 1 second. In some embodiments, computer system 600 displays (e.g., animated display) exposure slider 824 expanded from night mode display display 816. In some embodiments, exposure slider 824 represents multiple maximum exposure times. For example, the scale markings of exposure slider 824 may represent discrete increments (e.g., 0.1 seconds, 0.5 seconds, and / or 1 second) of the maximum exposure time within a total exposure range (e.g., 0.5 seconds to 3.0 seconds, 1 second to 5 seconds, and / or 0 seconds to 10 seconds). Figure 8F As illustrated, when the exposure slider 824 is displayed, the computer system 600 stops displaying other user interface elements, such as the media format indicator 616, in the area of ​​the exposure slider 824. In some embodiments, the computer system 600 stops displaying the exposure slider 824 after detecting a finger lift-off (lift-off) event of input 820 after a threshold time period, provided that no input is detected on the exposure slider 824 and / or the night mode indicator 816. In some embodiments, the computer system 600 stops displaying the exposure slider 824 in response to detecting input at a location other than the location of the exposure slider 824.

[0279] like Figure 8F As illustrated, while the exposure slider 824 is displayed, the computer system 600 detects input 826 along the exposure slider 824 (e.g., dragging and / or gesture input via a touch-sensitive surface of the display 608). In some embodiments, input 826 is a continuation of input 820 (e.g., a gesture detected after a finger press event and before any finger lift event is detected). In some embodiments, input 826 is a different input detected while the exposure slider 824 is still displayed (e.g., a gesture detected after a finger lift event of input 818 is detected). In some embodiments, in response to detecting that input 826 is dragged to the left, the computer system 600 increases the maximum exposure time of the low-light capture setting, and in response to detecting that input 826 is dragged to the right, the computer system 600 decreases the maximum exposure time of the low-light capture setting. In some embodiments, the computer system 600 determines the adjusted maximum exposure time based on the distance traveled by input 826 (e.g., net distance and / or normalized distance). For example, input 826 is the distance to the left corresponding to the distance between the scale mark of the exposure slider 824 representing the maximum exposure time of 1 second and the scale mark of the exposure slider 824 representing the maximum exposure time of 2 seconds.

[0280] like Figure 8F As illustrated, in response to the detection of input 826, computer system 600 changes the low-light capture setting to an enabled state with a maximum exposure time of 2 seconds. (For reference only) Figure 8EThe illustrated sidebar 822, with its 2-second maximum exposure time enabled state, is not included in the set of predetermined states that can be selected (e.g., cycled between) using the first type of input. Therefore, in response to input 820 (e.g., a second type of input), the computer system 600 provides an additional state for the low-light capture setting via the exposure slider 824. Figure 8G As illustrated, after releasing input 826, computer system 600 stops displaying exposure slider 824 and displays an enabled appearance (e.g., as shown in the image). Figure 8E The night mode indicator 816 includes a textual indication of the maximum exposure time (e.g., “2S”).

[0281] exist Figure 8G At this location, computer system 600 detects input 828 pointing to the position of night mode display 816. Input 828 is a first input type (e.g., as described above regarding input 818). In response to detecting input 828 (e.g., a first type of input), in Figure 8H At this point, computer system 600 changes the low-light capture setting to an off (e.g., disabled) state. Additionally, in response to input 828, computer system 600 updates the appearance of night mode display 816, for example, by visually emphasizing the lines of night mode display 816 (e.g., restoring the appearance of the lines of night mode display 816). Figure 8E The described appearance changes indicate that the low-light capture mode is off.

[0282] exist Figure 8H At this location, computer system 600 detects input 830 pointing to the location of media format display 616. Input 830 is a first type of input (e.g., a "short" tap and / or click input via the touch-sensitive surface of display 608, as described with respect to input 818). In response to detecting input 830 (e.g., first type of input), in Figure 8I At this point, computer system 600 changes the media format setting from 12 megapixel HEIC (e.g., the initially selected default media format option) to 48 megapixel HEIC (e.g., ...). Figure 8C (Another default media format option enabled in the settings), and updated the appearance of the media format display 616 to include the text "HEIC 48" indicating the updated media format settings. Figure 8I At this location, computer system 600 detects input 832 pointing to the position of media format display 616, i.e., another input of the first input type. In response to detecting input 832, in Figure 8J At this point, computer system 600 changes the media format setting from 48 megapixel HEIC to 12 megapixel HEIC (for example, Figure 8CThe system also updates the appearance of the media format display 616 to include the text "HEIC 12". Therefore, in response to the first type of input, the computer system 600 cycles through the set of predetermined states of the default settings user interface 810 selected for media format settings.

[0283] exist Figure 8I At this location, computer system 600 detects input 834 pointing to the position of media format display 616. Input 834 is a second type of input (e.g., a "long" press input via the touch-sensitive surface of display 608, as described with respect to input 820). In response to detecting input 834 (e.g., a second type of input), in Figure 8K At this point, computer system 600 displays a media format menu 836. In some embodiments, computer system 600 displays (e.g., an animated display) a media format menu 836 that expands from the night mode display 816. Figure 8F As illustrated, when the media format menu 836 is displayed, the computer system 600 stops displaying other user interface elements, such as the flash power indicator 614 and the night mode power indicator 816, in the area of ​​the media format menu 836. The media format menu 836 includes optional options corresponding to 12-megapixel HEIC (e.g., “HEIC 12”), 48-megapixel HEIC (e.g., “HEIC 48”), 12-megapixel raw decoded (e.g., “RAW 12”), and 48-megapixel raw decoded (e.g., “RAW 48”). In some embodiments, the computer system 600 stops displaying the media format menu 836 after a threshold time period if no input is detected on the media format menu 836 and / or after input is detected at a location other than the location of the media format menu 836, as described with respect to the exposure slider 824.

[0284] exist Figure 8K At the location where the media format menu 836 is displayed, the computer system 600 detects input 838 on the media format menu 836. In some implementations, input 838 is a continuation of input 834 and / or an input different from input 834, such as regarding Figure 8F As described in input 826. Figure 8K As illustrated, input 838 points to the location of the optional option in the media format menu 836 corresponding to 48-megapixel raw decoding (e.g., "RAW 48"). In response to detecting input 838, in Figure 8LAt this point, computer system 600 changes the media format setting to 48 megapixel raw decoding. Therefore, in response to input 834, computer system 600 provides media format options (e.g., such as...) via media format menu 836, which are not available for selection via input of the first input type. Figures 8H to 8J exemplified). like Figure 8L As illustrated, after releasing input 838, computer system 600 stops displaying media format menu 836 and updates the appearance of media format capability display 616 to include the text "RAW 48".

[0285] exist Figure 8L At this location, computer system 600 detects input 840 pointing to the position of depth indicator 630, which is an input of the first input type. In response to detecting input 840 (e.g., first type input), in Figure 8M At this point, computer system 600 displays a camera preview 612 with simulated depth-of-field effects, enabling media capture with simulated depth-of-field effects (e.g., as shown in the image). Figures 6A to 6U As described. Figure 8M As illustrated, using a simulated aperture setting of f / 2.8 to apply a simulated depth-of-field effect makes the person and cactus appear sharp while the background is blurred. The computer system 600 updates the appearance of the depth indicator 630, for example, by changing the color, shadows, and / or visual emphasis of the depth indicator 630 (e.g., in...). Figure 8M (The image shows areas in the middle and other places shaded to indicate that depth indicator 630 is selected, and displays aperture setting indicator 648 indicating that the current simulated aperture is set to f / 2.8.) In some embodiments, in response to detecting additional input of the first input type, computer system 600 switches between displaying a camera preview 612 without simulated depth-of-field effect and displaying a camera preview with simulated depth-of-field effect applied using simulated aperture setting f / 2.8.

[0286] exist Figure 8M At this location, computer system 600 detects input 842 pointing to the position of depth indicator 630, which is a second type of input. In response to detecting input 842 (e.g., a second type of input), in Figure 8N At this location, computer system 600 displays an aperture setting slider 644 including an aperture setting indicator 646 (e.g., as shown in the image). Figure 6G As described. Figure 8N As illustrated, when the aperture setting slider 644 is displayed, the computer system 600 detects input 844 along the aperture setting slider 644 (e.g., dragging and / or gesture input via the touch-sensitive surface of the display 608) (e.g., as per the description of the aperture setting slider 644). Figures 6H to 6I As described. Figure 8OAs illustrated, in response to the detection of input 844, computer system 600 changes the depth capture setting to an analog aperture setting of f / 1.4 so that computer system 600 displays a camera preview 612 in which the cactus is sharp and the person and background are blurred.

[0287] exist Figure 8O At this location, computer system 600 detects input 846 pointing to the position of depth indicator 630, which is an input of the first input type. In response to detecting input 846 (e.g., first type input), in Figure 8P At this point, computer system 600 stops displaying camera preview 612 with simulated depth-of-field effect and updates the appearance of depth indicator 630, for example, by changing the color, shadow, and / or visual emphasis of depth indicator 630 (e.g., restoring the appearance of depth indicator 630). Figure 8M The described appearance changes indicate that the depth indicator 630 has been deselected.

[0288] exist Figure 8P At this location, computer system 600 detects input 848 pointing to the position of depth indicator 630, which is an input of the first input type. In response to detecting input 848 (e.g., the first type of input), in Figure 8Q At this point, computer system 600 displays a camera preview 612 with simulated depth-of-field effects, and re-enables media capture with simulated depth-of-field effects. (Example:) Figure 8Q As illustrated, a simulated depth-of-field effect is applied using a simulated aperture setting of f / 1.4, with the aperture value selected via the aperture setting slider 644 before deselecting the depth indicator 630. Therefore, in response to detecting additional input of the first input type, the computer system 600 switches between displaying a camera preview 612 without a simulated depth-of-field effect and displaying a camera preview with a simulated depth-of-field effect applied using a single simulated aperture setting (e.g., the default and / or last used aperture value), while input of the second input type (e.g., turning on the aperture setting slider 644) is used to apply and select any other simulated aperture settings.

[0289] exist Figure 8Q At this location, computer system 600 detects input 850 pointing to the position of flash indicator 614, which is an input of the first input type. In response to detecting input 850 (e.g., the first type of input), in Figure 8RAt this point, computer system 600 changes the camera flash setting to automatic flash mode (e.g., where computer system 600 automatically determines whether to use flash for media capture based on detected ambient light and / or the determined brightness of camera preview 612). In response to detecting input 850 (e.g., and changing the camera flash setting to automatic), computer system 600 updates the appearance of flash indicator 614, for example, by removing the line through flash indicator 614 to indicate that automatic flash mode is selected.

[0290] exist Figure 8R At this location, computer system 600 detects input 852 pointing to the position of flash indicator 614, which is a second type of input. In response to detecting input 852 (e.g., a second type of input), in Figure 8S At this point, computer system 600 displays camera flash menu 856. (For example...) Figure 8S As illustrated, when the camera flash menu 856 is displayed, the computer system 600 stops displaying other user interface elements, such as the night mode enable indicator 816, in the area of ​​the camera flash menu 856. The camera flash menu 856 includes selectable options corresponding to an automatic flash mode (“Auto”), an off state (“Off”), and an on state (“On”) (e.g., an “Always On” mode, in which the computer system 600 always uses the flash for media capture). In some embodiments, the computer system 600 stops displaying the camera flash menu 856 after a threshold time period if no input is detected on the camera flash menu 856 and / or after input is detected at a location other than the location of the camera flash menu 856.

[0291] exist Figure 8S At the location where the camera flash menu 856 is displayed, the computer system 600 detects input 854 (e.g., a continuation of input 852 and / or an input different from 852, as described above regarding input 838). Figure 8S As illustrated, input 854 points to the location of the optional option corresponding to the on state. In response to detecting input 854, in Figure 8T At this point, computer system 600 changes the camera flash setting to on and updates the appearance of flash power indicator 614, for example, by visually emphasizing flash power indicator 614 (e.g., by...). Figure 8T The shaded area indicates that the camera flash is on.

[0292] Figure 9This is a flowchart illustrating a method for controlling media capture settings using a computer system according to some embodiments. Method 900 is performed at a computer system (e.g., 100, 300, 500, and / or 600) that communicates with display generation components (e.g., 608) (e.g., display controller; touch-sensitive display system; display (e.g., integrated and / or connected), 3D display, transparent display, projector, and / or head-up display), cameras (e.g., 604A, 604B, 604C, and / or 604D) (in some embodiments, the computer system includes one or more cameras, such as rear-view (e.g., user-facing) cameras and front-view (e.g., environment-facing) cameras and / or multiple front-view cameras (e.g., with different lenses, such as standard cameras, telephoto cameras, and / or wide-angle cameras)) (in some embodiments, the computer system includes one or more depth sensors). Some operations in method 900 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0293] As described below, Method 900 provides an intuitive way to control media capture settings. This method reduces the cognitive burden on users controlling media capture settings, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to control media capture settings faster and more efficiently saves power and increases the time interval between battery charging sessions.

[0294] A computer system (e.g., 600) displays (902) a camera user interface (e.g., 610) via a display generation component (e.g., 608) (e.g., a camera / capture UI including at least a portion of the camera's field of view; in some embodiments, the camera UI includes one or more zoom control indications, one or more capture mode indications (e.g., for standard photo capture, standard video capture, panoramic photo capture, portrait mode photo capture and / or high frame rate video capture), one or more camera selection indications (e.g., for switching between one or more lenses and / or front and rear cameras), one or more flash control indications, one or more... The camera user interface includes viewfinder elements, one or more captured media display representations (e.g., photo slots and / or display representations for accessing the photo library) and / or one or more capture display representations), wherein the camera user interface includes optional user interface objects (e.g., 614, 616, 630 and / or 816) corresponding to the current state of the first media capture setting, wherein the current state of the first media capture setting is a first state, and includes multiple states of the first state that can be used for the first media capture setting (e.g., status indicators / display representations of camera settings, such as depth capture / aperture setting, zoom setting, flash setting, night mode / exposure setting and / or media codec setting).

[0295] The computer system detects (904) (in some embodiments, via the touch-sensitive and / or pressure-sensitive surfaces of the display) a first user input (e.g., 818, 820, 828, 830, 832, 834, 840, 842, 846, 848, 850 and / or 852) corresponding to (e.g., selection; e.g., receiving at and / or near the location of the optional user interface object on the touch-sensitive / pressure-sensitive surface of the display) of the optional user interface object (e.g., 614, 616, 630 and / or 816).

[0296] In response to detecting first user input (906) and determining that the first user input is a first type of input (e.g., 818, 828, 830, 832, 840, 846, 848 and / or 850) (e.g., a button tap or rapid activation, and / or input lasting less than a predetermined threshold), the computer system switches the first media capture setting (908) to a second state among a plurality of states that is different from the first state (e.g., as...). Figure 8E , Figure 8H , Figure 8I , Figure 8J , Figure 8M , Figure 8P , Figure 8Q and Figure 8R (as illustrated) (e.g., switching the corresponding media capture setting from a first state to a second state) (e.g., switching between capture settings / options; in some embodiments, switching the setting from "off" to "on" and vice versa; in some embodiments, switching between different "on" settings / options (e.g., from f / 1.8 to f / 2.4 and / or from RAW to HEIC)).

[0297] In response to detecting a first user input and determining that the first user input is a second type of input (e.g., 820, 834, 842, and / or 852) (in some embodiments, a relatively long input, such as a long press and / or a click and hold input), the computer system displays (910) an optional user interface object (e.g., 644, 824, 836, and / or 856) corresponding to a third state among a plurality of states of the first media capture setting (e.g., an extended option display of the media capture setting, such as a menu and / or slider; in some embodiments, multiple options; in some embodiments, multiple additional optional options), wherein the optional user interface object corresponding to the third state, when selected (e.g., via 826, 838, 844, and / or 854) (e.g., via user input pointing to the optional user interface object, such as a tap gesture or an air gesture), switches the first media capture setting to the third state (e.g., as shown in the image). Figure 8G , Figure 8L , Figure 8O and Figure 8T (As illustrated) (e.g., from a current state such as a first state) (in some embodiments, in response to detecting input selecting an optional user interface object, the computer system switches the first media capture setting to a third state), where the third state differs from both the first and second states. Using a corresponding optional user interface object to switch settings in response to the first type of input and to provide additional options for setting in response to the second type of input provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps the user synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings. This enhances system operability and makes the user-system interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when interacting with the operating system / system), which in turn reduces power consumption and extends system battery life by enabling users to use the system more quickly and efficiently.

[0298] In some embodiments, the camera user interface includes (in some embodiments, simultaneously included with the optional user interface object) a second optional user interface object (e.g., 614, 616, 630, and / or 816), which corresponds to the current state of a second media capture setting different from the first media capture setting (e.g., another status indicator / power-up representation of camera settings, such as depth capture / aperture setting, zoom setting, flash setting, night mode / exposure setting, and / or media codec setting), wherein the current state of the second media capture setting is the first state. In some embodiments, the computer system detects a second user input (e.g., 818, 820, 828, 830, 832, 834, 840, 842, 846, 848, 850, and / or 852) corresponding to (e.g., selection) of the second optional user interface object.

[0299] In some implementations, in response to detecting a second user input and determining that the second user input is a first type of input (e.g., 818, 828, 830, 832, 840, 846, 848, and / or 850) (e.g., a button tap or rapid activation, and / or input lasting less than a predetermined threshold), the second media capture setting is switched to a second state different from the first state (e.g., as...). Figure 8E , Figure 8H , Figure 8I , Figure 8J , Figure 8M , Figure 8P , Figure 8Q and Figure 8R(as illustrated) (e.g., switching the corresponding media capture setting from a first state to a second state) (e.g., switching between capture settings / options; in some embodiments, switching the setting from "off" to "on" and vice versa; in some embodiments, switching between different "on" settings / options (e.g., from f / 1.8 to f / 2.4 and / or from RAW to HEIC)). In some implementations, in response to detecting a second user input and determining that the second user input is a second type of input (e.g., 820, 834, 842, and / or 852) (in some implementations, a relatively long input, such as a long press and / or a click and hold input), the computer system displays an optional user interface object (e.g., 644, 824, 836, and / or 856) corresponding to a third state of the second media capture setting (e.g., an extended option display of the media capture setting, such as a menu and / or slider; in some implementations, multiple options; in some implementations, multiple additional optional options), wherein the optional user interface object corresponding to the third state switches the second media capture setting to the third state (e.g., as shown in the image) when selected (e.g., via 826, 838, 844, and / or 854) (e.g., via user input pointing to the optional user interface object, such as a tap gesture or an air gesture). Figure 8G , Figure 8L , Figure 8O and Figure 8T (As illustrated) (e.g., from a current state such as a first state; in some implementations, the computer system switches a second media capture setting to a third state in response to detecting input selecting an optional user interface object), wherein the third state is different from both the first and second states. Having multiple different optional user interface objects that switch settings in response to a first type of input and provide additional options for setting in response to a second type of input provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps the user synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings. This enhances system operability and makes the user-system interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when interacting with the operating system / system), which in turn reduces power consumption and extends system battery life by enabling users to use the system more quickly and efficiently.

[0300] In some implementations, switching the first media capture setting to the corresponding state (e.g., in response to a first type of input or in response to a selection of an optional user interface object corresponding to a third state) is performed while displaying (e.g., maintaining the display) the camera user interface (e.g., 610) (e.g., changing the corresponding media capture setting while maintaining the display of the rest of the camera UI). Providing an indication that allows the user to switch media capture settings while the camera user interface is still displayed also helps the user synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities with unintended media capture settings. This enhances system operability and makes the user-system interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when interacting with the operating system / system), which in turn reduces power consumption and extends system battery life by enabling users to use the system more quickly and efficiently.

[0301] In some implementations, the camera user interface (e.g., 610) includes a camera preview (e.g., 612) that includes a representation of the camera's field of view (e.g., a live preview of a camera feed including a portion of the environment). Providing a power indication that allows users to switch media capture settings while still displaying a live camera preview helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended settings. This enhances system operability and makes the user-system interface more efficient.

[0302] In some embodiments, the camera user interface includes at least one optional user interface object (e.g., 624), which, when selected (e.g., via user input pointing to the optional user interface object, such as a tap gesture or air gesture; in some embodiments, in response to detecting input selecting the optional user interface object, the computer system initiates media capture (e.g., initiating the taking of a photograph and / or capturing video and storing the media in a media library)). Figure 6J , Figure 6L and Figure 6R (As illustrated) (e.g., the camera user interface is the camera capture user interface; for example, it includes a shutter / capture power indicator). Providing a power indicator that allows users to switch media capture settings while still displaying the capture user interface helps users synthesize media capture events and reduces the risk of missing or capturing fleeting media capture opportunities with unintended media capture settings. This enhances system operability and makes the user-system interface more efficient.

[0303] In some implementations, the first type of input includes selection input (e.g., 818, 828, 830, 832, 840, 846, 848, and / or 850) (e.g., tap gestures or air gestures) (e.g., received at a touch-sensitive and / or pressure-sensitive surface; in some implementations, received at a touch-sensitive and / or pressure-sensitive surface of the display), wherein the start and end of the selection input (e.g., the lifting of a contact detected as part of the start of a tap gesture) occur within a first threshold time period (e.g., a rapid tap; e.g., the initial application of pressure, followed by an increase in pressure within a predetermined time limit (e.g., within 0.05 seconds, 0.1 seconds, and / or 0.5 seconds)). Using tap input to switch media capture settings between at least two states helps users synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities in unintended settings, for example, because users can switch between settings using rapid input.

[0304] In some implementations, the second type of input includes press input (e.g., 820, 834, 842, and / or 852), (e.g., received at a touch-sensitive surface and / or a pressure-sensitive surface; in some implementations, received at a touch-sensitive surface and / or a pressure-sensitive surface of the display) wherein the initial touch of the press input is maintained for at least a second threshold time period (e.g., a long press / hold press; e.g., the initial application of pressure is maintained for at least a predetermined amount of time (e.g., held for more than 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, and / or 2 seconds) before an increase in pressure is detected). Using long press input to view additional setting options for media capture settings provides the user with improved media capture control options without cluttering the media capture user interface with additional display controls. This also helps the user synthesize media capture events and reduces the risk of missing or capturing momentary media capture opportunities with unintended media capture settings, for example, while figuring out ...

Claims

1. A method, the method comprising: At the computer system that communicates with the display generation components, one or more cameras, and one or more sensors: When displaying a camera preview via the display generation component, information about the physical environment corresponding to the field of view of the one or more cameras is obtained based on information from one or more sensors of the computer system; as well as In response to obtaining the information about the physical environment corresponding to the field of view of the one or more cameras: Based on information determined by the computer system regarding the physical environment corresponding to the field of view of the one or more cameras, a depth capture criterion is met, and a first optional user interface object is displayed via the display generation component. When the first optional user interface object is selected, a process for performing a first depth-based media capture function is initiated. as well as If the depth capture criteria are not met, the first optional user interface object is not displayed.

2. The method of claim 1, wherein the first optional user interface object indicates a capture state of depth information about the physical environment corresponding to the field of view of the one or more cameras.

3. The method of claim 2, wherein the depth information is based on information from one or more depth sensors among the one or more sensors.

4. The method according to any one of claims 2 to 3, wherein the depth information is based on a comparison of first image data obtained from a first camera among the one or more cameras with second image data obtained from a second camera among the one or more cameras.

5. The method according to any one of claims 1 to 3, further comprising: Detect input that selects the first optional user interface object; as well as In response to detecting the input that selects the first optional user interface object, a process for performing the first depth-based media capture function is initiated, wherein the process for performing the first depth-based media capture function includes switching between an enabled state and a disabled state to simulate a depth-of-field capture mode.

6. The method according to claim 5, further comprising: In response to detecting the input that selects the first optional user interface object: Based on the determination that the simulated depth-of-field capture mode is in the enabled state, and based on depth information about the physical environment corresponding to the field of view of the one or more cameras, a camera preview with simulated depth-of-field effect is displayed; and If the simulated depth-of-field capture mode is determined to be in the disabled state, the camera preview without the simulated depth-of-field effect is displayed.

7. The method of claim 5, wherein initiating the process for performing the first depth-based media capture function comprises: Based on the determination that the input for selecting the first optional user interface object is a first type of input, the simulated depth-of-field capture mode is switched between the enabled state and the disabled state; as well as Based on the determination that the input selecting the first optional user interface object is a second type of input different from the first type, a second optional user interface object is displayed via the display generation component. When the second optional user interface object is selected, it controls the settings of the simulated depth-of-field capture mode.

8. The method according to any one of claims 1 to 3, further comprising: When the camera preview is displayed, at least one optional user interface object different from the first optional user interface object is displayed via the display generation component. When the at least one optional user interface object is selected, it initiates a process for performing a media capture function different from the first depth-based media capture function.

9. The method according to any one of claims 1 to 3, further comprising: When the camera preview is displayed, a third optional user interface object, different from the first optional user interface object, is displayed via the display generation component. When the third optional user interface object is selected, it initiates a process for capturing media.

10. The method according to any one of claims 1 to 3, further comprising: Capture the first media; as well as After capturing the first media, a representation of the first media is displayed via the display generation component.

11. The method of claim 10, wherein displaying the representation of the first media comprises: Based on the determination that the first optional user interface object is displayed in a first state when the capture of the first media is initiated, the representation of the first media with simulated depth-of-field effect is displayed based on depth information about the physical environment corresponding to the field of view of the one or more cameras; as well as Based on the determination that the first optional user interface object was not displayed in the first state when the capture of the first media was initiated, the representation of the first media without simulated depth-of-field effect is displayed.

12. The method according to claim 11, further comprising: When displaying the representation of the first media: A fourth optional user interface object is displayed, which, when selected, controls whether the representation of the first medium displays a simulated depth-of-field effect or a non-simulated depth-of-field effect.

13. The method of claim 10, wherein: Based on the determination that the first optional user interface object is displayed when the capture of the first media is initiated, depth information about the physical environment corresponding to the field of view of the one or more cameras can be used to display the representation of the first media with a simulated depth-of-field effect; as well as Since the first optional user interface object was not displayed when the capture of the first media was initiated, the depth information cannot be used to display the representation of the first media with the simulated depth-of-field effect.

14. The method according to any one of claims 1 to 3, further comprising: Detect the first user input pointing to the first area of ​​the camera preview; as well as In response to detecting the first user input: Based on the determination that the first area of ​​the camera preview includes a representation of a corresponding subject in the physical environment, depth information about the physical environment corresponding to the field of view of the one or more cameras is captured.

15. The method according to any one of claims 1 to 3, further comprising: Based on the determination that the portrait capture mode is enabled, depth information about the physical environment corresponding to the field of view of the one or more cameras is captured.

16. The method according to claim 15, further comprising: When the camera preview includes a representation of the corresponding subject: Based on the determination that the portrait capture mode is enabled, and based on depth information about the physical environment corresponding to the field of view of the one or more cameras, a camera preview with a simulated depth-of-field effect is displayed; and Based on the determination that the portrait capture mode is not enabled and the first optional user interface object is not enabled, the camera preview without the simulated depth-of-field effect is displayed.

17. The method according to any one of claims 1 to 3, wherein the depth capture criterion includes a criterion that is satisfied when the scaling setting is set to at least a threshold magnification.

18. The method according to any one of claims 1 to 3, wherein the depth capture criterion comprises: The first criterion is met when a representation of the corresponding subject is detected in the camera preview; as well as The second criterion is satisfied when the distance from the one or more cameras to the corresponding subject in the physical environment falls within the distance range.

19. The method according to any one of claims 1 to 3, further comprising: When the first optional user interface object is displayed, a zoom interface object is displayed, which controls the zoom level of the camera preview when selected.

20. The method according to claim 19, further comprising: Detect input of the appropriate type directed to the first optional user interface object; as well as In response to detecting the input of the corresponding type to the first optional user interface object: Stop displaying the zoomed interface object; as well as A fifth optional user interface object displays the media capture settings associated with the first optional user interface object.

21. The method according to any one of claims 1 to 3, wherein the first optional user interface object, when displayed in an enabled state, includes an indication of the current value of the depth-based media capture setting, and the method further comprises: In response to detecting a change in the current value of the depth-based media capture setting, the indication of the current value of the depth-based media capture setting is updated.

22. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a display generating component, one or more cameras, and one or more sensors, the one or more programs including instructions for performing the method according to any one of claims 1 to 21.

23. A computer system configured to communicate with a display generation component, one or more cameras, and one or more sensors, the computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 21.

24. A computer system configured to communicate with a display generation component, one or more cameras, and one or more sensors, the computer system comprising: A module for performing the method according to any one of claims 1 to 21.

25. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component, one or more cameras, and one or more sensors, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 21.

Images