Methods, electronic devices, and computer-readable storage media for capturing and recording media in multiple modes

Abstract

The present disclosure relates to devices and methods for capturing and recording media in a variety of modes. More specifically, the present disclosure relates to a device displaying a camera user interface that includes a live view from a camera. While displaying the live view from the camera: the device records media images captured by the camera while continuing to display the live view from the camera; and the device also displays a representation of a plurality of media images that were recorded while displaying the live view from the camera as frames scrolling on the display in a first direction.

Description

[0001] This application is a divisional application of Chinese Patent Application 201780053139.4, filed on August 31, 2017, entitled "Apparatus, Method and Graphical User Interface for Capturing and Recording Media in Multiple Modes"; this application is directed to divisional application 202111213035.X. Technical Field

[0002] The present invention relates generally to electronic devices having a touch-sensitive surface and one or more cameras, including but not limited to electronic devices having a touch-sensitive surface and cameras that capture and record images in multiple modes. Background Technology

[0003] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary touch-sensitive surfaces include touchpads and touchscreen displays. Such surfaces are widely used to manipulate user interface objects on the display.

[0004] For electronic devices with one or more associated cameras, the camera user interface provides a power indicator for controlling various camera functions, such as selecting image recording modes (e.g., photo, video, or panorama recording modes) and choosing when to capture a shot or when to start and stop video recording. The camera user interface typically includes a live view from the camera associated with the electronic device. The live view from the camera helps the user view the images the camera is capturing and decide whether these images should be recorded and saved as corresponding media items. The camera user interface may include a power indicator for switching between live views from different cameras. When a user wants to view media items recorded using the camera user interface, a separate media viewing interface or application is typically used.

[0005] Current methods for controlling camera functions and viewing recorded media items are cumbersome and inefficient. For example, using an input sequence to access a desired image recording function or mode results in unacceptable delays and missed opportunities to capture and record the desired image. Users are required to make decisions and perform desired actions within a very short time window, without sufficient information and assistance from the device, which places an excessive cognitive burden on the user and increases the chance of user error. Typically, performing one function using the camera user interface (e.g., taking a still photo) will prevent another function of the camera user interface from being performed simultaneously (e.g., recording video). Furthermore, these methods take longer than necessary, thus wasting energy. This latter consideration is particularly important in battery-powered devices. Summary of the Invention

[0006] Therefore, there is a need for electronic devices with faster and more efficient methods and interfaces for capturing and recording images in multiple modes. Such methods and interfaces optionally complement or replace traditional methods for capturing and recording images in multiple modes. These methods and interfaces reduce the amount, extent, and / or nature of user input and result in a more efficient human-machine interface. For battery-powered devices, such methods and interfaces can save power and increase the time between battery charges.

[0007] The disclosed device can reduce or eliminate the aforementioned defects and other problems associated with the user interface of electronic devices with touch-sensitive surfaces. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, tablet, or handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also referred to as a "touchscreen" or "touchscreen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, and a program or set of instructions stored in memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, these functions optionally include image editing, drawing, presentation, word processing, spreadsheet creation, playing games, making and receiving phone calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, note-taking, and / or digital video playback. Executable instructions for performing these functions are optionally included in a nontransitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0008] According to some embodiments, a method is performed in an electronic device having one or more processors, memory, a display, and a camera. The method includes displaying a camera user interface on the display, the camera user interface including a live view from the camera. While displaying the live view from the camera, the method includes recording media images captured by the camera while continuing to display the live view from the camera; and displaying a representation of a plurality of media images recorded while displaying the live view from the camera as frames scrolling on the display in a first direction.

[0009] According to some embodiments, an electronic device includes: a display unit configured to display a user interface and detect contact; a camera unit configured to capture media; and a processing unit coupled to the display unit and the camera unit. In some embodiments, the processing unit includes a detection unit, a recording unit, a capture unit, a scrolling unit, a replacement unit, a change unit, and a creation unit. The processing unit is configured to: display a camera user interface on the display unit, the camera user interface including a live view from the camera unit; while displaying the live view from the camera unit; record media images captured by the camera unit while continuing to display the live view from the camera unit; and display a representation of multiple media images recorded while displaying the live view from the camera unit as frames scrolling on the display unit in a first direction.

[0010] According to some embodiments, a method is performed in an electronic device having one or more processors, memory, a camera, and a display. The method includes displaying a camera user interface on the display, wherein the camera user interface includes a live view from the camera. While displaying the live view from the camera, the method further includes detecting the start of input of a first contact, the first contact activating media recording using the camera. In response to detecting the start of input of the first contact and the first contact activating media recording using the camera, the method further includes displaying a first indication that the camera is in a first media recording mode corresponding to the recording of a single image. While displaying the first indication that the camera is in the first media recording mode, the method includes detecting the continuation of input of the first contact. In response to the detection of continued input of the first contact, and before the detection of the input terminates, the method further includes: displaying a second indication that the camera is in a second media recording mode, the second media recording mode corresponding to an image sequence recorded simultaneously with the continued input of the first contact, based on at least determining that the continued input satisfies a first predefined time threshold; detecting further continuation of the first contact input while displaying the second indication that the camera is in the second media recording mode; and in response to the detection of further continuation of the first contact input, and before the detection of the input terminates: displaying a third indication that the camera is in a third media recording mode corresponding to video recording, based on at least determining that the further continuation of the input satisfies a second predefined time threshold.

[0011] According to some embodiments, an electronic device includes: a display unit configured to display a user interface; a camera unit configured to capture media; and a processing unit coupled to the display unit and the camera unit. In some embodiments, the processing unit includes a detection unit, a capture unit, an acquisition unit, and a storage unit. The processing unit is configured to display the camera user interface on the display, the camera user interface including a live view from the camera unit. While displaying the live view from the camera unit, the processing unit is configured to detect the start of input of a first contact, the first contact activating media recording using the camera unit. In response to detecting the start of input of the first contact and the first contact activating media recording using the camera unit, the processing unit is configured to display a first indication that the camera unit is in a first media recording mode corresponding to the recording of a single image. While displaying the first indication that the camera unit is in the first media recording mode, the processing unit is configured to detect the continuation of input of the first contact. In response to the continued detection of the first contact input, and before the termination of the input detection, the processing unit is configured to: based on at least determining that the continued input satisfies a first predefined time threshold: be able to display a second indication that the camera unit is in a second media recording mode, the second media recording mode corresponding to an image sequence recorded simultaneously with the continued input of the first contact; while displaying the second indication that the camera unit is in the second media recording mode, detect further continuation of the first contact input; in response to the further detection of the first contact input, and before the termination of the input detection: based on at least determining that the further continuation of the input satisfies a second predefined time threshold, be able to display a third indication that the camera unit is in a third media recording mode corresponding to video recording.

[0012] According to some embodiments, a method is performed at an electronic device having a display, a touch-sensitive surface, and one or more cameras. The method includes displaying a first live view from a first camera of the electronic device in a first display area of ​​a camera user interface. When the first live view from the first camera is displayed in the first display area of ​​the camera user interface, the method includes detecting movement of a first contact on the touch-sensitive surface. In response to detecting movement of the first contact on the touch-sensitive surface, the method includes: determining, based on the movement of the first contact satisfying a first movement criterion; moving the first live view in the first display area based on the movement of the first contact on the touch-sensitive surface; displaying a second live view from a second camera of the electronic device in the first display area; and moving the second live view in the first display area based on the movement of the first contact on the touch-sensitive surface.

[0013] According to some embodiments, an electronic device includes: a display unit configured to display a user interface; a touch-sensitive surface unit configured to detect contact; one or more camera units (including a first camera unit and a second camera unit) configured to capture media; and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the camera units. In some embodiments, the processing unit includes a detection unit, a movement unit, a rotation unit, a sliding unit, a recording unit, a changing unit, a switching unit, and an adjustment unit. The processing unit is configured to display a first real-time view from the first camera unit of the electronic device in a first display area of ​​the camera user interface. When the first real-time view from the first camera unit is displayed in the first display area of ​​the camera user interface, the processing unit is configured to detect movement of a first contact on the touch-sensitive surface unit. In response to detecting movement of the first contact on the touch-sensitive surface unit, the processing unit is configured to: based on a determination that the movement of the first contact satisfies a first movement criterion; move the first real-time view in the first display area according to the movement of the first contact on the touch-sensitive surface unit; display a second real-time view from the second camera unit of the electronic device in the first display area; and move the second real-time view in the first display area according to the movement of the first contact on the touch-sensitive surface unit.

[0014] According to some embodiments, a method is performed on an electronic device having a display and one or more cameras. The method includes: displaying a camera user interface for recording media images in multiple media recording modes. When displaying the camera user interface, the method includes utilizing one or more captured media from the camera. In response to the captured media, the method includes: displaying a first prompt in the camera user interface based on determination that the captured media matches a first media recording mode of the multiple media recording modes, prompting a user to take one or more actions associated with the first media recording mode; and displaying a second prompt in the camera user interface based on determination that the captured media matches a second media recording mode of the multiple media recording modes, prompting a user to take one or more actions associated with the second media recording mode, wherein the first media recording mode is different from the second media recording mode.

[0015] According to some embodiments, an electronic device includes: a display unit configured to display a user interface; one or more camera units configured to capture media; and a processing unit coupled to the display unit and the one or more camera units. In some embodiments, the processing unit includes a capture unit, a shift unit, a confirmation unit, a determination unit, a playback unit, a designation unit, an identification unit, a storage unit, a detection unit, an annotation unit, a receiving unit, a generation unit, an extraction unit, a movement unit, and an extension unit. The processing unit is configured to display a camera user interface capable of recording media images in multiple media recording modes. When displaying the camera user interface, the processing unit is configured to capture media using one or more camera units. In response to capturing media, the processing unit is configured to: based on determination that the captured media matches a first media recording mode among multiple media recording modes, display a first prompt in the camera user interface to prompt the user to take one or more actions associated with the first media recording mode; and based on determination that the captured media matches a second media recording mode among multiple media recording modes, display a second prompt in the camera user interface to prompt the user to take one or more actions associated with the second media recording mode, wherein the first media recording mode is different from the second media recording mode.

[0016] According to some embodiments, a method is performed at an electronic device having a display. The method includes: displaying a first user interface on the display for presenting a sequence of media content having a plurality of identified still images, wherein the first user interface includes: a representation of a first still image from the media content sequence; a timeline corresponding to the media content sequence; and a plurality of image adjustment objects, wherein: the first image adjustment object has a first position on the timeline; and second image adjustment objects have a second position on the timeline different from the first position. While displaying the representation of the first still image, the method includes receiving a request to move a corresponding image adjustment object of the plurality of image adjustment objects in the timeline. In response to receiving the request to move a corresponding image adjustment object in the timeline, the method further includes: based on a determination that the corresponding image adjustment object is the first image adjustment object, moving the first image adjustment object in the timeline to a first updated position according to the request, and updating the representation of the first still image to correspond to the first updated position of the first image adjustment object on the timeline; and based on a determination that the corresponding image adjustment object is the second image adjustment object, moving the second image adjustment object along the timeline to a second updated position according to the request, and updating the representation of the second still image to correspond to the second updated position of the second image adjustment object on the timeline.

[0017] According to some embodiments, the electronic device includes: a display unit configured to display a user interface; and a processing unit coupled to the display unit. In some embodiments, the processing unit includes a receiving unit, a moving unit, a replacing unit, an updating unit, a recording unit, an identification unit, an adjustment unit, a detection unit, a holding unit, and a capture unit. The processing unit is configured to display a first user interface on the display unit for presenting a media content sequence having multiple identified still images, wherein the first user interface includes: a representation of a first still image from the media content sequence; a timeline corresponding to the media content sequence; and multiple image adjustment objects, wherein: the first image adjustment object has a first position on the timeline; and a second image adjustment object has a second position on the timeline different from the first position. When displaying the representation of the first still image, the processing unit is configured to receive requests to move corresponding image adjustment objects of the multiple image adjustment objects in the timeline. In response to receiving a request to move a corresponding image adjustment object in the timeline, the processing unit is configured to: based on the determination that the corresponding image adjustment object is a first image adjustment object, move the first image adjustment object in the timeline to a first updated position according to the request, and update the representation of a first still image to correspond to the first updated position of the first image adjustment object in the timeline; and based on the determination that the corresponding image adjustment object is a second image adjustment object, move the second image adjustment object along the timeline to a second updated position according to the request, and update the representation of a second still image to correspond to the second updated position of the second image adjustment object in the timeline.

[0018] According to some embodiments, an electronic device includes a display, a touch-sensitive surface, a memory, one or more processors, one or more programs, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface; one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for performing or causing to perform any of the methods described herein. According to some embodiments, a computer-readable storage medium stores instructions therein that, when executed by an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, cause the device to perform or cause to be performed any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, a memory, and one or more processors executing one or more programs stored in the memory includes one or more elements displayed in any of the methods described herein, which are updated in response to input as described in any of the methods described herein. According to some embodiments, an electronic device includes: a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface; and means for performing or causing to perform any of the methods described herein. According to some embodiments, an information processing device for an electronic device having a display and a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, includes means for performing operations of any of the methods described herein or causing operations of any of the methods described herein to be performed.

[0019] Therefore, faster and more efficient methods and interfaces are needed for electronic devices with displays, touch-sensitive surfaces, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface to capture and record images in multiple modes, thereby improving the performance, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace traditional methods for capturing and recording images in multiple modes. Attached Figure Description

[0020] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0021] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.

[0022] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0023] Figure 2 A portable multi-functional device with a touchscreen is shown according to some embodiments.

[0024] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0025] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some implementations is shown.

[0026] Figure 4B An example user interface is shown for a multi-functional device having a touch-sensitive surface separate from the display, according to some implementation schemes.

[0027] Figures 5A to 5AG An example user interface for simultaneously capturing and editing digital media, based on some implementation schemes, is shown.

[0028] Figures 6A to 6S An example user interface for capturing digital media in different imaging modes is shown according to some implementation schemes.

[0029] Figures 7A to 7R An example user interface for automatically detecting and prompting user actions regarding image recording modes, according to some implementation schemes, is shown.

[0030] Figures 8A to 8AC An example user interface for switching between camera live views during image capture is shown, according to some implementation schemes.

[0031] Figures 9A to 9S An example user interface for adjusting selected images and video clips from video recordings, according to some implementation schemes, is shown.

[0032] Figures 10A to 10E This is a flowchart illustrating a method for simultaneously capturing and editing digital media according to some implementation schemes.

[0033] Figure 11 It is a functional block diagram of an electronic device according to some implementation schemes.

[0034] Figures 12A to 12D This is a flowchart illustrating methods for capturing digital media in different imaging modes according to some implementation schemes.

[0035] Figure 13 It is a functional block diagram of an electronic device according to some implementation schemes.

[0036] Figures 14A to 14F This is a flowchart illustrating a method for switching between live views of a camera during image capture, according to some implementation schemes.

[0037] Figure 15 It is a functional block diagram of an electronic device according to some implementation schemes.

[0038] Figures 16A to 16H This is a flowchart illustrating a method for automatically detecting and prompting user actions regarding image recording modes according to some implementation schemes.

[0039] Figure 17 It is a functional block diagram of an electronic device according to some implementation schemes.

[0040] Figures 18A to 18D This is a flowchart illustrating a method for adjusting selected images and video clips from video recording according to some implementation schemes.

[0041] Figure 19 It is a functional block diagram of an electronic device according to some implementation schemes. Detailed Implementation

[0042] This invention provides a camera user interface that includes a live view from the camera. When media capture begins, the image captured by the camera and the live view are simultaneously displayed as scrolling frames on a display, for example, forming a live frame stream. The frame stream provides the user with feedback about the image that has just been captured. The user can interact with the image represented in the frame stream to mark the image as a media item to be persistently stored in different media recording modes, to view the media item, and to otherwise manipulate the image (e.g., delete a blurred image) without interrupting the ongoing media capture and / or recording.

[0043] When input pointing to the live view in the camera user interface is detected, the duration of the input determines which media recording mode will be activated and what type of media item will be recorded when the input ends. Without specific user commands or other input, the device automatically switches from one recording mode to the next, making the recording experience simpler and more efficient.

[0044] When the device has multiple associated cameras, switching between live views from the multiple cameras is performed in response to movement of contact on a live view display object from one or more of the cameras. Movement of contact on the live view display object causes an adjustment to the aspect ratio of the currently displayed live view. In some implementations, switching between live views from different cameras and adjusting the aspect ratio of the currently displayed live view is performed without interrupting ongoing media capture and / or recording.

[0045] The electronic device automatically prompts the user for actions associated with one or more image recording patterns detected by the device's motion-based analysis and images captured by the camera. The images captured by the camera are automatically processed and arranged into one or more media summaries, where media items are recorded in different image recording modes, some of which are automatically generated by the device based on motion-based analysis and image features of the captured images, without user intervention.

[0046] When assisting users in selecting representative still images for a media content sequence, the user interface switches between a grid view and a timeline view. In the timeline view, media adjustment objects (such as image and video adjustment objects) are moved along the timeline to update the selected still images representing the media content sequence. The timeline view provides users with an intuitive and efficient way to sift through large amounts of content to identify suitable still images to represent the content sequence, while the grid view provides a direct and complete overview of the currently identified still images.

[0047] under, Figures 1A to 1B , Figure 2 and Figure 3 Provide a description of the example device. Figures 4A to 4B An example user interface is shown on a portable multi-functional device according to some implementation schemes. Figures 5A to 5AG , Figures 6A to 6S , Figures 7A to 7R , Figures 8A to 8AC and Figures 9A to 9S Example user interfaces for capturing, recording, and viewing images are shown according to some implementation schemes. Figures 10A to 10E This is a flowchart illustrating a method for simultaneously capturing and editing digital media according to some implementation schemes. Figures 12A to 12D This is a flowchart illustrating methods for capturing digital media in different imaging modes according to some implementation schemes. Figures 14A to 14F This is a flowchart illustrating a method for switching between live views of a camera during image capture, according to some implementation schemes. Figures 16A to 16H This is a flowchart illustrating a method for automatically detecting and prompting user actions regarding image recording modes according to some implementation schemes. Figures 18A to 18D This is a flowchart illustrating a method for adjusting selected images and videos from video clips according to some implementation schemes. Figures 5A to 5AG , Figures 6A to 6S , Figures 7A to 7R , Figures 8A to 8AC and Figures 9A to 9S The user interface in the document is used to show Figures 10A to 10E , Figures 12A to 12D , Figures 14A to 14F , Figures 16A to 16H And the processes from 18A to 18D. Figure 11 , Figure 13 , Figure 15 , Figure 17 and Figure 19 Is execution Figures 10A to 10E , Figures 12A to 12D , Figures 14A to 14F , Figures 16A to 16H and Figures 18A to 18D Functional block diagram of the electronic device described in the paper.

[0048] Exemplary device

[0049] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to those skilled in the art that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the various aspects of the embodiments.

[0050] It will also be understood that, although in some cases the terms “first,” “second,” etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact, without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0051] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “includes”, “including”, “comprises”, and / or “comprising” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0052] As used herein, depending on the context, the term “if” is optionally interpreted as meaning “when” or “upon” or “in response to determination” or “in response to detection”. Similarly, depending on the context, the phrase “if determination…” or “if detection [the stated condition or event]” is optionally interpreted as meaning “in response to determination…” or “in response to detection [the stated condition or event]” or “in response to detection [the stated condition or event]”.

[0053] This document describes implementations of electronic devices, user interfaces for such devices, and related processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functionalities such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, the iPhone from Apple Inc. (Cupertino, California). ® iPod Touch ® and iPad ® Device. Optionally, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0054] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0055] The device typically supports a variety of applications, such as one or more of the following: note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0056] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0057] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a “touchscreen” for convenience, and sometimes simply as a touch-sensitive display. Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0058] As used in this specification and claims, the term "haptic output" refers to the physical displacement of a device relative to a previous position of the device, the physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or the displacement of a component relative to the center of mass of the device, detected by the user using the user's tactile sense. For example, when a device or a component of a device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement has not moved. For example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or sensed by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or common) user.

[0059] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.

[0060] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100, such as one or more CPUs 120 and peripheral interface 118, is optionally controlled by memory controller 122.

[0061] Peripheral interface 118 can be used to couple the device's input and output peripherals to memory 102 and one or more CPUs 120. The one or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0062] In some implementations, the peripheral interface 118, one or more CPUs 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In other implementations, they are optionally implemented on separate chips.

[0063] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via these electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks and other devices, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). This wireless communication may optionally use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolved Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE...). 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS)), or any other suitable communication protocol including those not yet developed as of the date of this document submission.

[0064] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset with both output (e.g., a mono or binaural headphone) and input (e.g., a microphone).

[0065] I / O subsystem 106 couples input / output peripherals on device 100, such as touch-sensitive display system 112 and other input or control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / send electrical signals to the other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, stylus, and / or pointing device such as mouse. One or more buttons (e.g., Figure 2 Optionally, 208 of the above includes an up / down button for volume control of the speaker 111 and / or microphone 113. One or more buttons optionally include a push-down button (e.g., Figure 2 (206 in the middle).

[0066] The touch-sensitive display system 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. Visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output corresponds to a user interface object. As used herein, the term "graphical user interface object" is a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to input directed to a graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0067] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or tactile contact. The touch-sensitive display system 112 and the display controller 156 (along with any associated modules and / or instruction sets in memory 102) detect contact on the touch-sensitive display system 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In one exemplary embodiment, the contact point between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0068] The touch-sensitive display system 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed thereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In one exemplary embodiment, projected mutual capacitance sensing technology, such as that used in the iPhone from Apple Inc. (Cupertino, California), is used. ® iPod Touch ® and iPad ® The technology discovered in [the text].

[0069] The touch-sensitive display system 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi, or greater). Users optionally use any suitable object or accessory such as a stylus, finger, etc., to interact with the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.

[0070] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0071] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0072] The device 100 may optionally also include one or more optical sensors 164. Figure 1A An optical sensor coupled to an optical sensor controller 158 in I / O subsystem 106 is shown. One or more optical sensors 164 optionally include charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) phototransistors. The one or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), the one or more optical sensors 164 optionally capture still images and / or video. In some embodiments, the optical sensor is located on the rear of device 100, opposite to the touch-sensitive display system 112 on the front of the device, enabling the touchscreen to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to acquire images of the user (e.g., for selfies, for video conferencing while the user views other video conference participants on the touchscreen, etc.).

[0073] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor coupled to a strength sensor controller 159 in I / O subsystem 106 is shown. One or more contact strength sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact strength sensors 165 receive contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touch-sensitive display system 112 located on the front of device 100.

[0074] The device 100 optionally also includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 is coupled to an input controller 160 in an I / O subsystem 106. In some embodiments, the proximity sensor turns off and disables the touch-sensitive display system 112 when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).

[0075] The device 100 may optionally also include one or more tactile output generators 167. Figure 1A A haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymerizers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device). The haptic output generator 167 receives haptic feedback generation instructions from haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the rear of the device 100 opposite to the touch-sensitive display system 112 located on the front of the device 100.

[0076] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral interface 118 is shown. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from the one or more accelerometers. The device 100 may optionally include, in addition to the accelerometer 168, a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for acquiring information about the location and orientation (e.g., portrait or landscape) of the device 100.

[0077] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a touch / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application program (or instruction set) 136. Furthermore, in some embodiments, memory 102 stores device / global internal state 157, as shown in Figures 1A and 1B. Figure 3 As shown in the diagram. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, which indicates what applications, views or other information occupy the various areas of the touch-sensitive display system 112; sensor state, including information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information regarding the device's position and / or orientation.

[0078] Operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0079] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is compatible with some iPhones from Apple Inc. (Cupertino, California). ® iPod Touch ® The external port is the same as or similar to and / or compatible with the 30-pin connector used in iPod devices. In some implementations, the external port is compatible with some iPhones from Apple Inc. (Cupertino, California). ® iPod Touch ® The same or similar and / or compatible Lightning connector used in iPod devices.

[0080] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., via a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the intensity of the contact (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations can optionally be applied to single-point contact (e.g., single-finger contact or stylus contact) or simultaneous multi-point contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0081] The touch / motion module 130 optionally detects user gesture input. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the icon location). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event. Similarly, stylus taps, swipes, drags, and other gestures are optionally detected by detecting specific contact patterns of the stylus.

[0082] In some implementations, detecting a finger tap depends on detecting the duration between a finger press event and a finger release event, but is independent of the intensity of finger contact between the two events. In some implementations, a tap is detected based on determining that the duration between the finger press event and the finger release event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of finger contact during the tap reaches a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light press or deep press intensity threshold. Therefore, a finger tap can satisfy a specific input criterion that does not require the characteristic intensity of the contact to meet a given intensity threshold to satisfy that criterion. For clarity, finger contact in a tap gesture typically needs to meet a nominal contact detection intensity threshold to detect a finger press event; below this threshold, no contact is detected. Similar analysis applies to detecting tap gestures via a stylus or other contact method. When the device is capable of detecting contact from a finger or stylus hovering above a touch-sensitive surface, the nominal contact detection strength threshold may optionally not correspond to the physical contact between the finger or stylus and the touch-sensitive surface.

[0083] The same concept applies to other types of gestures in a similar manner. For example, swipe gestures, pinch gestures, spread gestures, and / or long press gestures can be optionally detected based on criteria that are independent of the intensity of the contact involved in the gesture or do not require one or more contacts performing the gesture to reach an intensity threshold for recognition. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on the movement of two or more contacts toward each other; a spread gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of contact with less than a threshold amount of movement on a touch-sensitive surface. Therefore, the statement that a particular gesture recognition criterion does not require the contact intensity to meet a corresponding intensity threshold implies that a particular gesture recognition criterion can be met when the contact in the gesture does not reach the corresponding intensity threshold, and also when one or more contacts in the gesture reach or exceed the corresponding intensity threshold. In some implementations, tap gestures are detected based on determining that a finger press event and a finger lift event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding intensity threshold during the predefined time period. Similarly, swipe gestures are detected based on determining that the contact movement is greater than a predefined amount, even if the contact movement ends above a corresponding intensity threshold. Even in specific implementations where gesture detection is affected by the intensity of the contact performing the gesture (e.g., the device detects a long press faster when the contact intensity is above an intensity threshold, or delays the detection of a tap input when the contact intensity is even higher), the detection of these gestures does not require the contact to reach a specific intensity threshold (e.g., even if the amount of time required to recognize the gesture varies).

[0084] In some cases, contact intensity thresholds, duration thresholds, and movement thresholds are combined in various ways to create heuristic algorithms that distinguish between two or more different gestures targeting the same input element or region, allowing for a richer set of user interactions and responses from multiple different interactions with the same input element. Statements that a particular set of gesture recognition criteria does not require the contact intensity to meet a corresponding intensity threshold to satisfy a particular gesture recognition criterion do not preclude the simultaneous evaluation of other intensity-related gesture recognition criteria to identify other gestures that meet criteria when the gesture includes contact with an intensity higher than the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture (which does not require the contact intensity to meet a corresponding intensity threshold to satisfy the first gesture recognition criterion) competes with a second gesture recognition criterion for a second gesture (which depends on the contact reaching the corresponding intensity threshold). In such competition, if the second gesture recognition criterion for the second gesture is satisfied first, the gesture is optionally not recognized as satisfying the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount of movement, a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves a predefined amount of motion before reaching the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in such cases, the first gesture recognition criterion for the first gesture still does not require the contact intensity to meet the corresponding intensity threshold to satisfy the first gesture recognition criterion, because if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with a contact intensity that does not increase to above the corresponding intensity threshold), the gesture will be recognized as a swipe gesture by the first gesture recognition criterion. Therefore, a specific gesture recognition criterion that does not require the contact intensity to meet the corresponding intensity threshold to satisfy a specific gesture recognition criterion will (A) in some cases ignore the contact intensity relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases fail to satisfy the specific gesture recognition criterion (e.g., for a long press gesture) if a set of competing intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognize the input as corresponding to an intensity-related gesture before the specific gesture recognition criterion recognizes the gesture corresponding to the input, in this sense, still depend on the contact intensity relative to the intensity threshold (e.g., for a long press gesture that competes with a deep press gesture for recognition).

[0085] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, and non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0086] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like to specify the graphic to be displayed, and, if necessary, also receives coordinate data and other graphic attribute data, and then generates screen image data for output to the display controller 156.

[0087] The haptic feedback module 133 includes various software components for generating instructions that are used by the haptic output generator 167 to produce haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0088] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications, such as Contacts 137, Email 140, IM 141, Browser 147, and any other application that requires text input.

[0089] GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to a phone 138 for location-based dialing; to a camera 143 as image / video metadata; and to applications that provide location-based services such as weather desktop apps, local yellow pages desktop apps, and map / navigation desktop apps).

[0090] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0091] • Contacts module 137 (sometimes called address book or contact list);

[0092] • Telephone module 138;

[0093] • Video conferencing module 139;

[0094] • Email client module 140;

[0095] • Instant Messaging (IM) module 141;

[0096] • Fitness support module 142;

[0097] • Camera module 143 for still images and / or video images;

[0098] • Image management module 144;

[0099] • Browser module 147;

[0100] • Calendar module 148;

[0101] • Desktop app module 149, which optionally includes one or more of the following: weather app 149-1, stock market app 149-2, calculator app 149-3, alarm clock app 149-4, dictionary app 149-5, and other desktop apps obtained by the user, and user-created desktop apps 149-6.

[0102] • Desktop Mini Program Creator Module 150 for creating user-created desktop mini programs 149-6;

[0103] • Search module 151;

[0104] • Video and music player module 152, optionally composed of a video player module and a music player module;

[0105] •Notepad module 153;

[0106] • Map module 154; and / or

[0107] • Online video module 155.

[0108] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0109] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 includes executable instructions for managing an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers and / or email addresses to initiate and / or facilitate communication via telephone 138, video conferencing 139, email 140, or instant messaging 141; and so on.

[0110] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, telephone module 138 includes executable instructions for performing the following operations: inputting a character sequence corresponding to a telephone number, accessing one or more telephone numbers in the address book 137, modifying an input telephone number, dialing a corresponding telephone number, initiating a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communication optionally employs any of a variety of communication standards, protocols, and technologies.

[0111] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0112] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0113] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for performing the following operations: inputting a character sequence corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Services (APNs), or IMPS for internet-based instant messaging), receiving an instant message, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0114] Incorporating RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, fitness support module 142 includes executable instructions for creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (in sports equipment and smartwatches); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0115] In conjunction with the touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0116] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, displaying (e.g., in a digital slideshow or photo album), and storing still images and / or video images.

[0117] Incorporating RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) according to user instructions.

[0118] Incorporating RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0119] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet module 149 is optionally a micro-application downloaded and used by a user (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) or a user-created micro-application (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop applet includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop applet).

[0120] Combining RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet creator module 150 includes executable instructions for creating desktop applets (e.g., transferring user-specified portions of web pages to desktop applets).

[0121] In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0122] In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, the video and music player module 152 includes executable instructions allowing users to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on the touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0123] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0124] Combining RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and map-related data (e.g., driving routes; data on shops and other points of interest at or near specific locations; and other location-based data) according to user instructions.

[0125] In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes executable instructions that allow users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen 112 or on an external display connected wirelessly or via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos.

[0126] Each module and application identified above corresponds to a set of executable instructions for performing one or more of the functions described above and the methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures described above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0127] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on device 100 can be optionally reduced.

[0128] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, home menu, or root menu. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0129] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A (in the middle) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0130] Event classifier 170 receives event information and determines the application 136-1 to which the event information should be delivered and the application view 191 of application 136-1. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display system 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0131] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0132] Event monitor 171 receives event information from peripheral interface 118. The event information includes information about sub-events, such as user touches on touch-sensitive display system 112 as part of a multi-touch gesture. Peripheral interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or touch-sensitive surfaces.

[0133] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other implementations, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0134] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0135] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0136] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0137] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that forms an event or potential event) occurs. Once the hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0138] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0139] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver module 182.

[0140] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0141] In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit (not shown) or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0142] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies events from the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0143] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device pose).

[0144] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on a displayed object for a predetermined duration, movement of the touch on the touch-sensitive display system 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0145] In some implementations, event definition 187 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on a touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0146] In some implementations, the definition of the corresponding event 187 also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0147] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0148] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0149] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some implementations, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with that tag retrieves the tag and executes a predefined procedure.

[0150] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined procedure.

[0151] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module 145. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0152] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0153] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0154] Figure 2 The following are examples of implementations with a touchscreen (e.g., Figure 1AA portable multi-functional device 100 (a touch-sensitive display system 112). The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the diagram) or one or more styluses 203 (not drawn to scale in the diagram). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0155] Device 100 optionally also includes one or more physical buttons, such as a "home" button or a menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display.

[0156] In some embodiments, device 100 includes a touchscreen display, a menu button 204, a push-button 206 for powering on / off the device and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0157] Figure 3This is a block diagram of an exemplary multifunctional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 167 and sensors 359 (e.g., optical sensors, accelerometers, proximity sensors, touch sensors, and / or similar to those mentioned above) are described. Figure 1A The contact strength sensor 165 is one or more of the contact strength sensors. The memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU 310. In some embodiments, the memory 370 stores data with the portable multifunction device 100 (…). Figure 1A The memory 370 stores programs, modules, and data structures similar to those stored in the memory 102 of the portable multifunction device 100, or subgroups thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 ( Figure 1A The memory 102 may optionally not store these modules.

[0158] Figure 3Each of the elements identified above is optionally stored in one or more of the previously mentioned memory devices. Each of the modules identified above corresponds to a set of instructions for performing the functions described above. The modules or programs identified above (i.e., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subgroup of the modules and data structures described above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0159] Now let’s turn our attention to the implementation of the user interface (“UI”) optionally implemented on the portable multifunction device 100.

[0160] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is shown. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof:

[0161] • One or more signal strength indicators 402 for one or more wireless communications (such as cellular signals and Wi-Fi signals);

[0162] • Time 404;

[0163] • Bluetooth indicator 405;

[0164] • Battery status indicator 406;

[0165] • Tray 408 with icons of commonly used applications, such as:

[0166] The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemails.

[0167] The email client module 140 has an icon 418 labeled "Mail" which optionally includes an indicator 410 for the number of unread emails.

[0168] The browser module 147 has an icon 420 labeled "Browser"; and

[0169] o Video and music player module 152 (also known as iPod (Apple Inc. trademark) module 152) with icon 422 labeled "iPod"; and

[0170] • Icons of other applications, such as:

[0171] The icon 424 of oIM module 141 marked as "message";

[0172] The calendar module 148 has an icon 426 labeled "Calendar";

[0173] The image management module 144 has an icon 428 labeled "Photo".

[0174] The icon 430 of the camera module 143 is labeled "camera";

[0175] o The icon 432 of the online video module 155, which is labeled "Online Video";

[0176] The icon 434 in the oStock Market Desktop Mini Program 149-2 is labeled "Stock Market";

[0177] The icon 436 labeled "map" in the map module 154;

[0178] The icon 438 in the oWeather desktop mini-program 149-1 is labeled "Weather";

[0179] The icon labeled "Clock" in the o alarm clock desktop mini-program 149-4;

[0180] The icon 442 of the fitness support module 142 is labeled "fitness support";

[0181] The notepad module 153 has an icon 444 labeled "Notepad"; and

[0182] o An icon 446 is used to set up applications or modules, which provides access to settings for device 100 and its various applications 136.

[0183] It should be noted that Figure 4A The icon labels shown are merely exemplary. For example, in some embodiments, the icon 422 of the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0184] Figure 4B A touch-sensitive surface 451 separate from the display 450 is shown (e.g., Figure 3 Devices (e.g., tablets or touchpads 355) Figure 3An exemplary user interface on device 300. Device 300 may also optionally include one or more contact intensity sensors (e.g., one or more sensors in sensor 357) for detecting the intensity of contact on tactile surface 451 and / or one or more tactile output generators 359 for generating tactile output for the user of device 300.

[0185] Figure 4B A touch-sensitive surface 451 separate from the display 450 is shown (e.g., Figure 3 Devices (e.g., tablets or touchpads 355) Figure 3 An exemplary user interface on device 300. Although input on a touchscreen display 112 (which combines a touch-sensitive surface and a display) is given to many subsequent embodiments, in some embodiments the device detects input on a touch-sensitive surface separate from the display, such as Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., on the display (e.g., 450) that is aligned with the main axis on the display (e.g., 451). Figure 4B The principal axis corresponding to 453 in the middle (e.g., Figure 4B (452 in the middle). According to these embodiments, the device detects contact with the touch-sensitive surface 451 at a position corresponding to the corresponding position on the display (e.g., Figure 4B (460 and 462 in the example) Figure 4B In the diagram, 460 corresponds to 468 and 462 corresponds to 470. Thus, on a touch-sensitive surface (e.g., Figure 4B 451 in the middle) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated, user input detected by the device on the touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0186] Additionally, while the examples below are primarily given with reference to finger input (e.g., finger touch, single-finger tap gesture, finger swipe gesture, etc.), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the swipe path (e.g., instead of movement of the touch). Similarly, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Likewise, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0187] As used herein, the term "focus selector" refers to an input element that indicates the current portion of a user interface with which the user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays that enable direct interaction with user interface elements on the touchscreen display. Figure 1A The touch-sensitive display system 112 or Figure 4AIn some implementations of a touchscreen, a touch detected on the touchscreen acts as a "focus selector," causing the specific user interface element to be adjusted based on the detected input when input (e.g., a press input via a touch) is detected at the location of a specific user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display. In some implementations, focus moves from one area of ​​the user interface to another without corresponding cursor movement or touch movement on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these implementations, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, it is typically a user-controlled element (or touch on the touchscreen display) that conveys the user's desired interaction with the user interface (e.g., by indicating to the device the element the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate that the user expects to activate that button (rather than other user interface elements shown on the device display).

[0188] User interface and related processes

[0189] Now turn attention to implementations of user interfaces (“UIs”) and associated processes that can be implemented on electronic devices such as portable multifunction devices 100 or 300, which have displays, touch-sensitive surfaces, and (optionally) one or more sensors for detecting the intensity of contact with the touch-sensitive surfaces.

[0190] Figures 5A to 5AG Exemplary user interfaces for simultaneously capturing and editing digital media, according to some embodiments, are shown. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 10A to 10E The process is described below. For ease of explanation, some embodiments of the implementation will be discussed with reference to the operation performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representation point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on the touch-sensitive display system 112. However, similar operation is optionally performed on a device having a display 450 and a separate touch-sensitive surface 451 in response to a contact detected on the touch-sensitive surface 451 when the user interface shown in the figures, along with the focus selector, is displayed on the display 450.

[0191] Figures 5A to 5J and Figures 50 to 5AA as well as Figures 5AE to 5AG An exemplary user interface 500 for a digital camera application is shown. The user interface includes a live view from an associated digital camera (e.g., optical sensor 164) of device 100, and the live view includes a live display of tree branch 502, cat 510, bird 522, and eagle 542. The user interface also includes a scroll 506 (e.g., overlaid on the bottom portion of the live view) of previously captured images 508, etc., at the bottom of the user interface 500. Tapping an image in scroll 506 opens the previously captured image on the display and / or marks the image (e.g., for later viewing and / or long-term storage as an image separate from other images captured during the recording session). Similarly, tapping the live display creates a timestamp, marking the captured image upon detection of contact. Figures 5K to 5N and Figures 5AB to 5AD Exemplary user interfaces 526 and 568 are shown for viewing previously captured images and video clips within a digital camera application.

[0192] Figures 5A to 5G An exemplary implementation is shown, in which the device simultaneously captures live images from an associated digital camera and displays frames corresponding to a subset of the captured images overlaid on a live display. Figure 5A In the process, the device displays a real-time image of the tree branch 502 in the user interface 500. This is in response to the detection of... Figure 5B Upon contact 504 on the camera's live view, the device begins capturing live images from the camera (e.g., for subsequent processing and long-term storage) and begins displaying frames of the captured images. In some embodiments, in addition to capturing live images, the device also responds to detection from... Figure 5B Contact 504 on the camera's live view to begin recording live images from the camera for long-term storage. (As...) Figures 5C to 5D As shown, frame 508, which displays a captured image of branch 502, appears to scroll onto the display from the lower right, followed by frame 509, which shows the subsequent captured image.

[0193] like Figures 5E to 5G As shown, when frame 512 begins to scroll onto the screen, it displays a live image from the digital camera, which reflects a larger live image on the screen. For example, when cat 510 appears in the live view from the camera, in Figure 5E It is also shown in the image that appears in frame 512. As the cat continues into frame 510... Figure 5F In the live view, similarly update frame 512 to show the cat's movement (e.g., frame 512 shows...). Figure 5E The cat's two legs and Figure 5F (The cat's three legs in the image). When frame 512 is fully displayed on the screen, as shown... Figure 5G As shown, the live image displayed in frame 512 is frozen at that point in time. For example, in Figure 5G In the live view, all four legs of the cat are visible in frame 510, but only three legs are visible in frame 512 because the image is frozen once the frame is fully migrated to the screen.

[0194] Figures 5H to 5I An exemplary implementation is shown that captures an image simultaneously by tapping a live image marker in a live view. The device via... Figure 5H Touch 518 on the real-time image detects a tap gesture and, in response, marks the image captured at that point in time. Frame 516 corresponding to the marked image is displayed in larger size in scroll 506, as... Figure 5I As shown. Metadata is created associated with the tagged image, making it possible to identify the tagged image from the captured images during subsequent processing. In some implementations, the tagged image is stored independently of the rest of the captured images.

[0195] Figures 5J to 5O An exemplary implementation is shown, in which tapping a frame activates an image viewing mode, opens the image associated with the frame on the display, and allows manual navigation of the captured image within the image viewing mode. Figure 5J As shown, the device detects a tap 524 on the marked frame 516 and, in response, opens the image associated with frame 516 in the image viewing user interface 526, as... Figure 5K As shown. Figure 5L As shown, the device then detects a swipe gesture, including a rightward movement 528 of contact 526, and in response, navigates to an earlier captured image, for example, just before the cat 510 lay down on the ground. Figure 5M As shown. Then, the device detects data including data from... Figure 5N The top of the display is touched downwards by a swipe gesture 530, which moves 532, and in response, returns to the live view in the user interface 500, as shown. Figure 5O As shown.

[0196] As shown in the advanced state of image scrolling 506 (e.g., showing a bird 522 flying into the image and landing on branch 502), the device continues to capture live images from the camera while displaying the image viewing user interface 526. Navigation between images in the image viewing user interface 526 also changes the timestamped images created when a tap 518 is detected (e.g., as shown in the image scrolling 506). Figure 5H As shown), such as by scrolling 506, frame 515 (the frame displayed before exiting the image viewing user interface 526 (e.g., the image captured before the cat 510 lays down)) replaces frame 516 (in response to detecting a tap 518). Figure 5IThis is reflected in the magnified frame.

[0197] Figures 5P to 5U An exemplary implementation is shown, in which scrolling along the bottom of the screen is independent of real-time image navigation in the live view. (See example...) Figure 5P As shown, the device detects a swipe gesture, including a movement 546 of contact 544 to the right on scroll 506. In response, the device navigates scroll 506 backward in time (e.g., moves scroll 506 to the right), as... Figure 5Q The reproduction of frames 515 and 514 in the image is shown. Figure 5R and Figure 5S As shown, automatic forward scrolling to the left is also paused during navigation, as indicated by the static nature of the frame after the swipe gesture is completed.

[0198] Then, the device detects a second swipe gesture, including a movement 550 of contact 548 moving to the left on scroll 506, such as... Figure 5S As shown. In response, the device navigates forward and scrolls accordingly, as... Figure 5T The reproduction of frames 536, 538, and 540 is shown. After being lifted away from contact 548, the rolling continues to move forward in a timely manner according to the simulated inertia, as... Figure 5U The appearance of frames 554 and 556 in the image is shown.

[0199] Figures 5V to 5Z An exemplary implementation is shown, in which contact markers are maintained on the live image to simultaneously captured video clips for subsequent processing and long-term storage. The device detects contact markers from... Figure 5V The camera detects contact 558 in the live image of the camera's live view, and in response, begins to mark the image captured when contact is detected. Frame 556, corresponding to the first image marked when contact 558 is detected, is displayed larger in scroll 506. As the device continues to detect contact 558 in the live image, subsequent frames 560 and 562 are also displayed larger in scroll 506, as... Figures 5W to 5X As shown, this indicates that the images corresponding to those frames are included in the marked video clip.

[0200] Then, the device detected Figure 5X Upon contact lift-off at frame 558, and in response, stop marking the simultaneously captured image as part of the video clip. During contact lift-off, the device also freezes the image in frame 562, as... Figure 5Y As shown, the image in frame 562—otherwise, a live image would be displayed because it hasn't fully migrated to the screen yet—shows the tail of cat 510, even though the cat has left the live image above. Figure 5Z In the next frame, 564 is displayed at normal size because the device no longer tags video clips.

[0201] Although not in Figure 5V to 5X As shown, however, in some implementations, the device analyzes images in the tagged video clips in real time and automatically identifies frames of poor quality (e.g., images blurred due to camera movement or rapid movement of objects in the live view). When a frame of poor quality is detected, the device displays a prompt (e.g., a pop-up text box scrolling above 506) asking the user if they want to remove the frame of poor quality. The user can confirm or decline the suggestion with a single tap (e.g., on a confirmation indicator) to remove the blurry frame, or ignore the prompt.

[0202] Figures 5AA to 5AE An exemplary implementation is shown, in which tapping a frame corresponding to a marked video clip opens the video clip on the display. For example... Figure 5AA As shown, the device detects a tap gesture via touch 568 on a marked frame 560, corresponding to a marked video clip that spans the images corresponding to marked frames 556 to 562. In response, the device opens the associated video clip in the video viewing user interface 568, as shown. Figure 5AB As shown. Then the device plays the video clip, as... Figures 5AC to 5AD The cat 510 in the image walks off the screen as shown. Then, the device detects that it includes... Figure 5AD The top of the display is touched downwards by a swipe gesture 570, which moves 572, and in response, returns to the live view in the user interface 500, as shown. Figure 5AE As shown.

[0203] Figures 5AF to 5AG An exemplary implementation is shown, in which tapping an unmarked frame during scrolling marks the image associated with that frame without opening the image display in the user interface. Figure 5AG In the process, the device detects a tap gesture via touch 580 on an unlabeled frame 576 within scroll 506. In response, the device labels the image associated with frame 576 and increases the frame size to indicate that the associated image has been labeled. Figure 5J The touch response of the 516 in the device differs from that of a tap gesture—the device display and... Figure 5K The image corresponding to frame 516 in the image viewer is not displayed in the image viewing user interface because frame 516 was marked before the tap gesture was detected. Figure 5AG The image corresponding to frame 576 in the image is not previously labeled.

[0204] Figures 6A to 6S Exemplary user interfaces for capturing digital media in different imaging modes are illustrated according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 12A to 12DThe process is described below. For ease of explanation, some embodiments of the implementation will be discussed with reference to the operation performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representation point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on the touch-sensitive display system 112. However, similar operation is optionally performed on a device having a display 450 and a separate touch-sensitive surface 451 in response to a contact detected on the touch-sensitive surface 451 when the user interface shown in the figures, along with the focus selector, is displayed on the display 450.

[0205] Figures 6A to 6S An exemplary user interface 600 for a camera application is shown. The user interface includes a live view from an associated digital camera (e.g., optical sensor 164) of device 100. The live view includes a live display of tree branch 602, cat 610, bird 622, and eagle 642. The accompanying drawings illustrate one embodiment where tapping the live view initiates the capture of a digital image in a first image recording mode, corresponding to a single image acquired upon detection of contact. Maintaining contact on the live view automatically changes the image acquisition mode from single-image recording mode to burst image recording mode, and then to video recording mode. Figures 6A to 6S The exemplary user interface in the example shows a visual prompt warning the user that the image recording mode is automatically switched.

[0206] Figures 6A to 6C An exemplary implementation is shown, in which the device records digital media in a single-image recording mode in response to a short tap gesture. Figure 6A In the image, the real-time view from the camera includes a real-time image of branch 602. (In response to detection...) Figure 6B Upon contact 604, the device begins recording digital media in a first image recording mode associated with the recording of a single image. The device indicates the first image recording mode by displaying a ring 606 around the location where contact 604 is detected. In response to the detection of the lifting of contact 604, as... Figure 6C As shown, the device stops recording digital media. This is because the length of contact 604 is shorter than the first threshold time length (e.g., TT). s Therefore, the device does not switch to the second image recording mode and mark a single image for long-term storage. In other words, a single still image is recorded in response to a tap gesture, and the single still image corresponds to the image captured when the tap gesture is detected (e.g., an image captured when contact 604 is initially detected or when contact 604 is lifted, or an image selected from multiple images captured when contact 604 is detected).

[0207] Figures 6D to 6FAn exemplary implementation is shown, wherein in response to a brief tap and held gesture detected by contact, the device transitions from a single image recording mode to a burst image recording mode. Figure 6D In this process, the device detects contact 608 and, in response, begins recording digital media in a first image recording mode associated with the recording of a single image. The device indicates the first image recording mode by displaying a ring 606 around the location where contact 608 is detected.

[0208] As the device continues to detect contact 608 on the real-time view exceeding a first time threshold (e.g., time threshold TT), S The device changed from a single image recording mode to Figure 6E The continuous shooting recording mode in the image. Figure 6E In the middle, the device replaces the counter mark 612 Figure 6D A ring 606 is used to indicate this change in image recording mode, with the counter marker surrounding the detected contact 608. The device also displays the number of burst images that have been collected (e.g., "8"). In response to the detection of the lifting off of contact 608, as... Figure 6F As shown, the device stops recording digital media. This is because the length of the gesture via touch 608 exceeds the duration of the first threshold amount (e.g., TT). S However, the time shorter than the second threshold (e.g., TT) S > TTs), so the device does not switch to the third image recording mode and mark the burst images for long-term storage. In other words, a sequence of still images is recorded in response to a brief tap and hold gesture, and the sequence of still images corresponds to the images captured during the detection of contact 608. In some embodiments, the burst images include a subset of images selected from all images captured during the brief tap and hold gesture.

[0209] Figures 6G to 6J An exemplary embodiment is shown, in which the device transitions from a single-image recording mode to a burst-image recording mode, and then to a video recording mode that responds to the detection of an intermediate tap and holds the gesture, terminating when the contact is lifted. Figure 6G In this process, the device detects contact 614 and, in response, begins recording digital media in a first image recording mode associated with the recording of a single image. The device indicates the first image recording mode by displaying a ring 606 around the location where contact 614 is detected.

[0210] As the device continues to detect contact 614 on the real-time view exceeding a first time threshold (e.g., time threshold TT), S The device changed from a single image recording mode to Figure 6H The continuous shooting recording mode in the image. Figure 6HIn the middle, the device replaces the counter mark 612 Figure 6G The ring 606 in the middle is used to indicate this change in image recording mode, wherein the counting marker surrounds the position of the detected contact 614.

[0211] As the device continues to detect contact 614 in the real-time view exceeding a second time threshold (e.g., time threshold TT), M >Time threshold TTs), device from Figure 6I The continuous shooting recording mode in the image has been changed to Figure 6J The video recording mode in the program. By using... Figure 6I Replacement of the 616 mark ring in the middle Figure 6H The counter mark 612 indicates this change in image recording mode. The device also displays the length of time the video has been recorded (e.g., "0:02").

[0212] In response to the removal of contact 614, such as Figure 6J As shown, the device stops recording digital media because the duration of contact 614 is shorter than the third time threshold (e.g., TT). L Because the duration of contact with 614 is longer than the second threshold time length (e.g., TT). M Therefore, the device switches to a third imaging mode and tags the video for subsequent processing and long-term storage. The tagged video includes a footnote of a cat 610 walking and lying under branch 602, both of which occur before the device switches to video recording mode, as the device continuously collects digital images from the initial detection of contact 614. In other words, video is recorded in response to the intermediate tap and hold gesture, and the video includes images captured during the time contact 614 is detected. In some embodiments, the video includes a subset of images selected from all images captured during the intermediate tap and hold gesture (e.g., the frame rate of image capture is higher than the frame rate of video recording).

[0213] Figures 6K to 6S An exemplary implementation is shown, in which the device transitions from a single-image recording mode to a burst-image recording mode, and then to a video recording mode that responds to the detection of a long tap and maintains the gesture without terminating upon removal of the contact. Figure 6K In this process, the device detects contact 618 and, in response, begins recording digital media in a first image recording mode associated with the recording of a single image. The device indicates the first image recording mode by displaying a ring 606 around the location where contact 618 is detected.

[0214] As the device continues to detect contact 618 on the real-time view exceeding a first time threshold (e.g., time threshold TT), S The device changed from a single image recording mode to Figure 6L The continuous shooting recording mode in the image. Figure 6L In the middle, the device replaces the counter mark 612 Figure 6K The ring 606 in the middle is used to indicate this change in image recording mode, wherein the counting marker surrounds the position of the detected contact 618.

[0215] As the device continues to detect contact 618 in the real-time view exceeding a second time threshold (e.g., time threshold TT), M ), equipment from Figure 6L The continuous shooting recording mode in the image has been changed to Figure 6L The video recording mode in the program. By using... Figure 6M Replacement of the 616 mark ring in the middle Figure 6L The counter mark 612 indicates this change in image recording mode. The device also displays the length of time the video has been recorded (e.g., "0:02").

[0216] The device continues to detect contact 618 in the real-time view, and in response, when the duration of contact 618 approaches a third time threshold (e.g., time threshold TT). L > TT M When this occurs, the record button 620 is displayed. This record button appears to originate from contact 618 and gradually fill the mark ring 616, as shown. Figure 6N As shown. When the duration of contact with 618 reaches the third time threshold (e.g., time threshold TT). L When the record button 620 is fully filled and the marker ring 616 is replaced, as shown... Figure 6O As shown. The device is now in the second video recording mode, as... Figure 6O The recording button 620 has been moved from below contact 618 to the bottom of the display as indicated, as shown below. Figure 6P As shown.

[0217] Because the device has switched to the second video recording mode, it continues recording digital media after detecting the removal of contact 618. Figure 6Q The record button 620 is held in the display. The device then detects a tap gesture via contact 624 on the record button 620 and, in response, stops recording digital media, as shown. Figure 6S The disappearance of the record button 620 is indicated by this. This is because the duration of contact 618 is longer than the second (and third) threshold time length (e.g., TT). M (and TT) L Therefore, the device tags the video for subsequent processing and long-term storage.

[0218] Figures 7A to 7RExemplary user interfaces for automatically detecting and prompting user actions regarding image recording modes, according to some embodiments, are shown. The device automatically detects image recording modes consistent with the media content currently captured by the device and automatically prompts user actions associated with the detected image recording modes. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 16A to 16H The process is described below. For ease of explanation, some embodiments will be discussed with reference to the operation performed on a device having a touch-sensitive screen 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representation point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on the touch-sensitive display system 112. However, similar operations are optionally performed on a device having a display 450 and a separate touch-sensitive surface 451 in response to a contact detected on the touch-sensitive surface 451 when the user interface shown in the figures, along with the focus selector, is displayed on the display 450.

[0219] Figures 7A to 7R An exemplary user interface 702 for a camera application is shown. The camera user interface 702 includes a live view 704 from a first camera (e.g., a rear camera) of device 100. Figures 7A to 7R In the example shown, the live view 704 overlays the user interface 702 within the display area (e.g., a live view display object, window, or viewport). The live view in this example has the same shape and size as the user interface 702. In some embodiments, the live view may be smaller than the user interface 702 and have a different shape. The live view 704 presents an image captured in real-time by a first camera.

[0220] exist Figure 7A In response to a time period in which the detection device 100 is substantially stationary and the horizontal edges of the device 100 are substantially horizontal at least a threshold amount, the device 100 has automatically begun capturing images using the first camera. The images captured by the device 100 have a sufficiently large resolution, size, frame rate, and / or aspect ratio to allow for the extraction of images from them at a later time from multiple different image recording modes (e.g., single image recording mode, HD video recording mode, square image recording mode, etc.). Figure 7A In the middle, when automatic image capture is initiated and performed, indicator 706 is displayed. In some implementations, the frame stream (e.g., such as...) Figures 5A to 5AEThe frame scrolling (506) shown is overlaid on or shown below the live view to show the image just captured by the device. The images captured by the device are temporarily stored and can be used for tagging and subsequent processing by the device or user; however, if the images captured by the device are not tagged by the user for long-term storage or meet other preset conditions, these captured images are discarded or overwritten after a predetermined amount of time (e.g., 24 hours).

[0221] Figures 7A to 7D The diagram illustrates that during image capture, device 100 detects that the device, including the first camera, is moving horizontally (e.g., as indicated by movement 703) for at least a threshold amount of time or distance. Live view 704 shows image frames gradually entering the field of view from the left side of user interface 702 as the device and its first camera are moving horizontally to the right. Based on the movement pattern of the device and its camera (e.g., stable horizontal movement) and / or changes in the captured images (e.g., new pixel patterns appearing primarily at the right edge of each newly captured image), the device determines that the image being captured by the camera is consistent with the initial portion of the panorama. Figure 7D The device 100 displays a set of panoramic recording guide objects to assist the user in taking the correct actions to complete the panorama when it is determined that the image being captured is consistent with the panoramic recording mode.

[0222] like Figure 7D As shown, the set of panoramic recording guidance objects includes a panoramic view object 708, which displays the currently completed portion of the panorama being captured. In some embodiments, the completed portion of the panorama includes one or more images captured before the device determines that a panorama is being captured. The set of panoramic recording guidance objects also includes a guide window 714 with a center line 716 and an arrow 718. The display of the panoramic view object 708 and the guide window 714 serves to prompt the user to continue moving the device horizontally to complete the panorama that has been initiated. In some embodiments, a text prompt 710 (e.g., “Continue moving to capture the panorama”) is displayed to explain to the user the actions required to complete the panorama. If the user does not follow the instructions specified in the text prompt 710 and moves the camera to the left or keeps the camera stationary for more than a threshold amount of time, the device 100 will determine that the user does not wish to capture the panorama and will stop displaying the panoramic recording guidance objects.

[0223] Figure 7E The user continues to move the device horizontally to the right according to the instructions in text prompt 710 (e.g., as indicated by move 703). As the device 100 moves to the right, the live view 704 is continuously updated to display the currently captured image. The panoramic view object 708 is continuously updated with additional portions of the completed panorama. Figure 7EIt is also shown that during panoramic recording, device 100 detects that the movement of the device and camera is slightly unstable and slightly deviates from the horizontal direction, and in response, device 100 displays another prompt (e.g., text prompt 720) requesting the user to keep arrow 718 along centerline 716.

[0224] Figure 7F The image shows the user has moved the device further to the right and the additional portion of the panorama has been completed and is presented in the panorama view object 708. Live view 704 shows a bird flying into the camera's field of view. The user slows down the device's movement (e.g., as indicated by a small movement 703) to better observe the bird. In response to detecting movement of the bird within the newly captured image (e.g., movement of pixels inconsistent with the panorama capture) and / or the slower movement of the device, the device determines that the currently captured image is consistent with the video recording mode. Based on the determination that the currently captured image is consistent with the video recording mode, device 100 displays a prompt (e.g., text prompt 722 "Stay still to start video") to request the user's confirmation that video recording should begin. In some embodiments, the device pauses panorama recording when the bird moves within the camera's field of view. In some embodiments, the device automatically removes moving pixels representing the bird from the completed portion of the panorama. In some embodiments, the device merges the moving pixels representing the bird into a location within the completed portion of the panorama (e.g., an image of the bird from one frame overlays the completed portion of the panorama).

[0225] Figure 7G The illustration shows that when a bird continues to fly within the camera's field of view, the user follows instructions in text prompt 722 and stops moving the camera. After the camera remains stationary for a threshold amount of time, the device activates video recording mode. In some embodiments, the recorded video includes at least some frames captured before the device activated video recording mode (e.g., frames captured since the device determined that the captured image was suitable for video recording). Figure 7G In the device 100, after activating the video recording mode, the device 100 displays a notification 724 indicating that the video recording mode has been activated and video is currently being recorded. In some embodiments, the recording indicator 706 changes its appearance to indicate that video is being recorded. Figure 7H The video recording continues as long as the device remains substantially stable and / or the captured images continue to change (e.g., due to the movement of birds).

[0226] Figure 7IThe device 100 has detected that one or more blurry frames have been recorded (e.g., due to the fast speed of a bird in flight). In response to the detection of blurry frames, the device 100 displays a prompt asking the user if he / she wants to remove the blurry frames now (e.g., text prompt 726). In some embodiments, the device 100 removes the blurry frames in response to a tap on the text prompt 724.

[0227] Figure 7J The diagram illustrates that if the user does not provide any input to remove the blurred frame (e.g., if no tap input is detected within a threshold amount of time after displaying prompt 724), the device then determines that the blurred frame should not be removed at this time and continues recording the video. In some embodiments, device 100 displays a notification (e.g., notification 728) to inform the user that the blurred frame can still be removed in video viewing mode after video recording is complete. In some embodiments, when a blurred frame is detected and / or when movement of the device is detected during video recording, the device restarts a new video clip (e.g., or inserts a marker in the video recording to indicate a change in scene). In some embodiments, if the user refuses to remove the blurred frame while video recording is still in progress, the device restarts a new video clip or inserts a marker to indicate a change in scene.

[0228] Figure 7K The device 100 indicates that when the bird flies out of the camera's field of view, the image captured by the camera no longer contains a large number of moving pixels (e.g., because the bird is no longer in the live field of view), and determines that panoramic shooting can be resumed from where it stopped. The device 100 redisplays the guide object 714 in its previous position with arrow 718, as well as the panoramic view object 708, which includes a portion of the panorama 712 that was completed at the start of video recording (see [link]). Figure 7K Device 100 also displays a prompt 730 requesting the user to restore the panorama via horizontal movement.

[0229] Figures 7L to 7M This demonstrates that panoramic shooting resumes when the user moves the device horizontally to the right as instructed in prompt 730 (e.g., as indicated by movement 705). Figures 7L to 7M In the middle, the additional part of the panorama was completed and displayed in the panorama view object 708.

[0230] exist Figure 7N The panorama is complete, and the user taps on the panorama viewing object 708 (e.g., as indicated by touch 713) to enter image viewing mode. In response to a tap gesture detected on the panorama viewing object 708, the device replaces the camera user interface 702 with a media viewing user interface 732, as shown below. Figure 7O As shown.

[0231] like Figure 7OAs shown, the media viewing user interface 732 includes a representation of a selected subset of captured and recorded images. For example, the selected image subset includes panorama 734 and video 736, corresponding to the panorama recording mode and video recording mode activated during image capture. Additionally, the media viewing user interface 732 includes a representation of several still images (e.g., image 738, image 740, and image 742). These still images are extracted from the captured images based on their content (e.g., image features) and their temporal relationship to the recorded video 736 and panorama 734. No individual image recording mode is activated during image capture; however, the device selects these images from the captured images to provide a representative subset of the content (e.g., a highlight of the media capture) for the user to view.

[0232] Figure 7P A tap gesture (e.g., by touch 743) is detected on a representation of video 736. In response to the detected tap gesture on the representation of video 736, device 100 displays image scrolling 750, which shows representative frames from video 736, including a detected blurred frame 752 (e.g., a blurred frame detected during capture or after video completion). The device displays a prompt 746, requesting the user to confirm whether the detected blurred frame should be removed, such as... Figure 7Q As shown.

[0233] Figure 7Q It is also shown that a tap gesture is detected on the "Yes" button in prompt 746 (e.g., by touching 748). In response to the detected tap gesture, device 100 removes the detected blurred frame 752 (as well as other blurred frames detected in video 736 optionally). Figure 7R The updated image scrolling 750 is shown, in which the blurred frame 752 has been removed.

[0234] Figures 8A to 8AC Exemplary user interfaces for switching between camera live views during image capture, according to some embodiments, are shown. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 14A to 14F The process is described below. For ease of explanation, some embodiments will be discussed with reference to the operation performed on a device having a touch-sensitive screen 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representation point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on the touch-sensitive display system 112. However, similar operations are optionally performed on a device having a display 450 and a separate touch-sensitive surface 451 in response to a contact detected on the touch-sensitive surface 451 when the user interface shown in the figures, along with the focus selector, is displayed on the display 450.

[0235] Figures 8A to 8AC An exemplary user interface 802 for a camera application is shown. The camera user interface 802 includes a live view 804 from a first camera (e.g., a rear camera) of device 100. Figures 8A to 8AC In the example shown, the live view 804 overlays the user interface 802 within a display area (e.g., a live view rendering object, window, or viewport), which is smaller than the user interface 802 and has a different shape. The live view 804 displays images captured in real time by a first camera.

[0236] exist Figure 8A In response to a time period in which the detection device 100 is substantially stationary and the horizontal edges of the device 100 are substantially horizontal at least a threshold amount, the device 100 has automatically begun capturing images using the first camera. The images captured by the device 100 have sufficiently high resolution, size, frame rate, and / or aspect ratio to allow for the extraction of images from multiple different image recording modes (e.g., single image recording mode, HD video recording mode, square image recording mode, etc.) at a later time. Figure 8A In the process of initiating and performing automatic image capture, indicator 806 is displayed. In some implementations, the frame stream (e.g., such as...) Figures 5A to 5AE The frame scrolling (506) shown is overlaid on the live view or shown below the live view to show the image just captured by the camera.

[0237] exist Figure 8A In the user interface 802, there is a control area 810, which includes: a plurality of camera control display options, including a photo album display option for launching the media viewing user interface (e.g., Figure 7O The control area 810 is shown as follows: a media viewing user interface; a movie control display for showing the movie recording mode menu (e.g., slow motion recording mode, time-lapse recording mode, etc.); a photo control display for showing the photo recording mode menu (e.g., square photo recording mode, 4:3 photo recording mode, etc.); a panorama control display for activating panorama recording mode; an editing control display for showing the media editing user interface; a flash control control display for changing camera flash settings; and a camera switching control display for switching between front and rear camera views. In some embodiments, if a frame stream is displayed, the control area 810 may optionally not be displayed.

[0238] Figures 8A to 8C This illustrates that while displaying live view 804 and image capture is in progress, a contact (e.g., contact 808) is detected on live view 804. Figure 8A(As shown). A touch 808 is detected in the upper portion of the live view 804 and moves downward on the touchscreen 112. In response to detecting the downward movement of the touch 808, the device 100 reduces the vertical scale of the live view 804 to adjust its aspect ratio. In some embodiments, the aspect ratio of the live view is not changed, and a frame size indicator (e.g., a rectangle) within the live view is adjusted according to the movement of the touch 808 to indicate the current aspect ratio of the image being captured. In some embodiments, the aspect ratio of the captured image is adjusted according to the aspect ratio of the live view 804. In some embodiments, metadata indicating the current aspect ratio is created, and the device 100 continues to capture images with the original aspect ratio. The captured images are then processed to obtain an image with the correct aspect ratio during post-processing.

[0239] like Figure 8D As shown, when the movement of contact 808 reverses in direction (e.g., upward), device 100 increases the vertical scale of real-time view 804 to adjust the aspect ratio of real-time view 804 according to the upward movement of contact 808.

[0240] exist Figure 8E During the capture, the lifting of contact 808 is detected, and the device 100 maintains the current aspect ratio of the real-time view 804 when the lifting of contact 808 is detected. Image capture continues with the current aspect ratio.

[0241] exist Figures 8A to 8E In this embodiment, the movement of contact 808 is a slow and intentional movement, lasting longer than a predetermined time threshold and / or slower than a predetermined speed threshold. In response to such long and / or slow movement of the contact, device 100 continuously and smoothly adjusts the aspect ratio of the live view (and the image being captured) based on the movement of the contact (e.g., by adjusting the vertical scale of the live view 804) until the removal of the contact is detected. In some embodiments, if the horizontal movement is not used to trigger other functions in the camera user interface (e.g., switching camera views), the horizontal movement can be used to adjust the horizontal scale of the image being captured.

[0242] and Figures 8A to 8E on the contrary, Figures 8F to 8I This demonstrates how to quickly switch between two preset aspect ratios (e.g., 4:3 and square aspect ratio) using a quick flick gesture. Figures 8F to 8GAs shown, a contact (e.g., contact 812) is detected in the upper portion of the live view 804. Contact 812 moves rapidly downwards, and a lift-off of contact 812 is detected shortly after the touch press of contact 812. Based on the determination that contact 812 has performed a flicking gesture (e.g., the characteristic velocity of contact 812 (e.g., the contact velocity 812 at lift) is greater than a threshold velocity, and / or the duration of contact 812 is less than a threshold amount of time), device 100 changes the current aspect ratio to a square aspect ratio based on the downward movement of contact 812. In some embodiments, the horizontal scale of the live view 814 is also adjusted to achieve a preset square aspect ratio, such as... Figure 8G As shown.

[0243] exist Figures 8H to 8I In the live view 804, another contact (e.g., contact 814) is detected in the upper portion. Contact 814 moves rapidly upward, and a lift-off of contact 814 is detected shortly after the touch press of contact 814. Based on the determination that contact 814 has performed a flicking gesture (e.g., the characteristic velocity of contact 814 (e.g., the contact velocity 814 at the time of lift) is greater than a threshold velocity, and / or the duration of contact 814 is less than a threshold amount of time), device 100 changes the current aspect ratio from a square aspect ratio to a 4:3 aspect ratio based on the upward movement of contact 814. In some embodiments, the horizontal scale of the live view 814 is also adjusted to achieve the preset 4:3 aspect ratio.

[0244] Figures 8J to 8N A process is illustrated in which a horizontal movement of a contact (e.g., contact 816) rotates a live view display object on a live view display 804. On the opposite side of the live view display object, a live view from another camera is displayed. In some embodiments, image capture continues while the live view display objects on both sides are rotated. In some embodiments, the device captures the rotation of the live view display object, including how the live view appears on the live view display object during its rotation. In some embodiments, the device continues to capture images from one of the cameras in the live view until the live view from the other camera moves to the front center position in the camera user interface.

[0245] like Figure 8J As shown, when the live view 804 from the rear camera is displayed on the first side of the live view display object, a contact 816 is detected on the live view 804 (e.g., on the right side of the live view 804). Figures 8K to 8LIn the process, device 100 detects leftward movement of contact 816, and in response to leftward movement of contact 816, device 100 rotates the live view display object around the vertical central axis in the camera user interface plane, while live view 804 continues to display and update on the first side of the live view display object based on the image being captured by the rear camera.

[0246] Figure 8M This illustrates the point at which the live view object has not yet rotated to the point where the second side of the live view object becomes visible in the camera user interface when the lifting of contact 816 is detected. Figure 8N In response to the removal of the contact 816, the device reverses the rotation of the live view object to redisplay the live view 804 at the front center position in the camera user interface. If the live view object has been rotated to a point where its second side becomes visible in the camera user interface, the device will continue to rotate the live view object upon removal of the contact until the live view associated with its second side is displayed at the front center position in the camera user interface. In other words, when the live view object is rotated by slowly moving the contact, it captures a stable position closest to the position when the contact is removed.

[0247] Figures 80 to 8T The process of flipping a dual-sided live view display object with a horizontal flick gesture is shown, which displays live view 804 and live view 820 on opposite sides of the live view display object.

[0248] like Figure 8O As shown, during image capture, device 100 detects a flicking gesture via contact 818. Movement of contact 818 begins in the left portion of the live view 804 and proceeds at high speed toward the right side of the display. Lifting of contact 818 is detected after a brief movement on the touch-sensitive surface, as... Figure 8J As shown. In response to detecting a flicking gesture via contact 818 (e.g., based on the rapid and / or brief duration of movement via contact 818), device 100 rotates a real-time view of the object displayed according to the direction of movement of contact 818, as shown. Figure 8P As shown.

[0249] Figures 8Q to 8R This demonstrates that after the flick gesture ends, the live view shows the object continuing to rotate, and the second side of the object becomes visible in the camera user interface (e.g., as shown). Figure 8R (As shown). A live view 820 from another camera (e.g., a front-facing camera) of device 100 is displayed on the second side of the live view display object.

[0250] Figures 8S to 8JThe live view object continues to rotate until its second side occupies the front center position of the camera user interface. Live view 820 is displayed on the second side of the live view object. Live view 820 shows an image of the user operating device 100.

[0251] In some implementations, image capture continues while the live view display object is rotated. In some implementations, the device captures the rotation of the live view display object, including how the live view appears on the live view display object during its rotation. In some implementations, the device continues to capture images of the live view from one of the cameras until the live view from the other camera occupies the front center position in the camera user interface.

[0252] Figures 8U to 8Y The following process is illustrated: a swipe gesture slides a multi-panel live view display object to move the currently displayed live view (e.g., live view 820) out of the visible area of ​​the camera user interface and bring another live view (e.g., live view 804) that is not currently visible into the visible area of ​​the camera user interface.

[0253] exist Figure 8U In the real-time view 820, a contact (e.g., contact 822) is detected near the left edge (e.g., within a threshold number of pixels from the edge of the real-time view 820). Figure 8V to Figure 8W In the middle, contact 822 moves toward the right side of the display. Based on the rightward movement of contact 822, device 100 slides the live view display object to the right, and live view 804 moves to the right using the live view display object. The right portion of live view 820 gradually moves out of the visible area of ​​the camera user interface, and the left portion of live view 804 gradually enters the visible area of ​​the camera user interface.

[0254] Figure 8X The image shows contact 822 reversing its direction of movement and moving towards the left side of the display. Based on the leftward movement of contact 822, device 200 slides the live view display object to the left, and live view 804 and live view 820 move to the left. The left portion of live view 804 gradually moves out of the visible area of ​​the camera user interface, and the right portion of live view 820 gradually re-enters the visible area of ​​the camera user interface.

[0255] Figure 8Y As shown, when contact 822 is lifted away, live view 804 and live view 820 are partially visible on the camera user interface, and the visible portion of the object displayed in the live view is divided between live view 804 and live view 820.

[0256] Figures 8Z to 8AAThe diagram illustrates how a flick gesture detected at the left edge of a live view object will shift the live view object to the next live view that will appear in the direction of the flick gesture. If only two live views are available on the device, the flick gesture will switch back and forth between the two live views.

[0257] like Figure 8Z As shown, if the flick gesture to touch 824 is to the right, the live view that will appear in the direction of the flick gesture is live view 804. In response to the flick gesture to touch 824, live view 804 is moved to the right by the live view display object and occupies the front center position of the camera user interface. If the flick gesture is to the left, live view 820 will move to the left along with the live view display object and occupy the front center position of the camera user interface. Figure 8AA If another flick gesture is detected on the live view 804 shown, the device will slide the live view display object and live view 804 to the left or right depending on the direction of the flick gesture, and display live view 820 at the front center position of the camera user interface.

[0258] Figure 8AB A tap gesture (e.g., by touching 826) is detected on the cinematic power indicator 828 in the control display area 810. In response to the tap gesture activating the cinematic power indicator 828, device 100 activates video recording of the image captured by the rear camera and shown in the live view 804 (e.g., as indicated by a change in appearance of the recording indicator 806) (e.g., in the default video recording mode). Figure 8AC As shown. Device 100 also displays a new set of power indicators in the control area 810 (e.g., power indicator 834 for activating slow-motion recording mode, power indicator 830 for stopping current video recording, power indicator 832 for activating time-lapse recording mode, and power indicator 836 for taking a still photo during video recording), such as Figure 8AC As shown. In some implementations, a swipe input to the left or right on the control area 810 causes the device to switch the video recording mode, for example, from the default video recording mode to a slow-motion recording mode or to a time-lapse recording mode, respectively.

[0259] Figures 8J to 8AAThis illustrates switching live views from different cameras during ongoing image capture. In some embodiments, a dedicated power indicator (e.g., power indicator 838) in control area 810 can be used to switch live views from different cameras. For example, a tap gesture detected on power indicator 838 causes the live view displayed in the camera user interface to switch to a live view from another camera not currently displayed in the camera user interface. In some embodiments, image capture may not necessarily be in progress when the above-described method for switching live views is implemented.

[0260] Figures 9A to 9S Exemplary user interfaces for adjusting selected images and video clips from video recordings, according to some embodiments, are shown. The user interfaces in these figures are used to illustrate the processes described below, including... Figures 18A to 18D The process in the process. Although some examples in the following examples will refer to input on a touchscreen display (which combines a touch-sensitive surface and a display), in some implementations, the device detects input on a touch-sensitive surface 651 separate from the display 650, such as Figure 6B As shown in the image.

[0261] In some embodiments, the device is an electronic device having a separate display (e.g., display 650) and a separate touch-sensitive surface (e.g., touch-sensitive surface 651). In some embodiments, the device is a portable multi-functional device 100, the display is a touch-sensitive display system 112, and the touch-sensitive surface includes a tactile output generator 167 on the display. Figure 1A For ease of explanation, see reference. Figures 9A to 9S and Figures 18A to 18D The described implementation will be discussed with reference to operation performed on a device having a touch-sensitive display system 112. In such implementations, the focus selector is optionally: a corresponding finger or stylus contact, a representation point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on the touch-sensitive display system 112. However, optionally, in response to the detection of a focus on the touch-sensitive surface 651, such as... Figures 9A to 9S The contact simultaneously displays on the display 650, such as Figures 9A to 9S The user interface shown performs similar operations on a device having a display 650, a separate touch-sensitive surface 651, and a focus selector.

[0262] Figure 9AA grid view user interface 902 is shown for selecting still images from a video recording, for example, to provide an image summary of the video recording. The grid view user interface 902 includes a "timeline" button 904 (e.g., from the grid view user interface 902) for accessing a timeline view user interface 935 and a "grid" button 906 (e.g., from the timeline view user interface 935) for accessing the grid view user interface 902. In some embodiments, the "grid" button is not displayed in the grid view user interface, and the "timeline" button is not displayed in the timeline view user interface.

[0263] The grid view user interface 902 includes still images 908, 910, 912, 914, 916, 918, 920, 922, 924, 926, 928, and 930 displayed in a grid layout. Still images 908 to 930 are selected images from video recording. For example, still images 908 to 930 are images selected in response to user input during video capture or recording, and / or images automatically selected (e.g., at periodic intervals) during video capture or recording. In some embodiments, still images 908 to 930 are a sequence of images ordered (at least initially) by, for example, timestamps in a video sequence or the order in which the images were selected. See below regarding... Figure 9E To be further described, still image 924 is a thumbnail corresponding to the selected video sub-clip of the video recording.

[0264] exist Figure 9B Input is detected at the location corresponding to the timeline button 904 (e.g., a tap gesture on touch 931). In response to the detected input, the timeline view user interface 935 is displayed, as shown below. Figure 9E As shown. In some implementations, such as Figure 9B As shown, the transition from the grid view user interface 902 to the timeline view user interface 935 is as follows: Figure 9E As shown, including as Figures 9B to 9E The animation transition is shown. For example, during the animation, still images 908 to 930 gradually descend from the grid, as... Figures 9C to 9D As shown, and as still images 908 to 930 descend, the still images are rearranged on timeline 934 into a linear arrangement of image adjustment objects and / or video adjustment objects, as... Figure 9E As shown. In some embodiments, during the animated transition from the grid view user interface 902 to the timeline view user interface 935, a representation 932 of images from still images 908 to 930 (e.g., a representation of still image 908) gradually fades into the view.

[0265] like Figure 9EAs shown, the timeline view user interface 935 includes a timeline 934 corresponding to the video recording (e.g., a video recording from which still images 908 to 930 are selected). Image adjustment objects 936, 938, 940, 942, 944, 946, 948, 950, 952, 954, and 956 are displayed on the timeline 934. Each of the image adjustment objects 936 to 958 corresponds to a still image (e.g., a still image displayed in the grid view interface 902). The positions of the image adjustment objects 936 to 958 correspond to the positions of the still images 908 to 930 within the video recording.

[0266] In some embodiments, the still image corresponding to the image adjustment object is displayed at the position corresponding to the image adjustment object. For example, the first still image 908 is displayed on one side of the first image adjustment object 936, and the second still image 910 is displayed on one side of the second image adjustment object 938, etc. In some embodiments, the image adjustment object is displayed at a position on the timeline 934, which indicates the position of still images 908 to 930 within the video recording.

[0267] like Figure 9E As shown, video adjustment object 958 is displayed on timeline 934. Video adjustment object 958 represents a video sub-clip that is a portion (e.g., less than the entirety) of the video recording represented by timeline 934. The left edge 960 of video adjustment object 956 corresponds to the start time of the video sub-clip, and the right edge 962 of video adjustment object 958 corresponds to the end time of the video sub-clip.

[0268] In some implementations, a thumbnail corresponding to the video sub-clip is displayed at a location corresponding to the video adjustment object. For example, thumbnail 924 is displayed at the left edge 960 of the video adjustment object 958. In some implementations, the thumbnail is displayed at the right edge 962 of the video adjustment object 958 or within the video adjustment object 958. The thumbnail does not necessarily correspond to the start or end frame of the video sub-clip, and optionally is a selected representative frame from the video sub-clip.

[0269] In some embodiments, input received at a position corresponding to the left edge 960 of the video adjustment object 958 (e.g., drag input) adjusts the start time of the video sub-clip. When drag input is received, the representation of the image displayed in the timeline view user interface 935 is updated to indicate the image corresponding to the current position of the left edge 960 on the timeline 934. When the drag input is complete, the start time of the video sub-clip is adjusted to the time within the video clip, which corresponds to the position of the left edge 960 of the video adjustment object 958 on the timeline 934. In some embodiments, input received at a position corresponding to the right edge 962 of the video adjustment object 958 (e.g., drag input) adjusts the end time of the video sub-clip. When drag input is received, the representation of the image displayed in the timeline view user interface 935 is updated to indicate the image corresponding to the current position of the right edge 962 on the timeline 934. When the drag input is complete, the end time of the video sub-clips is adjusted to the time within the video clip, which corresponds to the position of the right edge 962 of the video adjustment object 958 on timeline 934.

[0270] Figures 9F to 9H The input is shown as selecting and moving the first image adjustment object 936 to the right to update the still image 908 associated with the first image adjustment object 936.

[0271] exist Figure 9F In the process, input indicating selection of the first image adjustment object 936 is detected, as indicated by the focus selector 960. In response to this input, the appearance of the first image adjustment object 936 is changed (e.g., the width of the image adjustment object 936 is increased) to indicate its selection status.

[0272] exist Figure 9G In the process, when the focus selector 960 selects the first image adjustment object 936, the first image adjustment object 936 begins to move along the path indicated by arrow 962 according to the movement of the focus selector.

[0273] exist Figure 9H In response to the movement of focus selector 960 along the path indicated by arrow 962, the first image adjustment object 936 has moved to a new position on timeline 934, which is to the right of the previous position of the first image adjustment object 936 (e.g., ...). Figure 9G (As shown), and updated the representation 932 of still image 908. The updated representation 932 of still image 908 is the image corresponding to the new position of the first image adjustment object 936 on timeline 934. In Figure 9I In the current timeline, the termination of the previous input has been detected, and no image adjustment object is currently selected (e.g., no image adjustment is displayed as widening or highlighting).

[0274] Figures 9J to 9K The input is shown as selecting and moving the first image adjustment object 936 to the left to update the still image 908 associated with the first image adjustment object 936.

[0275] exist Figure 9J During the process, input indicating the selection of a first image adjustment object 936 is detected, as indicated by the focus selector 964. When the focus selector 964 selects the first image adjustment object 936, the first image adjustment object 936 begins to move along the path indicated by arrow 966 according to the movement of the focus selector 964.

[0276] exist Figure 9K In response to the movement of focus selector 964 along the path indicated by arrow 966, the first image adjustment object 936 has moved to a new position on timeline 934, which is to the left of the previous position of the first image adjustment object 936 (e.g., ...). Figure 9J (As shown), and updated the representation 932 of still image 908. The updated representation 932 of still image 908 is the image corresponding to the new position of image adjustment object 936 on timeline 934. In Figure 9L In the current timeline, the termination of the previous input has been detected, and no image adjustment object is currently selected (e.g., no image adjustment is displayed as widening or highlighting).

[0277] Figures 9M to 9O The input is shown as selecting and moving the second image adjustment object 938 to the right to update the still image 910 associated with the image adjustment object 938.

[0278] exist Figure 9M During the process, input indicating the selection of a second image adjustment object 938 is detected, as indicated by the focus selector 968. In response to this input, the appearance of the second image adjustment object 938 is changed (e.g., the width of the second image adjustment object 938 is increased), and a representation 970 of the still image 910 is displayed in the timeline view user interface 935.

[0279] exist Figure 9N When the focus selector 968 selects the second image adjustment object 938, the second image adjustment object 938 begins to move along the path indicated by arrow 972 according to the movement of the focus selector 968.

[0280] exist Figure 9O In response to the movement of focus selector 968 along the path indicated by arrow 972, the second image adjustment object 938 has moved to a new position on timeline 934, which is to the right of the previous position of the second image adjustment object 938 (e.g., ...). Figure 9N(As shown), and updated the representation 970 of still image 910. The updated representation 970 of still image 910 is the image corresponding to the new position of the second image adjustment object 938 on timeline 934. In Figure 9P In the current timeline, the termination of the previous input has been detected, and no image adjustment object is currently selected (e.g., no image adjustment is displayed as widening or highlighting).

[0281] exist Figure 9Q In the context of this input, input (e.g., a tap) is detected at the location corresponding to the grid button 906, as indicated by the focus selector 974. In response to this input, the grid view user interface 902 is redisplayed, as... Figure 9S As instructed. In some implementations, such as Figure 9Q As shown, the transition from the user interface 935 displaying the timeline view to the user interface 902 displaying the grid view is as follows: Figure 9S As shown, including as Figures 9Q to 9S The animation transition is shown. For example, during the animation, still images 908 to 930 gradually rise from timeline 934, and as still images 908 to 930 rise, these still images are rearranged into a grid arrangement of still images 908 to 930. The grid of still images 908 to 930 displayed in the grid view interface 902 includes an updated version of still image 908, such as... Figure 9K The still image 908 in the image is indicated by representation 932 (e.g., when adjusting the first image object 936). Figures 9F to 9K The adjustments shown are to update still image 908 (afterwards), and the updated version of still image 910, as follows: Figure 9O The still image 910 is shown in representation 970 (for example, when adjusting the object 938 of the second image). Figures 9M to 9O The adjustments shown are made to update the still image 910 afterward.

[0282] Figures 10A to 10EThis is a flowchart illustrating a method 1000 for simultaneously capturing and editing digital media according to some embodiments. As described below, method 1000 involves displaying a frame stream (e.g., multiple recently captured and recorded media images scrolling as frames on a display in a first direction) using a live view from a camera in a camera user interface. The frame stream scrolling through the camera user interface provides visual feedback to the user and serves as a visual reminder of the media images that have just been captured and / or recorded. The frame stream also allows the user to take some time to provide additional input to mark, view, or manipulate the images represented in the frame stream without interrupting the capture and recording of the media. Providing a live view from the camera to the frame stream enhances the operability of the device and makes the user-device interface more efficient (e.g., by providing timely visual reminders of recently recorded images, providing opportunities to perform additional operations on these recently recorded images without interrupting ongoing media capture and recording, and assisting the user in providing appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling the user to use the device more quickly and efficiently.

[0283] Method 1000 is performed on an electronic device having one or more processors, memory, a camera (e.g., mounted on the back of the electronic device, away from the user), and a display (e.g., a touch-sensitive display). In some embodiments, the electronic device includes a second camera (e.g., mounted on the front of the electronic device, facing the user). Some operations in method 1000 may be optionally combined, and / or the order of some operations may be optionally changed.

[0284] The device displays (1002) the camera user interface on the monitor (e.g., Figure 5C The user interface 500 on the touchscreen 112 in the camera includes a live view from the camera (e.g., showing...). Figure 5C (Branch 502 in the image). In some embodiments, the camera user interface also includes a power indicator for initiating image recording, one or more power indicators for setting operating modes (e.g., flash / illumination settings, dynamic range settings, and / or image filter settings), and a power indicator for switching between the (first) camera and the second camera.

[0285] While displaying the live view from the camera, the device (1004) records (e.g., images) media captured by the camera (e.g., recording video with aspect ratio, resolution, and frame rate that supports post-processing into any video, burst group photo, and high-resolution photography), while continuing to display the live view from the camera (e.g., device 100 is recording). Figure 5C The image of tree branch 502, or the image that device 100 is recording. Figures 5V to 5X(Video in the video). In some implementations, audio is also recorded. Additionally, as frames scroll on the display in a first direction (e.g., when capturing images in recording, each Xth image scrolls on the touch-sensitive display from right to left at the bottom of the live view), the device displays a representation of multiple media images recorded while displaying the live view from the camera (e.g., a representation of some of the multiple captured media frames displayed in the scrolling). For example, when device 100 records images including tree branch 502 and cat 510, in Figures 5D to 5G In frame 508, frame 509, etc., scroll from right to left across the bottom portion of the live view in scroll 506. In another example, the device... Figures 5V to 5W Images are recorded, and frames 556 through 562 scroll from right to left across the bottom portion of the live view in scroll 506. These frames display smaller versions of the images captured by the device. Not every recorded image is displayed in scroll 506 (e.g., the cat 510 is not visible in frame 509, and then, in the next frame 512, three-quarters of the cat is in the camera's field of view). In some embodiments, when the media capture application is first launched, the device displays a live view from the camera, but the images captured by the camera are not recorded (e.g., designated for persistent storage in memory, subject to automatic and / or manual processing or deletion) until user input activates recording (e.g., in some embodiments, in...). Figure 5A In the process, device 100 captures an image of tree branch 502, but does not start recording images or scrolling frames of the image on the screen until it detects a tree branch 502. Figure 5B (A light tap on contact 504). Figure 5C In the process, after a tap is detected, the device begins recording the captured image and displays scroll 506. (For example, in some implementations, if device 100 only captures an image, but detects a tap, it may also record the captured image.) Figure 5B If no image recording starts after a tap of contact 504, then when a long press input is detected, for example in... Figures 5V to 5W The device begins recording the captured images, and... Figures 5V to 5X The recorded frames are displayed in a scrolling 506 format. In some implementations, the device begins recording media as soon as the media capture application is launched (e.g., without a separate user activating recording).

[0286] In some embodiments, before recording media images captured by the camera, while displaying a live view from the camera, the device captures (1006) a media image corresponding to the live view from the camera without recording; and detects a first input to activate media image recording with the camera (e.g., detecting a tap input on the live view from the camera or a power indication for initiating media capture (e.g., a record button) (e.g., displayed on the live view)). In some embodiments, recording of the media image begins in response to the detection of the first input. For example, in some embodiments, device 100 captures... Figure 5A The image of tree branch 502 was displayed, but recording of images and scrolling of image frames on the screen did not begin until the image included... Figure 5B A light tap on the contact 504. Figure 5C In the middle, after detecting a tap, the device begins recording the captured image and displays scrolling 506.

[0287] In some implementations, “captured” media refers to media automatically, temporarily, and in real-time stored by an electronic device without active user intervention. In some implementations, media captured by a camera is automatically stored (e.g., in the memory of the electronic device) for a predetermined duration (e.g., while the camera user interface is active, or 24 hours after capture), and is automatically deleted or overwritten unless instructed by the user to mark the captured media for subsequent processing (e.g., tagging, editing, compressing, converting, etc.) and / or persistent storage. In some implementations, media marked for subsequent processing or persistent storage (e.g., based on user input or pre-configured device settings) is considered “recorded.” Responding to user input to initiate recording helps reduce unnecessary storage space usage and allows the user better control over the device's operation. Saving storage space and improving user control enhance device operability and make the user-device interface more efficient (e.g., by freeing up storage space for other uses, helping users provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0288] In some implementations, in response to the detection of the first input, a representation of multiple media images is started to be displayed as frames scrolling on the display (1008) (e.g., in some implementation examples, via...). Figure 5B (e.g., in the touch gesture of contact 504) Figure 5VIn some implementations, capture and recording are initiated only by an earlier tap input (via a long press of the 558 contact). Responding to user input by displaying the frame stream helps reduce unnecessary visual clutter and allows the user better control over the device's operation. Saving display space and improving user control enhance device operability and make the user-device interface more efficient (e.g., by freeing up display space for other uses, helping users provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this reduces power consumption and extends the device's battery life by enabling users to use the device more quickly and efficiently.

[0289] In some implementations, prior to the detection of the first input, a representation of multiple media images is displayed as scrolling frames on the display (1010) (e.g., once media image capture begins, a representation of multiple media images is displayed as scrolling frames on the display); for example, in some implementations, the application is launched to... Figure 5C The status shown will be displayed and recording will start automatically. Automatic frame stream initiation helps reduce the amount of input required to view captured / recorded images. Automatic frame stream initiation enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to perform desired operations). Additionally, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0290] In some implementations, it is represented as a frame scrolling across the display (e.g., Figures 5B to 5J Frames 508, 509, etc. (1012) overlay a portion of the live view from the camera (e.g., at the bottom of the live view, such as...). Figures 5B to 5J (See scrolling 506 in the image). Displaying the frame stream on a live view helps reduce unnecessary visual clutter and saves display space. Saving display space enhances device operability and makes the user-device interface more efficient (e.g., by freeing up display space for other uses). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0291] In some implementations, displaying a representation of multiple media images as frames scrolling on the display includes (1014) scrolling the corresponding frame (e.g., the last or rightmost frame in the current frame sequence, such as...) from a first side of the display. Figure 5E Frame 512 in the image scrolls onto the display (e.g., each frame gradually slides in from the right side of the display, such as...). Figures 5E to 5G(Frame 512 in the image); when the corresponding frame is scrolled onto the display and before the entire corresponding frame is displayed, a scaled-down copy of the live view from the camera is displayed within the corresponding frame (e.g., only the left portion of the frame is visible on the display; for example, frame 512 shows...). Figures 5E to 5F A copy of the live image displayed in the user interface 500 is used; and, when it is determined that the entire corresponding frame has been scrolled onto the display, a scaled-down copy of the live view in the corresponding frame is replaced with a media image corresponding to the current image object in the live view (e.g., once the corresponding frame has been scrolled onto the display, the scaled-down copy of the live view is frozen; for example, once the entire frame 512 is visible on the touchscreen 112, the image is frozen, such as...). Figure 5G (As shown). In some implementations, the media image replacing the live image in the frame is the last media image captured or recorded before the frame is fully scrolled onto the screen (e.g., immediately before frame 512). Figure 5F The recorded images become fully visible on the touchscreen 112. Figure 5G (The image displayed in frame 512). In some implementations, the media image replacing the live image in the frame is a first media image recorded after the frame has fully scrolled onto the screen. Displaying the live image at the end of the frame stream helps provide visual feedback on the progress of the relative positions of the frames in the timeline. Enhanced visual feedback improves device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0292] In some implementations, the representation of multiple media images scrolling on the display as frames includes (1016) fewer than all media images recorded by the camera (e.g., in one implementation, thirty images are recorded per second, but only two images are displayed during scrolling, e.g., every fifteenth recorded image is displayed during scrolling). For example, as... Figure 5I As shown, it depicts a cat 510 walking under a branch 502 in the image shown in frame 514, and then lying down in the image shown in frame 516. Although the camera also records images of the cat while it is lying down, these images are not displayed in scroll 506. Displaying only a subset of the recorded images in the frame stream helps reduce unnecessary visual clutter and makes it easier to check frames. This reduction in visual clutter also enhances the operability of the device and makes the user-device interface more efficient (e.g., by freeing up display space for other purposes, helping users provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0293] In some implementations, when displaying a live view from a camera and representing multiple images as scrolling frames on the display, the device detects (1018) a second input (e.g., a tap gesture on a touch-sensitive display, or a press input on a virtual or physical button on an electronic device), while the focus selector is positioned corresponding to the corresponding representation of the first media image currently scrolling on the display (e.g., when the second input is detected, when closest to detecting the second input, or an image captured by the camera immediately preceding the detection of the second input); and in response to detecting the second input, the appearance of the corresponding representation of the first media image in the scrolling frames of the display is changed (e.g., highlighting, coloring, and / or zooming in on the scrolling frames / images) to indicate that the first media image has been marked for later reference (e.g., independently of recorded video storage that also includes the first media image). For example, device 100 detects including Figure 5AF The tap gesture on frame 576 of the middle element is touched on 580, and in response, increases... Figure 5AG The size of frame 576 in the frame indicates that the image is tagged for later reference. In some implementations, the device creates metadata to tag the first media image (e.g., creating a timestamp or flag on the first image and persistently storing the first media image as a separate image). Allowing the user to tag images in a scrolling frame stream provides the user with appropriate visual context while tagging images. Changes in the appearance of the tagged frames provide visual feedback about the actions performed. Visual content and improved visual feedback enhance the operability of the device and make the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0294] In some embodiments, copies (1020) of the corresponding tagged media images are stored independently (e.g., as individual images) from a recorded video storage that includes the corresponding tagged media images in the device's memory (e.g., a first copy of the tagged image is stored as a frame in the recorded video (e.g., as unprocessed or processed video), and a second copy of the tagged image is stored independently as a photo). In some embodiments, after the continuous capture of media images is terminated, all tagged media frames and / or videos are collected into a dedicated user interface within an associated media management application (e.g., the device picks out all tagged photos and videos from the collected media stream). In some embodiments, the collected media items are displayed with an automatic summary that includes various types of media items, such as... Figure 7O As shown. In some implementations, the collected media is displayed in a timeline or grid, such as... Figure 9A and Figure 9E As shown, separating the tagged images from the video storage that also includes them allows users to access the tagged images independently of the stored video, thus giving users greater control over the stored content. Improved control enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to access and manipulate tagged images). Additionally, enabling users to use the device more quickly and efficiently reduces power consumption and extends device battery life.

[0295] In some implementations, a copy (1022) of the corresponding tagged media image is configured to be displayed at a different aspect ratio (and / or resolution) than the recorded video that includes the corresponding tagged media image (e.g., the copy of the tagged image is processed differently from the recorded video, such that the tagged image is configured to be displayed in a photo aspect ratio and the captured media is configured to be displayed in a video aspect ratio).

[0296] The marker images have a display aspect ratio independent of the video that also includes the marker images. This allows users greater control over how the stored content is displayed. Improved control enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to view marker images at an appropriate aspect ratio). Additionally, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0297] In some implementations, when a live view from a camera is displayed and a representation of multiple images is scrolled across the display as frames, the device detects (1024) a third input (e.g., a tap gesture on the live view displayed on a touch-sensitive display, or a press input on a virtual or physical shutter button on an electronic device), which marks the simultaneously captured image (e.g., an image being captured by the camera at the moment the third input is detected, at the moment closest to the detection of the third input, or immediately before the detection of the third input) (e.g., the simultaneously captured image is an image shown in the live view or in a live image in a representation / frame that has just scrolled onto the display or is about to be fully scrolled onto the display); and in response to the detection of the third input, a corresponding representation of the simultaneously marked and captured image is displayed, wherein the appearance of the corresponding representation of the simultaneously marked and captured image is altered (e.g., highlighted, colored, and / or magnified relative to other representations of the scrolling image) to indicate that the simultaneously marked and captured image has been marked for later reference (e.g., independently of recorded video storage that also includes simultaneously captured and marked images). For example, device 100 detects a tap gesture, including a touch 518 on the live image, corresponding to frame 516 in scrolling 506, as... Figure 5H As shown. In response, the device marks the captured image and zooms in. Figure 5I The display of frame 516 in the image. Allowing users to mark simultaneously captured images enables users to control image marking more precisely and promptly. Changes in the appearance of the marked frames provide visual feedback on the actions performed. Improved control and visual feedback enhance device operability and make the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0298] In some implementations, images that are simultaneously marked and captured are displayed (1026) among multiple digital media images on a scrolling display. Figure 5J The corresponding representation of the simultaneously tagged and captured digital media image (516) is displayed regardless of whether a corresponding representation of the simultaneously tagged and captured image has already been displayed, if no such representation has already been displayed. For example, in one embodiment, thirty images are captured per second, and only two images are displayed in the scroll (e.g., every fifteenth captured image is displayed in the scroll). In such embodiments, the representation of the image is added to the scroll even if the simultaneously tagged and captured images are not exactly the two images that will be displayed in the scroll. In some embodiments, if a representation of the tagged media image is not displayed in the scroll, the device will change the appearance of the representation of the media image that is closest in time to the appearance of the tagged image displayed in the scroll. Displaying tagged frames in the frame stream provides visual feedback about the actions that have been performed. Improved visual feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0299] In some implementations, when a representation of multiple images is displayed as frames scrolling on the display in a first direction (and when a live view from a camera is displayed), the device detects (1028) a third input that causes the focus selector to move in a second direction opposite to the first direction (e.g., a swipe input that begins when the focus selector is in a first position in the camera user interface, which corresponds to a representation of multiple images as frames scrolling on the display, wherein the swipe input includes movement of the focus selector in the second direction opposite to the first direction (e.g., a swipe or drag gesture for scrolling); for example, a swipe gesture including movement 546 of touch 544 on scroll 506 from... Figure 5P Position 544-a in the middle to Figure 5Q(position 544-b); and, in response to detecting a third input, according to the movement of the focus selector, scrolling a representation of multiple images in a second direction as frames on the display (e.g., scrolling backwards) (e.g., movement of touch 546 navigates scrolling 506 of frames, independent of the live images displayed in the user interface 500, causing previously displayed frames 514 and 515 to return to the touchscreen 112, as shown). Figures 5P to 5Q (As shown). In some implementations, as a representation of multiple images is scrolled on the display in a second direction, the live view from the camera is replaced by one or more of the previously captured images (e.g., the live image after scrolling 506 is also rewound). Allowing the user to swipe on the frame stream to restore frames no longer in view allows the user to have better control over the stored content. Improved control enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to view stored content). Additionally, by enabling users to use the device more quickly and efficiently, this can also reduce power consumption and extend the device's battery life.

[0300] In some implementations, when displaying representations of multiple images as scrolling frames on the display (and when displaying a live view from a camera), the device detects (1030) a fourth input (e.g., a tap input detected when the focus selector is located at a position corresponding to a first representation of an image scrolling on the display as a frame; for example, including...). Figure 5J The fourth input activates (e.g., turns on) the image viewing mode (e.g., a tap gesture on the frame) on frame 516; and, in response to detecting the fourth input activating the image viewing mode, the display of the live view from the camera is replaced by displaying the first recorded media image (e.g., the recorded image corresponding to the first representation) on multiple media images (e.g., in response to detecting the fourth input activating the image viewing mode). Figure 5J Contact 524, equipment 100 Figure 5K The image viewing user interface 526 replaces the display of the real-time image user interface 500. By allowing users to enter image viewing mode from the user interface using a real-time view and frame stream, the device enables users to view recorded images in more detail. The ability to access image viewing mode from the user interface using a real-time view enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to inspect stored content). Additionally, by enabling users to use the device more quickly and efficiently, this can reduce power consumption and extend the device's battery life.

[0301] In some implementations, in response to detecting a fourth input activating the image viewing mode, the device creates (1032) metadata to tag the first recorded media image among multiple media images. In some implementations, the representation of the first recorded media image may also be visually tagged when it is subsequently displayed in frame scrolling. Automatically tagging images upon entering image viewing mode enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to inspect and tag stored images). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0302] In some implementations, when a first recorded media image (e.g., a recorded image corresponding to the first representation) is displayed in image viewing mode, the device detects (1034) a fifth input that causes the focus selector to move along a third direction (e.g., in some implementations, the third direction is perpendicular to the first direction; in some implementations, the third direction is parallel to the first direction) (e.g., the fifth input is a swipe input that begins when the focus selector is in a first position in the camera user interface corresponding to the display of the first recorded media image, wherein the swipe input includes movement of the focus selector in a third direction (e.g., in a direction having a component movement perpendicular to the main axis of the recorded media image); for example, in Figure 5L This includes a swipe gesture (528) to the right (contact 526); and in response to detecting a fifth input, the display of the first recorded media image is replaced with a second recorded media image (e.g., a recorded image immediately before or after the first recorded media image; for example, device 100 uses...). Figure 5M Replace the image of the cat lying down in the middle. Figure 5L The image shown is of a cat 100 located below a branch 502. Allowing users to switch images in image viewing mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to check attached stored images). In addition, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0303] In some implementations, when the first recorded media is displayed in image viewing mode, in response to detecting a fifth input that causes the focus selector to move upwards on a third side, the device creates (1036) metadata that marks the second recorded media image (and optionally, deletes the metadata that marks the first recorded media image; for example, in...). Figure 5J Frame 516, which corresponds to the image of cat 510 lying under branch 502, is marked in the image. Figures 5K to 5L In the image viewing user interface 526, navigation markers display images taken just before the cat lay down, such as... Figure 5LAs shown, and without labeling the image of the cat lying down, such as... Figure 5J As shown. Additionally, when viewing the image, the user interface 526 is comprised of components from... Figure 5N When the swipe gesture 532, which involves touching 530 downwards from the top of the screen, is closed, the previously marked frame 516 is replaced by a newly marked frame 515, thus corresponding to... Figure 5O (e.g., an image of the cat before it lay down, shown in image 510) (e.g., the timestamp of a recorded image is adjusted to indicate which image should be stored as an image independent of the entire recorded image data). In some implementations, a power indication is selected for each image displayed within the image viewing mode, and the user is allowed to select multiple images to be marked within the image viewing mode. Automatic image marking when switching images in image viewing mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to check and mark stored images). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0304] In some implementations, when displaying the first recorded media image in image viewing mode (e.g., as...) Figure 5O As shown in the scrolling 506, after the image viewing user interface 526 is replaced by the real-time image user interface 500, the device continues (1038) to record media images with the camera (e.g., in background processing). Continuing media recording in image viewing mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input and time required to record media and check stored content). In addition, this can also reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0305] In some implementations, when the corresponding recorded media image (e.g., a first, second, or subsequent photo, rather than a live image from the camera) is displayed in image viewing mode, the device detects (1040) deactivating (e.g., turning off) the sixth input (e.g., including from...) of the image viewing mode. Figure 5N The gesture is a swipe gesture (530) to move the screen downwards from the top of the screen (532). Furthermore, in response to a sixth input that detects the deactivation of image viewing mode, the display of the corresponding recorded media image is replaced with a live view from the camera (e.g., before activating image viewing mode, the media capture application is returned to its current state; for example, ...). Figure 5N Equipment 100 in Figure 5OThe real-time image user interface 500 replaces the image viewing user interface 526. Allowing users to return to the real-time view when exiting image viewing mode enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to record media and inspect stored media content). In addition, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0306] In some implementations, when a representation of multiple images is displayed as frames scrolling on the display in a first direction (and when displaying a live view from a camera), the device detects (1042) a seventh input that specifies the start and end of a recorded media image sequence (e.g., detecting an initial touch press corresponding to a long press or deep press input on the live view to mark the currently recorded media image as the start of the recorded media image sequence (e.g., creating metadata to identify the start of a video clip), continuing to detect the touch to mark the body of the recorded media image sequence (e.g., creating metadata to identify the body of a video clip), and detecting the release of the touch to mark the currently recorded media image as the end of the recorded media image sequence (e.g., creating metadata to identify the end of a video clip); for example, device 100 in Figures 5V to 5X The device detects a long press gesture, including contact 558. In response to a seventh input that detects the start and end of a specified sequence of recorded media images, the device creates metadata that labels the recorded media image sequence as a unit of a first type of media (e.g., a video clip); and an indication that the sequence of recorded media images is labeled as a first type of media unit among the plurality of images scrolling as frames on the display (e.g., highlighting, coloring, and / or zooming in on the frame / image sequence in scrolling increments; e.g., in...). Figures 5V to 5X Frames 556, 560, and 562 have been enlarged. Allowing users to mark video while displaying the frame stream enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required for video recording). Additionally, by enabling users to use the device more quickly and efficiently, this can reduce power consumption and extend device battery life.

[0307] In some implementations, copies of the recorded media image sequence (e.g., tagged video clips) are (1044) stored in the device's memory independently of (e.g., as video clips) the recorded media images (e.g., full-length recorded video). (e.g., a first copy of the recorded media image sequence is stored as a frame in the recorded video (e.g., as unprocessed or processed video), and a second copy of the tagged images is stored separately as a video clip.) In some implementations, after the continuous capture of media images is terminated, all tagged media frames and / or videos are collected into a dedicated user interface within an associated media management application (e.g., the device picks out all tagged photos and videos from the collected media stream). Storing the tagged video independently of the remaining recorded image data, which also includes the tagged video, allows the user to access the tagged video independently of the rest of the stored image data, thus allowing the user greater control over the stored content. Improved control enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to access and manipulate the tagged video). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0308] In some implementations, a copy (1046) of the recorded media image sequence is configured to be displayed at a different aspect ratio (and / or resolution) than the recorded media images (e.g., a copy of a video clip is processed differently from the recorded video, such that the marked video clip is configured to be displayed at a different aspect ratio than the full-length recorded video). The marked images have a display aspect ratio independent of the display aspect ratio of the recorded media images that also include the marked video. This allows the user greater control over how the stored content is displayed. Improved control enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to view marked video at an appropriate aspect ratio). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0309] In some implementations, when a representation of multiple images is displayed as frames scrolling on the display (and when a live view from a camera is displayed), the device detects (1048) an eighth input (e.g., a tap gesture detected when the focus selector is located at a position corresponding to at least a portion of the marked media image sequence, in the multiple images scrolling on the display as frames; e.g., in...). Figure 5AAThe marked frame 560 includes a tap gesture of contact 568, which activates (e.g., opens) a video viewing mode (e.g., a tap gesture on a frame in the marked frame sequence); and, in response to detecting an eighth input activating the video viewing mode, the display of the live view from the camera is replaced with playback of a first recorded video clip (e.g., the first recorded video clip corresponds to a marked sequence of recorded media images; for example, the device uses...). Figure 5AB The video viewing user interface in 568 has been replaced. Figure 5AA The real-time image user interface 500 in the video is played. Figures 5AB to 5AD (The marked video). In some implementations, when in video viewing mode, the device displays controls (e.g., adjustable boundary lines) for selecting different start and / or end points for marked video clips from a representation of multiple images. By allowing users to access video viewing mode from the user interface using a live view and frame stream, the device enables users to view recorded video in more detail. The ability to access video viewing mode from the user interface using a live view enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to inspect stored content). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0310] In some implementations, when playing back the first video clip (e.g., as... Figure 5AE As shown in the scrolling 506, after the video viewing user interface 568 is replaced by the live image user interface 500, the device continues (1050) to record media images with the camera. Continuing media recording in video viewing mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input and time required to record media and check stored content). In addition, this can also reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0311] In some implementations, during playback of the first video clip, the device detects (1052) the ninth input (e.g., including input from...). Figure 5A The ninth input (a top-down contact 570 of the touchscreen 112 with a swipe gesture 572) deactivates (e.g., disables) the video viewing mode; and, in response to detecting the ninth input deactivating the video viewing mode, the playback of the first video clip is replaced with a live view from the camera (e.g., before activating the video viewing mode, the media capture application is returned to its current state; for example, ...). Figure 5AD Equipment 100 in Figure 5AEThe real-time image user interface 500 replaces the display of the video viewing user interface 568. Allowing users to return to the live view when exiting video viewing mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of input required to record media and inspect stored media content). In addition, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0312] It should be understood that, Figures 10A to 10E The specific order of operations described herein is merely illustrative and not intended to indicate that this order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that details of other processes described herein relative to other methods (e.g., methods 1200, 1400, 1600, and 1800) also apply in a similar manner to the above descriptions relative to other methods described herein. Figures 10A to 10E The method described above with reference to method 1000 may optionally have one or more features of the touch, gesture, user interface object, various types of thresholds, focus selector, live view, frame stream, captured media content, media item and live view display object, power representation and animation described herein with reference to other methods described herein (e.g., method 1200, method 1400, method 1600 and method 1800). For the sake of brevity, these details will not be repeated here.

[0313] According to some implementation plans Figure 11 A functional block diagram of an electronic device 1100 configured according to the principles of various described embodiments is shown. The functional blocks of this device are optionally implemented by hardware, software, or a combination of hardware and software that execute the principles of the various described embodiments. Those skilled in the art will understand that... Figure 11 The functional blocks described herein may be optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0314] like Figure 11As shown, the electronic device 1100 includes: a display unit 1102 configured to display a user interface; a camera unit 1106 configured to capture media; and a processing unit 1108 coupled to the display unit 1102 and the camera unit 1106. In some embodiments, the processing unit includes a detection unit 1110, a recording unit 1112, a capture unit 1114, a scrolling unit 1116, a replacement unit 1118, a change unit 1120, and a creation unit 1122.

[0315] The processing unit 1108 is configured to: display (e.g., using the display unit 1102) a camera user interface on the display unit 1102, the camera user interface including a live view from the camera unit 1106; while displaying the live view from the camera unit 1106: while continuing to display the live view from the camera unit 1106, record (e.g., using the recording unit 1112) media images captured by the camera unit 1106; and display (e.g., using the display unit 1102) a representation of multiple media images, the media images being recorded while displaying the live view from the camera unit 1106 as a frame scrolling (e.g., using the scrolling unit 1116) on the display unit 1102 in a first direction.

[0316] In some embodiments, the processing unit 1108 is further configured to: before recording media images captured by the camera unit 1106; while displaying a live view from the camera unit 1106; capture (e.g., using the capture unit 1114) media images corresponding to the live view from the camera unit 1106 without recording; and detect (e.g., using the detection unit 1110) a first input that activates media image recording using the camera unit 1106, wherein recording of the media image begins in response to the detection of the first input.

[0317] In some implementations, the display unit 1102, which represents multiple media images, is initiated as a frame that scrolls on the display unit 1102 (e.g., using the scrolling unit 1116) in response to the detection of the first input.

[0318] In some implementations, the display unit 1102, which represents multiple media images, as a frame that scrolls on the display unit 1102 (e.g., using the scrolling unit 1116) begins before the detection of the first input.

[0319] In some implementations, representations of multiple images are scrolled as frames on display unit 1102 (e.g., using scrolling unit 1116) to cover a portion of the real-time view from camera unit 1106.

[0320] In some embodiments, displaying a representation of multiple media images as frames scrolling on display unit 1102 (e.g., using scrolling unit 1116) includes: scrolling a corresponding frame from a first side of display unit 1102 (e.g., using scrolling unit 1116) onto display unit 1102; while scrolling the corresponding frame onto display unit 1102 (e.g., using scrolling unit 1116) and before displaying the entire corresponding frame, displaying (e.g., using display unit 1102) a scaled-down copy of the live view from camera unit 1106 within the corresponding frame; and when it is determined that the entire corresponding frame has been scrolled onto display unit 1102, replacing (e.g., using replacement unit 1118) the scaled-down copy of the live view in the corresponding frame with a corresponding media image corresponding to the current image in the live view.

[0321] In some implementations, a representation of multiple media images is scrolled as frames on display unit 1102 (e.g., using scrolling unit 1116) and includes fewer than all media images recorded by camera unit 1106.

[0322] In some embodiments, the processing unit 1108 is further configured to: when displaying a real-time view from the camera unit 1106 and displaying representations of multiple images as frames scrolling on the display unit 1102 (e.g., using the scrolling unit 1116); detect (e.g., using the detection unit 1110) a second input when the focus selector is located at a position corresponding to the corresponding representation of the first media image currently scrolling (e.g., using the scrolling unit 1116) on the display unit 1102; and in response to detecting the second input, change (e.g., using the change unit 1120) the appearance of the corresponding representation of the first media image in the frame scrolling (e.g., using the scrolling unit 1116) on the display unit 1102 to indicate that the first media image has been marked for later reference.

[0323] In some embodiments, the processing unit 1108 is further configured to: detect (e.g., using the detection unit 1110) a third input that marks the currently captured image when displaying a real-time view from the camera unit 1106 and displaying representations of multiple images as frames scrolling on the display unit 1102 (e.g., using the scrolling unit 1116); and in response to the detection of the third input, be able to display (e.g., using the display unit 1102) a corresponding representation of the simultaneously marked and captured images, wherein the appearance of the corresponding representation of the simultaneously marked and captured images is changed to indicate that the simultaneously marked and captured images have been marked for later reference.

[0324] In some implementations, among a plurality of digital media images scrolling on display unit 1102 (e.g., using scrolling unit 1116), a corresponding representation of the simultaneously marked and captured image is displayed, regardless of whether the corresponding representation of the simultaneously marked and captured image has been displayed if the simultaneously marked and captured digital media image has not yet been marked.

[0325] In some implementations, a copy of the marked media image is stored in the device's memory independently of the recorded video including the marked media image.

[0326] In some implementations, copies of the marked media images are configured to be displayed at a different aspect ratio than the recorded video that includes the marked media images.

[0327] In some embodiments, the processing unit 1108 is further configured to: detect (e.g., using the detection unit 1110) a third input when displaying a representation of multiple images as frames scrolled on the display unit 1102 in a first direction (e.g., using the scrolling unit 1116), the third input causing the focus selector to move in a second direction opposite to the first direction; and in response to detecting the third input, scroll (e.g., using the scrolling unit 1116) the representation of multiple images as frames on the display unit 1102 in the second direction according to the movement of the focus selector.

[0328] In some embodiments, the processing unit 1108 is further configured to: when displaying a representation of multiple images as frames scrolling on the display unit 1102 (e.g., using the scrolling unit 1116): detect (e.g., using the detection unit 1110) a fourth input that activates an image viewing mode; and in response to detecting the fourth input that activates the image viewing mode, replace (e.g., using the replacement unit 1118) the display of the live view from the camera unit 1106 with the display of a first recorded media image among the multiple media images.

[0329] In some embodiments, processing unit 1108 is further configured to: in response to detecting a fourth input activating the image viewing mode, create (e.g., using creation unit 1122) metadata that tags the first recorded media image among a plurality of media images. 50. The electronic device according to claim 48 or 49, further configured to: when displaying the first recorded media image in the image viewing mode: detect (e.g., using detection unit 1110) a fifth input that causes the focus selector to move upwards on the third; and in response to detecting the fifth input, replace (e.g., using replacement unit 1118) the display of the first recorded media image with a second recorded media image.

[0330] In some implementations, processing unit 1108 is also configured to: when the first recorded media is displayed in image viewing mode, in response to detecting a fifth input that causes the focus selector to move upwards on the third side, create (e.g., using creation unit 1122) metadata that marks the image of the second recorded media.

[0331] In some implementations, while the first recorded media image is displayed in image viewing mode, the device continues to record media images using camera unit 1106 (e.g., using recording unit 1112).

[0332] In some embodiments, the processing unit 1108 is further configured to: when displaying a corresponding recorded media image in image viewing mode: detect (e.g., using detection unit 1110) a sixth input that disables the image viewing mode; and in response to detecting the sixth input that disables the image viewing mode, replace (e.g., using replacement unit 1118) the display of the corresponding recorded media image with a real-time view from the camera unit 1106.

[0333] In some embodiments, the processing unit 1108 is further configured to: when displaying a representation of multiple images as frames scrolling on the display unit 1102 in a first direction (e.g., using the scrolling unit 1116); detect (e.g., using the detection unit 1110) a seventh input specifying the start and end of the recorded media image sequence; in response to detecting the seventh input specifying the start and end of the recorded media image sequence, create (e.g., using the creation unit 1122) metadata that marks the recorded media image sequence as a unit of a first type of media; and in the multiple images, be able to display (e.g., using the display unit 1102) an indication of scrolling on the display unit 1102 (e.g., using the scrolling unit 1116) as a frame, the recorded media image sequence being marked as a unit of the first type of media.

[0334] In some implementations, copies of the recorded media image sequence are stored independently of the recorded media images.

[0335] In some implementations, copies of the recorded media image sequence are configured to be displayed at a different aspect ratio than the recorded media images.

[0336] In some implementations, the processing unit 1108 is further configured to: detect (e.g., using the detection unit 1110) an eighth input that activates the video viewing mode when displaying a representation of multiple images as frames scrolling on the display unit 1102 (e.g., using the scrolling unit 1116); and in response to detecting the eighth input that activates the video viewing mode, replace (e.g., using the replacement unit 1118) the display of the real-time view from the camera unit 1106 with a playback of a first recorded video clip.

[0337] In some embodiments, while the first video clip is played back, the device continues to record media images using camera unit 1106 (e.g., using recording unit 1112). 59. The electronic device according to claim 57 or 58, wherein processing unit 1108 is further configured to: detect (e.g., using detection unit 1110) a ninth input that disables video viewing mode while the first video clip is played back; and in response to detecting the ninth input that disables image viewing mode, replace (e.g., using replacement unit 1118) the playback of the first video clip with a live view from camera unit 1106.

[0338] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above relative to...). Figure 1A and Figure 3 (as described) or application-specific chips.

[0339] The above is for reference only. Figures 10A to 10E The operation is optionally performed by Figures 1A to 1B or Figure 11 The components depicted herein are used to implement this. For example, the recording operation in operation 1004 and the detection operation in operation 1006 are optionally implemented by the event classifier 170, the event recognizer 180, and the event handler 190. The event monitor 171 in the event classifier 170 detects a touch on the touch-sensitive display 112, and the event dispatcher module 174 transmits the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first touch at the first position on the touch-sensitive surface corresponds to a predefined event or sub-event, such as the selection of an object on the user interface. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes the data updater 176 or the object updater 177 to update the internal state 192 of the application. In some embodiments, event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will readily understand that based on... Figures 1A to 1B How the components described herein can implement other processes.

[0340] Figures 12A to 12DThis is a flowchart illustrating a method 1200 for capturing digital media in different imaging modes according to some embodiments. As described below, method 1200 involves automatically switching to one or more additional image recording modes at a subsequent time point when, in response to continued detection of input that has already activated a single image recording mode on the device, the input is continuously detected for a period exceeding a corresponding threshold amount to trigger one or more additional image recording modes. Additionally, a visual indication of the currently active media recording mode is also displayed. Automatically switching media recording modes based on continuous input satisfying different time thresholds at different time points and corresponding visual feedback enhances the operability of the device and makes the user-device interface more efficient (e.g., by performing mode switching operations based on satisfying preset conditions without requiring separate user input and without interrupting ongoing media recording). Furthermore, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0341] Method 1200 applies to electronic devices having one or more processors, memory, a camera (e.g., mounted on the back of the electronic device, away from the user), and a display (e.g., a touch-sensitive display). Figure 3 Device 300, or Figure 1A The operation is performed at a portable multi-functional device 100. In some embodiments, the electronic device includes a second camera (e.g., mounted on the front of the electronic device, facing the user). Some operations in method 1200 may be combined, and / or the order of some operations may be changed.

[0342] The device display (1202) displays the camera user interface (e.g., Figure 6A The user interface 600 in the camera interface includes a live view from the camera (e.g., including...). Figure 6A (A real-time view of branch 602 in the image). In some embodiments, the camera user interface also includes a power indicator for initiating image recording, one or more power indicators for setting operating modes (e.g., flash / illumination settings, dynamic range settings, and / or image filter settings), and a power indicator for switching between the (first) camera and the second camera.

[0343] When displaying a live view from the camera, the device makes a first contact (e.g., Figure 6B Contact 604 in Figure 6D Contact 608 in Figure 6G Contact 614 or Figure 6K The first contact (618) detects the start of the input (1204), which activates media recording with the camera (e.g., the device detects a tap on the live view or a power indication for initiating image capture (e.g., a power indication displayed on the live view from the camera)).

[0344] In response to the start of the detection of input for the first contact, which activates media recording using the camera, the device displays (1206) a first indication that the camera is in a first media recording mode corresponding to the recording of a single image (e.g., Figure 6B , Figure 6D , Figure 6G and Figure 6K (Ring 606 in the middle). In some embodiments, in response to the start of detecting the input of the first contact, the camera records a single image (e.g., device 100 acquires...). Figure 6B A single image of branch 602 in the image. In some embodiments, in response to the detection of the start of input of the first contact, the camera begins to capture media in its original format, which can be processed into various different types of media (e.g., SD video, HD video, images of different aspect ratios, resolutions, etc.), and if the device does not continue to detect the first contact beyond a predefined time threshold, a single image is retained in the memory (or external memory) of the electronic device (e.g., in the case of a tap, a single image is retained and all other media data is automatically deleted from memory after a predefined time period (e.g., after the input terminates) (e.g., device 100 begins recording). Figure 6G The walking of the cat in 610).

[0345] In some implementations, "captured" media refers to media automatically, temporarily, and in real-time stored by an electronic device without active user intervention. Media captured by a camera is automatically stored (e.g., in the memory of the electronic device) for a predetermined duration (e.g., while the camera user interface is active, or 24 hours after capture), and is automatically deleted or overwritten unless instructed by the user to mark the captured media for subsequent processing (e.g., tagging, editing, compressing, converting, etc.) and / or persistent storage. In some implementations, media marked for subsequent processing and / or persistent storage (e.g., based on user input or pre-configured device settings) is considered "recorded."

[0346] When the device detects (1208) the continuation of the input of the first contact (e.g., continued detection of the first contact) while displaying the first indication that the camera is in a first media recording mode (e.g., single image recording mode), the device detects (1208) the continuation of the input of the first contact (e.g., continued detection of the first contact). Figure 6E Contact 608 in Figure 6H Contact 614, or Figure 6L Contact 618 in the middle).

[0347] In response to the continuation of the input detecting the first contact, and before the termination of the input detection (e.g., continuing to detect the first contact) (1210), the continuation of the input is determined to satisfy at least a first predefined time threshold (e.g., the electronic device continuously detects the first contact for a period of time during which the first predefined time threshold is satisfied; for example, Figure 6E , Figure 6H and Figure 6L The time threshold TT in S (1212), the device displays (1214) a second indication (e.g., in...). Figure 6E , Figure 6H and Figure 6L A counter marker 612 surrounding the detected contact location indicates that the camera is in a second media recording mode, which corresponds to a sequence of images that continue to be recorded simultaneously with the input of the first contact (e.g., continuing to detect the first contact (e.g., where the camera stops capturing images when the first contact is lifted off the ground)). In some embodiments, in the second image recording mode, the camera records a burst of images, each image being temporally separated by a predefined time period longer than the minimum time period that defines the video frame rate at which the camera is configured to capture (e.g., device 100 records a cat 610 in...). Figure 6E (Buzzer shot of photos entering the field of view). In some implementations, while in a second image recording mode, the camera records multiple images (e.g., video) in a configuration compatible with multiple different recording modes, but if the device does not continue to detect a first contact exceeding a second time threshold, some (less than all) of the multiple images are marked to be retained in the electronic device's memory (or external memory) (e.g., in cases where the input is longer than a tap but shorter than a long press, the images are retained in a burst, and all other images are automatically deleted from memory after the input ends; for example, when a tap is detected...). Figure 6F (When contact 608 is lifted away).

[0348] When the device displays a second indication that the camera is in a second media recording mode (e.g., continuous image recording mode), it detects (1216) further continuation of the input of the first contact (e.g., continued detection of the first contact).

[0349] In response to further continuation of the input detecting the first contact, and before the termination of the input detection (e.g., continue detecting the first contact) (1218) (e.g., continue detecting) Figure 6I Contact 614 or Figure 6M (Contact 618) is determined based on at least the further satisfaction of a second predefined time threshold (e.g., the electronic device has continuously detected the first contact for a period of time that satisfies a second predefined time threshold longer than the first predefined time threshold; for example, ...). Figure 6I and Figure 6M The time threshold TT inM The device displays (1220) a third instruction (e.g., Figure 6I and Figure 6M The symbol (marked as ring 616) indicates that the camera is in a third media recording mode corresponding to video recording. In some embodiments, the camera continues recording video upon input termination (e.g., removal of the first contact) and continues recording video until an input terminating video recording is detected (e.g., device 100 detects...). Figure 6P Video recording continued as contact 618 was lifted away. During the detection... Figure 6R When contact 624 occurs, the equipment is in Figure 6S Recording stops during this process. In some implementations, when the third image recording mode is activated, the camera remains on the video that began when the input met a second time threshold (e.g., before the second time threshold is reached, the camera records a single image or only retains some individual images from all images captured before the second time threshold is reached; for example, in...). Figure 6I The continued testing of videos recorded after contact 614 included footage of cat 610 walking and lying down under tree branch 602, such as... Figures 6G to 6H As shown, the video is captured before the device 100 enters the third media recording mode. In some implementations, when the third image recording mode is activated, the camera retains the video that began when the first contact in the input was initially detected.

[0350] In some implementations, in response to the continuation of the input detecting the first contact, and before the termination of the input detection (e.g., continuing to detect the first contact) (1210) (e.g., continuing to detect) Figure 6E Contact 608 in Figure 6H Contact 614, or Figure 6L (Contact 618) is determined based on at least the continued satisfaction of a first predefined time threshold (e.g., the electronic device continuously detects the first contact within a time period that satisfies the first predefined time threshold; for example, ...). Figure 6E , Figure 6H and Figure 6L The time threshold TT in S (1212), in response to further input of the first contact being detected, and before the detection input terminates (e.g., continue detecting the first contact) (1218), and based on the determination that further continuation of the input does not satisfy a second predefined time threshold (e.g., Figure 6E , Figure 6H and Figure 6L The time threshold TT in M The device maintains (1222) a second indication that the camera is in a second media recording mode (e.g., around the device in...). Figure 6E , Figure 6H and Figure 6L(Counting marker 612 indicating the location of contact detected in the middle).

[0351] In some implementations, in response to the continuation of the input detecting the first contact, and before the termination of the input detection (e.g., continuing to detect the first contact) (1210) (e.g., continuing to detect) Figure 6B Contact 604 in Figure 6D Contact 608 in Figure 6G Contact 614, or Figure 6K Contact 618 in the middle), based on the determination of the continued satisfaction of the first predefined time threshold of the input (e.g., Figure 6B , Figure 6D , Figure 6G and Figure 6K The time threshold (TTS) in the device maintains (1224) the camera in the first media recording mode (e.g., Figure 6B , Figure 6D , Figure 6G and Figure 6K The display of ring 606 in the middle.

[0352] In some implementations, in response to the start of the detection of the input of the first contact, the first contact activates media recording using the camera, and with a configuration compatible with multiple media recording modes including the first, second and third media recording modes, the device begins (1226) to continue capturing media images (e.g., capturing and temporarily storing video captured at maximum resolution, frame rate and size and aspect ratio).

[0353] In some implementations, when displaying a corresponding one of the first, second, and third indications corresponding to a corresponding one of the first, second, and third media recording modes, the device detects (1228) the termination of the input of the first contact (e.g., detecting the lifting of the first contact; e.g., Figure 6C Contact 604 in Figure 6F Contact 608 or Figure 6J The device retrieves media of the first type (e.g., a single image, a burst of images, or a video) from continuously captured media images, corresponding to one of the first, second, and third media recording modes, and persistently stores the retrieved first type of media (e.g., persistently storing a single image, a burst of images, or a video, and discarding the remaining media image data). In some embodiments, the device also stops displaying the corresponding one of the first, second, and third indications. In some embodiments, the termination of the input only terminates recording and triggers the output of the first type of media (e.g., in the case of the lifting off of contact 614 in the first contact) if the input is terminated before the time for activating the continuous video recording mode has expired. Figure 6Q The third threshold TTL Subsequent contact with 618 will not terminate image recording. Based on the activated media recording mode, automatically storing the appropriate type of media upon input termination enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to record the desired type of media). Additionally, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0354] In some implementations, the input is terminated in response to the detection of the first contact (e.g., Figure 6C Contact 604 in Figure 6F Contact 608 or Figure 6J (Uplift of contact 614) stops the device (1230) from continuously capturing media images with a configuration compatible with multiple media recording modes. In some implementations, the termination of input only terminates continuous capture of media images if the input is terminated before the time for activating the continuous video recording mode has expired (e.g., in...). Figure 6Q The third threshold TT L The subsequent lifting of contact 618 will not terminate image recording. Automatically stopping media recording when input terminates enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of input required to stop recording at the desired time). Additionally, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0355] In some implementations, when the camera is in a first media recording mode, the termination of the first contact input (1232) is detected (e.g., single image recording mode; for example, Figure 6C The first type of media includes a single image with a first set of media attributes (e.g., a first size, aspect ratio, and resolution of the photograph, which differ from the media attribute set of the captured multimodal media). For example, cropping, downsampling, and / or shrinking frames from the multimodal media can generate a single image with the first set of media attributes. Automatically storing the appropriate type of media (e.g., a single image) upon input termination, based on the activated media recording mode, enhances device operability and makes the user-device interface more efficient (e.g., by performing operations based on satisfied preset conditions). Furthermore, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0356] In some implementations, when the camera is in a second media recording mode, the termination of the first contact input (1234) is detected (e.g., single burst recording mode; e.g., Figure 6FThe first type of media includes two or more image sequences with a second set of media properties (e.g., a first size, aspect ratio, resolution, and frame rate of a photo sequence that differ from the media property set of the captured multimodal media). For example, cropping, downsampling, and / or shrinking two or more frames from the multimodal media to generate an image sequence with a second set of media properties (e.g., the frame rate of the image sequence is lower than the frame rate of the captured multimodal media and lower than the video that can be extracted from the multimodal media). In some embodiments, the two or more image sequences begin at or near the time when the first input is detected (e.g., before or after the first input is detected). Automatically storing the appropriate type of media (e.g., an image sequence in a burst of images) upon termination of input based on the activated media recording mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by performing operations based on the fulfillment of preset conditions). Additionally, this can reduce power consumption and extend the battery life of the device by enabling users to use the device more quickly and efficiently.

[0357] In some implementations, the two or more image sequences include (1236) at least one image captured before the camera enters the second media recording mode (e.g., in...). Figure 6F In the process, the image sequence stored in response to the lifting and removal of the detection contact 608 includes images stored before the device enters the second media recording mode. Figure 6D Images captured without cats (610), such as Figure 6E (As shown). Including images captured before the second media recording mode into the image sequence stored in the second media recording mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the number of steps required to include media content captured in media items created earlier in the subsequently activated media recording mode). In addition, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0358] In some implementations, when the camera is in third media recording mode, the termination of the first contact input (1238) is detected (e.g., video recording mode; e.g., ...). Figure 6JThe first type of media includes video with a third set of media attributes (e.g., the first size, aspect ratio, resolution, and frame rate of a photograph, which differ from the media attribute set of the captured multimodal media). For example, cropping, downsampling, and / or shrinking the frame sequence from the multimodal media to generate video with a third set of media attributes (e.g., the frame rate of the video is lower than or equal to the frame rate of the captured multimodal media, and higher than the burst of images that can be extracted from the multimodal media). Automatically storing the appropriate type of media (e.g., video) upon input termination based on the activated media recording mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by performing operations based on satisfying preset conditions). In addition, this can reduce power consumption and extend the battery life of the device by enabling users to use the device more quickly and efficiently.

[0359] In some embodiments, the video includes (1240) at least one frame captured before the camera enters the third media recording mode. In some embodiments, the video begins at or near the time when the first input is detected (e.g., slightly before or after the start of the first input detection). Figure 6J The video stored in response to the detection of the lifting of contact 614 includes images of a cat 610 walking under branch 602, such as... Figure 6G The captured footage, before the device entered third-media recording mode, such as Figure 6I (As shown). Including images captured before the third media recording mode into the video stored in the third media recording mode enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the number of steps required to include media content captured in media items created earlier in the subsequently activated media recording mode). In addition, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0360] In some implementations, the first indication that the camera is in a first media recording mode includes (1242) displaying a recording capability indication having a first appearance (e.g., Figure 6B , Figure 6D , Figure 6G and Figure 6K (Ring 606 in the middle); the second indication that the camera is in a second media recording mode includes displaying a recording capability indicator with a second appearance different from the first appearance (e.g., the first appearance is a circular button, the second appearance includes a series of scale marks around the circular button and / or a burst count indicator above the circular button, for example, surrounding...). Figure 6E , Figure 6H and Figure 6LThe third indication that the camera is in a third media recording mode includes displaying a recording capability indicator with a third appearance different from the first and second appearances (e.g., the first appearance is a circular shutter button, the second appearance includes a series of scale marks around the circular shutter button and / or a burst count indicator above the circular shutter button, and the third appearance includes a circular record button with a series of more closely spaced scale marks around the circular record button and / or a recording duration indicator above the circular record button; for example...). Figure 6I and Figure 6M (Ring 616 marked in the middle). Displaying different indicators for different media recording modes enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). In addition, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0361] In some implementations, a first indication that the camera is in a first media recording mode includes displaying a live view from the camera with first visual features (e.g., a first aspect ratio, size, and / or resolution); and a third indication that the camera is in a third media recording mode includes displaying a live view from the camera with second visual features (e.g., a second aspect ratio, size, and / or resolution) different from the first visual features. Displaying indications with different visual features for different media recording modes enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0362] In some implementations, a first indication that the camera is in a first media recording mode includes (1246) displaying a mode indicator (e.g., a toggle switch for camera status) in the first state (e.g., a video indicator in an inactive state); a third indication that the camera is in a third media recording mode includes displaying a mode indicator (e.g., a toggle switch for video status) in the second state (e.g., a video indicator in an inactive state); and, upon detection of continued input from the first contact, the device displays an animated transition from the mode indicator in the first state to the mode indicator in the second state (e.g., the toggle switch of the mode selector gradually expands to cover the video icon) (e.g., the video indicator gradually changes from an inactive state to an active state). Displaying animated transitions when the device transitions between different media recording modes enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0363] In some implementations, when displaying a third indication corresponding to the third media recording mode, the device detects (1248) the termination of the first contact input (e.g., detecting the lifting of the first contact; for example, Figure 6Q (The lifting of contact 618 in the middle). Additionally, in response to the termination of the input detecting the first contact, a third predefined time threshold is determined based on the duration of the input (e.g., the device continuously detects the first contact for a period exceeding the third threshold amount, which is longer than the threshold amount required to enter the video recording mode; for example...). Figure 6O The time threshold TT in L The device continues to capture media images using a configuration compatible with multiple media recording modes until a termination input different from the first input is detected; upon detection of the termination input, a first video is acquired from the continuously captured media (e.g., the first video includes frames captured throughout the entire duration from the initial detection of the first contact input to the detection of the termination input); the acquired first video is persistently stored (e.g., the video is persistently stored, and the remaining media image data is discarded; for example, ...). Figure 6R The video recording was stopped by a tap gesture that touched 624, such as... Figure 6S (As shown). The determination is based on the duration of the input not meeting a third predefined time threshold (e.g., the device continuously detects a first contact for a time exceeding a second threshold to enter video recording mode, but for a time not exceeding a third threshold to activate continuous video recording mode; for example...). Figure 6J(Upon lifting contact 614), the device stops the continued capture of media images using a configuration compatible with multiple media recording modes; acquires a second video from the continuously captured media (e.g., the second video includes frames captured throughout the entire duration of the first contact input); and persistently stores the acquired second video (e.g., persistently stores the video and discards the remaining media image data).

[0364] Entering continuous recording mode when input is sustained for a sufficient amount of time, and requiring separate input to terminate video recording after entering continuous recording mode, enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to start continuous recording mode and providing users with more precise control over when to stop continuous recording). Additionally, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0365] It should be understood that, Figures 12A-12D The specific order in which the operations described herein are presented is merely an example and is not intended to indicate that this order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that the details of other processes described herein relative to other methods (e.g., methods 1000, 1400, 1600, and 1800) also apply in a similar manner to the above descriptions relative to other methods described herein. Figures 12A to 12D The method described above with reference to method 1200 may optionally have one or more features of the touch, gesture, user interface object, various types of thresholds, focus selector, live view, frame stream, captured media content, media item and live view display object, power representation and animation described herein with reference to other methods described herein (e.g., method 1000, method 1400, method 1600 and method 1800). For the sake of brevity, these details will not be repeated here.

[0366] According to some implementation plans Figure 13 A functional block diagram of an electronic device 1300 configured according to the principles of various embodiments is shown. The functional blocks of this device are optionally implemented by hardware, software, or a combination of hardware and software that execute the principles of the various embodiments. Those skilled in the art will understand that... Figure 13 The functional blocks described herein may optionally be combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0367] like Figure 13 As shown, the electronic device 1300 includes: a display unit 1302 configured to display a user interface; a camera unit 1306 configured to capture media; and a processing unit 1308 coupled to the display unit 1302 and the camera unit 1306. In some embodiments, the processing unit includes a detection unit 1310, a capture unit 1312, an acquisition unit 1314, and a storage unit 1316.

[0368] Processing unit 1308 is configured to: display (e.g., using display unit 1302) a camera user interface on a display, the camera user interface including a live view from camera unit 1306; while displaying the live view from camera unit 1306, detect the start of input via a first contact (e.g., using detection unit 1310), the first contact activating media recording using camera unit 1306; in response to the start of the first contact input, the first contact activating media recording using camera unit 1306, display (e.g., using display unit 1302) a first indication that camera unit 1306 is in a first media recording mode corresponding to the recording of a single image; while displaying the first indication that camera unit 1306 is in the first media recording mode, continue detecting (e.g., using detection unit 1310) the input of the first contact; in response to the detection Detecting the continuation of the first contact input, and before the termination of the input detection: based on at least determining that the continuation of the input satisfies a first predefined time threshold: being able to display (e.g., using display unit 1302) a second indication that the camera unit 1306 is in a second media recording mode, the second media recording mode corresponding to an image sequence recorded simultaneously with the continuation of the first contact input; while displaying the second indication that the camera unit 1306 is in the second media recording mode, detecting (e.g., using detection unit 1310) a further continuation of the first contact input; in response to detecting a further continuation of the first contact input, and before the termination of the input detection: based on at least determining that the further continuation of the input satisfies a second predefined time threshold, being able to display (e.g., using display unit 1302) a third indication that the camera unit 1306 is in a third media recording mode corresponding to video recording.

[0369] In some implementations, the processing unit 1308 is further configured to: in response to the detection of continued input of the first contact and before the detection of the input terminates: based on at least determining that the continuation of the input satisfies a first predefined time threshold; in response to the detection of further continuation of input of the first contact and before the detection of the input terminates: based on the determination that the further continuation of the input does not satisfy a second predefined time threshold, maintain a second indication (e.g., using the display unit 1302) that the camera unit 1306 is in a second media recording mode.

[0370] In some implementations, the processing unit 1308 is further configured to: in response to the continued detection of the first contact input and before the detection of the input terminates: based on the determination that the continued input does not meet a first predefined time threshold, maintain the display (e.g., using the display unit 1302) a first indication that the camera unit 1306 is in a first media recording mode.

[0371] In some implementations, the processing unit 1308 is also configured to: in response to the start of input of a first contact, which activates media recording using the camera unit 1306, stop further capture of media images using a configuration compatible with multiple media recording modes including first, second, and third media recording modes (e.g., using the capture unit 1312).

[0372] In some embodiments, the processing unit 1308 is further configured to: detect (e.g., using the detection unit 1310) the termination of input via the first contact when displaying a corresponding one of the first, second, and third indications corresponding to a corresponding one of the first, second, and third media recording modes; and in response to detecting the termination of input via the first contact: acquire (e.g., using the acquisition unit 1314) media of a first type corresponding to a corresponding one of the first, second, and third media recording modes from continuously captured media images; and persistently store (e.g., using the storage unit 1316) the acquired media of the first type.

[0373] In some implementations, the processing unit 1308 is also configured to stop further capture of media images (e.g., using the capture unit 1312) in response to the termination of the input of the first contact.

[0374] In some implementations, when the camera unit 1306 is in a first media recording mode, the termination of the first contact input is detected; and the first type of media includes a single image having a first set of media attributes.

[0375] In some implementations, when the camera unit 1306 is in a second media recording mode, the termination of the first contact input is detected; and the first type of media includes a sequence of two or more images having a second set of media attributes.

[0376] In some implementations, the two or more image sequences include at least one image captured before the camera unit 1306 enters the second media recording mode.

[0377] In some implementations, when the camera unit 1306 is in the third media recording mode, the termination of the first contact input is detected; and the first type of media includes video with a third set of media attributes.

[0378] In some implementations, the video includes at least one frame captured before the camera unit 1306 enters the third media recording mode.

[0379] In some embodiments, a first indication that the display camera unit 1306 is in a first media recording mode includes displaying (e.g., using display unit 1302) a recording capability indication having a first appearance; a second indication that the display camera unit 1306 is in a second media recording mode includes displaying (e.g., using display unit 1302) a recording capability indication having a second appearance different from the first appearance; and a third indication that the display camera unit 1306 is in a third media recording mode includes displaying (e.g., using display unit 1302) a recording capability indication having a third appearance different from the first and second appearances.

[0380] In some embodiments, a first indication that the camera unit 1306 is in a first media recording mode includes displaying (e.g., using display unit 1302) a live view from the camera unit 1306 using a first visual feature; and a third indication that the camera unit 1306 is in a third media recording mode includes displaying (e.g., using display unit 1302) a live view from the camera unit 1306 using a second visual feature different from the first visual feature.

[0381] In some embodiments, a first indication that the camera unit 1306 is in a first media recording mode includes displaying (e.g., using the display unit 1302) a mode indicator in a first state; a third indication that the camera unit 1306 is in a third media recording mode includes displaying (e.g., using the display unit 1302) a mode indicator in a second state; and the processing unit 1308 is further configured to display (e.g., using the display unit 1302) an animated transition from the mode indicator in the first state to the mode indicator in the second state when the input of the first contact is detected to continue.

[0382] In some embodiments, the processing unit 1308 is further configured to: detect (e.g., using detection unit 1310) the termination of the first contact input when a third indication corresponding to a third media recording mode is displayed; and in response to detecting the termination of the first contact input: based on the determination that the duration of the input meets a third predefined time threshold: continue capturing media images using a configuration compatible with multiple media recording modes (e.g., using capture unit 1312) until a termination input different from the first input is detected; acquire (e.g., using acquisition unit 1314) a first video from continuously captured media upon detecting the termination input; and persistently store (e.g., using storage unit 1316) the acquired first video; and based on the determination that the duration of the input does not meet the third predefined time threshold: stop the continued capture of media images using a configuration compatible with multiple media recording modes (e.g., using capture unit 1312); acquire (e.g., using acquisition unit 1314) a second video from continuously captured media; and persistently store (e.g., using storage unit 1316) the acquired second video.

[0383] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (e.g., as described above relative to...). Figure 1A and Figure 3 (as described) or application-specific chips.

[0384] The above is for reference only. Figures 12A to 12D The operation is optionally performed by Figures 1A to 1B or Figure 13 The components depicted are used to implement this. For example, detection operations 1204, determination operations 1212, 1220, 1222, and 1224 are optionally implemented by an event classifier 170, an event recognizer 180, and an event handler 190. An event monitor 171 in the event classifier 170 detects a touch on the touch-sensitive display 112, and an event dispatcher module 174 transmits the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 compares the event information with a corresponding event definition 186 and determines whether a first touch at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as a selection of an object on the user interface. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates the event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally utilizes or invokes a data updater 176 or an object updater 177 to update the application's internal state 192. In some embodiments, event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will readily understand that based on... Figures 1A to 1B How the components described herein can implement other processes.

[0385] Figures 14A to 14F This is a flowchart illustrating a method 1400 for switching between camera live views during image capture according to some embodiments. As described below, method 1400 involves manipulating a user interface object currently displaying a live view from a first camera via touch movement, such that a live view from a second camera is displayed in the user interface object. Additionally, both the live views from the first and second cameras move according to the touch movement. Switching from the second camera to the live view in response to a touch movement detected while displaying the live view from the first camera, and moving the live views from both cameras according to the touch movement, allows the user to better control the speed and extent to which the live views from different cameras move within the user interface. This additional control over the speed and extent to which the live views from different cameras move and / or switch within the user interface enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). Furthermore, this can reduce power consumption and extend the device's battery life by enabling the user to use the device more quickly and efficiently.

[0386] Method 1400 is used in electronic devices having a display, a touch-sensitive surface, and one or more cameras (e.g., Figure 3 Equipment 300, or Figure 1A The method is performed at a portable multi-functional device 100. In some embodiments, the touch-sensitive surface and the display are integrated into a touch-sensitive display. In some embodiments, the display is a touchscreen display, and the touch-sensitive surface is on or integrated with the display. In some embodiments, the display and the touch-sensitive surface are separate. Some operations in method 1400 are optionally combined, and / or the order of some operations is optionally changed.

[0387] The device displays (1402) a first live view from the first camera of the electronic device in the first display area of ​​the camera user interface (e.g., Figure 8A (Live view 804 in the image). For example, in some embodiments, the first display area is a live viewer that includes a first live view. In some embodiments, the first display area does not simultaneously include a second live view captured by a second camera of the electronic device (e.g., the first camera is a front-facing camera and the second camera is a rear-facing camera, or vice versa). In some embodiments, the camera user interface includes a power indicator for activating or deactivating one or more media recording modes.

[0388] When a first live view from a first camera is displayed in the first display area of ​​the camera user interface, the device detects movement of a first contact on the touch-sensitive surface (1404) (e.g., detecting a first swipe gesture on the touch-sensitive display in the first live view in the first display area) (e.g., no first contact is detected at any location corresponding to a back-and-forth switching capability for switching between the front and rear cameras). For example, the detected first contact is Figures 8A to 8D Contact 808 in Figures 8F to 8G Contact 812 in Figures 8H to 8I Contact 814 in Figures 8J to 8M Contact 816 in Figures 80 to 8P Contact 818 in Figures 8U to 8Y Contact 822 or Figures 8Z to 8AA Contact 824 in the middle.

[0389] In response to detecting movement of the first contact on the touch-sensitive surface, and based on the determination that the movement of the first contact satisfies a first movement criterion (e.g., the first movement criterion requires the movement of the first contact to be in a first direction (e.g., horizontal)), the device moves (1406) a first real-time view in a first display area according to the movement of the first contact on the touch-sensitive surface (e.g., the first real-time view is displayed in front of a dual-user interface object, and the dual-user interface object rotates about an axis in the display plane according to the movement of the first contact) (e.g., the first real-time view moves horizontally according to the movement of the first contact, such that a portion of the first real-time view moves outside the first display area and is no longer visible to the user). This is in Figures 8J to 8N (For example, the real-time view 804 rotates as the contact 816 moves.) Figures 80 to 8P (For example, the real-time view 804 rotates as the contact 818 moves.) Figures 8T to 8X (For example, the live view 820 moves to the right as the contact 822 moves.) Figures 8Z to 8AAAs shown in the example (e.g., the live view 820 moves to the right as the contact 824 moves), the device displays a second live view from a second camera of the electronic device in the first display area (e.g., the second live view is displayed behind the dual-user interface object; and when the dual-user interface object is completely flipped according to the movement of the first contact (e.g., rotated more than 90 degrees about an axis in the display plane), the first live view is no longer displayed in the first display area, and the second live view is displayed in the first display area) (e.g., when the first live view moves horizontally according to the movement of the first contact, the second live view moves into the space vacated by the first live view). Furthermore, the device moves the second live view in the first display area according to the movement of the first contact on the touch-sensitive surface (e.g., after the rear side of the dual-user interface object (with the second live view) becomes visible in the first display area, the dual-user interface object rotates about an axis in the display plane according to the movement of the first contact until the rear side of the dual-user interface object (with the second live view) has reached the front position of the first display area) (e.g., when the first and second live views are continuously horizontally shifted according to the movement of the first contact, the second live view eventually occupies the entire first display area and the first live view is no longer visible to the user). This in Figures 80 to 8T (For example, the real-time view 820 rotates due to the simulated inertia caused by the movement of the contact 818.) Figures 8T to 8X (For example, the real-time view 804 moves to the right as the contact 822 moves.) Figures 8Z to 8AA (For example, the real-time view 804 moves to the right due to the simulated inertia caused by the movement of contact 824) is shown, for example.

[0390] In some implementations, a first live view (1408) is displayed on a first side of a multi-sided user interface object (e.g., a dual-sided user interface object); a second live view is displayed on a second side of the multi-sided user interface object; and moving the first and second live views according to movement of a first contact on a touch-sensitive display includes rotating the multi-sided user interface object according to movement of the first contact on the touch-sensitive display. For example, this is in Figures 8J to 8T As shown in the accompanying description, switching between live views from different cameras by rotating multi-sided user interface objects enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0391] In some implementations, the movement of the first contact includes (1410) a first portion of movement along a first direction (e.g., from left to right) and a second portion of movement along a second direction opposite to the first direction (e.g., from right to left); and rotating the multi-sided user interface object according to the movement of the first contact on the touch-sensitive display includes rotating the multi-sided user interface object in the first direction in response to the first portion of movement; and rotating the multi-sided user interface object in the second direction in response to the second portion of movement. For example, a user can begin to flip the multi-sided user interface object to view a second live view by swiping from left to right, and then reverse the swipe direction to return to the first live view before lifting the contact. Allowing the user to reverse the rotation of the multi-sided user interface object enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). In addition, this can also reduce power consumption and extend the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0392] In some implementations, after rotating the multi-sided user interface object according to a first contact on the touch-sensitive display, the device detects (1412) the lifting off of the first contact; and in response to detecting the lifting off of the first contact, based on the determination that the movement of the first contact satisfies a second movement criterion (e.g., movement exceeding a threshold distance, a threshold speed, and / or a threshold duration), the device continues to rotate the multi-sided user interface object after the lifting off of the first contact until the second side of the multi-sided user interface object occupies the entire first display area. For example, this is in Figures 80 to 8T As shown in the accompanying description, automatically capturing a second real-time view in response to rotation / movement that meets predetermined movement criteria enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input and lowering the accuracy requirements of user input). In addition, this can also reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0393] In some implementations, in response to detecting the lifting off of the first contact, and based on the determination that the movement of the first contact meets a third movement criterion (e.g., the movement does not exceed a threshold distance, a threshold speed, and / or a threshold duration), the device reverses (1414) the rotation of the multi-sided user interface object after the lifting off of the first contact until the first side of the multi-sided user interface object once again occupies the entire first display area. For example, this is in Figures 8J to 8NAs shown in the accompanying description, automatically returning to the first real-time view after rotation / movement terminates upon meeting predetermined movement criteria enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input and lowering the accuracy requirements of user input). Additionally, this reduces power consumption and extends the device's battery life by enabling users to use the device more quickly and efficiently.

[0394] In some implementations, a first live view (1416) is displayed in a first panel of a multi-pane user interface object (e.g., a dual-pane user interface object); a second live view is displayed in a second panel of the multi-pane user interface object; and moving the first and second live views according to the movement of a first contact on a touch-sensitive display includes sliding the multi-pane user interface object in a first display area according to the movement of the first contact on the touch-sensitive display. For example, when a user horizontally slides a dual-pane user interface object, the first live view gradually moves out of the first display area, and the second live view gradually moves into the first display area and eventually occupies the entire first display area. For example, this is in Figures 8U to 8Y As shown in the accompanying description, automatically segmenting user interface objects to display live views from both cameras enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to display live views from both cameras and providing more precise control over the relative size of the two live views). Additionally, this reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0395] In some implementations, the movement of the first contact includes (1418) a first portion of movement along a first direction and a second portion of movement in a second direction opposite to the first direction, and sliding the multi-panel user interface object according to the movement of the first contact on the touch-sensitive display includes sliding the multi-panel user interface object in the first direction in response to the first portion of movement; and sliding the multi-panel user interface object in the second direction in response to the second portion of movement. For example, this is in Figure 8W and Figure 8X As shown in the accompanying description, allowing users to reverse the swiping of multi-panel user interface objects enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0396] In some implementations, after sliding a multi-panel user interface object according to the movement of a first contact on a touch-sensitive display, the device detects (1420) the lifting off of the first contact; and in response to detecting the lifting off of the first contact, based on the determination that the movement of the first contact satisfies a fourth movement criterion (e.g., movement exceeding a threshold distance, a threshold speed, and / or a threshold duration), the device continues to slide the multi-panel user interface object after the lifting off of the first contact until the second panel of the multi-panel user interface object occupies the entire first display area. For example, this is in Figure 8Z and Figure 8AA As shown in the accompanying description, automatically capturing a second real-time view in response to movement that meets predetermined movement criteria enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input and lowering the accuracy requirements of user input). In addition, this can also reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0397] In some implementations, in response to detecting the lifting off of the first contact, and based on the determination that the movement of the first contact meets a fifth movement criterion (e.g., the movement does not exceed a threshold distance, a threshold speed, and / or a threshold duration), the device reverses (1422) the sliding of the multi-sided user interface object after the lifting off of the first contact until the first panel of the multi-panel user interface object once again occupies the entire first display area. Automatically restoring to the first real-time view after the movement terminates upon meeting the predetermined movement criteria enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input and lowering the accuracy requirements of user input). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0398] In some embodiments, after sliding a multi-panel user interface object based on movement of a first contact on a touch-sensitive display, the device detects (1424) the lifting of the first contact; and, in response to detecting the lifting of the first contact, stops sliding the multi-panel user interface object after the first contact is lifted. In some embodiments, the first display area simultaneously displays a portion of a first panel of the multi-panel user interface object and a portion of a second panel of the multi-panel user interface object. For example, this is in Figure 8Y As shown in the accompanying description, splitting the user interface objects to display live views of two cameras from one location based on the lifting and lowering of touch in swipe input enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to display live views from both cameras and providing more precise control over the relative size of the two live views). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0399] In some embodiments, the device detects (1426) the lifting of the first contact. In some embodiments, after the first contact is lifted, based on the determination that the movement of the first contact satisfies a sixth movement criterion (e.g., the first swipe gesture is a fast swipe with a fast speed (e.g., a speed greater than a threshold speed), a short distance (e.g., a distance less than a threshold distance), and / or a short duration (e.g., a duration less than a threshold duration), the first live view stops displaying in the first display area, and the second live view occupies the entire first display area; and based on the determination that the movement of the first contact satisfies a seventh movement criterion (e.g., the first swipe gesture is a slow swipe with a slow speed (e.g., a speed not greater than a threshold speed), a long distance (e.g., a distance not less than a threshold distance), and / or a long duration (e.g., a duration not less than a threshold duration), the first display area is divided between displaying a portion of the first live view and a portion of the second live view. For example, this is in Figures 8U to 8AA As shown in the accompanying description. Figures 8U to 8Y In response to a slow swipe gesture on touch 822, the live view display objects are split between live view 804 and live view 820, and... Figures 8Z to 8AA In response to a quick flick gesture, the live view display object snaps to a position where only one live view is displayed. Determining whether to snap to another live view or split the user interface object between two live views based on which motion criteria the input meets at the time of input lift-off enhances device operability and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to display the desired live view and providing more precise control over the relative size of the live view). Additionally, this can reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0400] In some implementations, movement of the first contact is detected (1428) while video recording (or media capture) via the first camera is in progress (e.g., causing the recorded video to display a first live view before the movement of the first contact is detected, and displaying movement of the first live view and movement of a second live view displayed in response to the movement of the first contact). Continuing to record media when input for switching the live view is detected enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to switch the live view and continue recording media). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0401] In some implementations, the device continues (1430) recording video (or continues capturing media) after detecting movement of the first contact on the touch-sensitive display (e.g., such that the recorded video displays a first live view before the movement of the first contact is detected, and displays movement of the first live view and movement of a second live view displayed in response to the movement of the first contact). Continuing to record media while detecting input for switching the live view enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to switch the live view and continue recording media). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0402] In some implementations, continuing video recording after detecting movement of the first contact on the touch-sensitive display includes (1432) continuing to record video captured by the first camera while moving the first live view and the second live view according to the movement of the first contact (e.g., such that the recorded video shows the first live view before the movement of the first contact is detected, and shows the movement of the first live view and the movement of the second live view displayed in response to the movement of the first contact). Continuing to record media captured by the first camera while switching live views enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the amount of user input required to switch live views and continue recording media from the camera that provides the initial live view). Additionally, this can reduce power consumption and extend the device's battery life by enabling users to use the device more quickly and efficiently.

[0403] In some implementations, continuing video recording after detecting movement of the first touch on the touch-sensitive display includes (1434) stopping recording of video captured by the first camera when the first live view captured by the first camera is no longer displayed in the first display area; and starting recording of video captured by the second camera when the second live view captured by the second camera is displayed in the first display area. Switching the recording camera based on real-time switching of camera views enhances the operability of the device and makes the user-device interface more efficient (e.g., by changing the recording camera by reducing the amount of user input). In addition, this can reduce power consumption and extend the battery l...

Claims

1. A method for displaying a camera user interface, comprising: In electronic devices that have one or more processors, memory, cameras, and displays: A camera user interface is displayed on the monitor, the camera user interface including a live view from the camera; While displaying the live view from the camera, the start of input of the first contact is detected, the first contact activating the recording of media using the camera; In response to the detection of the start of the input that activates the first contact for recording media using the camera, a first indication is displayed that the camera is in a first media recording mode corresponding to the recording of a single image; When the first indication that the camera is in the first media recording mode is displayed, the input of the first contact is detected to continue; In response to the detection of the input of the first contact, the continuation, and before the detection of the termination of the input: Based on at least the determination that the input continues to satisfy a first predefined time threshold: A second indication shows that the camera is in a second media recording mode, which corresponds to the recording of an image sequence that continues simultaneously with the input from the first contact; When the second indication that the camera is in the second media recording mode is displayed, further continuation of detecting the input of the first contact; In response to the further continuation of the input upon detecting the first contact, and prior to the termination of the input: Based on the determination that the input further continues to satisfy a second predefined time threshold, a third indication is displayed that the camera is in a third media recording mode corresponding to video recording.

2. The method according to claim 1, comprising: In response to the detection of the input of the first contact, the continuation, and before the detection of the termination of the input: Based on at least determining that the input continues to satisfy a first predefined time threshold: In response to the further continuation of the input upon detecting the first contact, and prior to the termination of the input detection: If the input is determined to continue without satisfying the second predefined time threshold, the second indication that the camera is in the second media recording mode is maintained.

3. The method according to claim 2, comprising: In response to the detection of the input of the first contact, the continuation, and before the detection of the termination of the input: Based on the determination that the input does not meet the first predefined time threshold, the first indication that the camera is in the first media recording mode is maintained.

4. The method according to any one of claims 1 to 3, comprising: In response to the detection of the start of the input that activates the first contact for recording media using the camera, continuous capture of media images begins with a configuration compatible with multiple media recording modes including the first media recording mode, the second media recording mode, and the third media recording mode.

5. The method according to claim 4, comprising: When displaying the corresponding one of the first indication, the second indication, and the third indication corresponding to the first media recording mode, the second media recording mode, and the third media recording mode, the termination of the input of the first contact is detected; as well as Termination in response to the detection of the input of the first contact: Obtain media of a first type corresponding to one of the first media recording mode, the second media recording mode, and the third media recording mode from continuously captured media images; as well as The first type of media acquired through persistent storage.

6. The method of claim 5, comprising: In response to the termination of the input detected by the first contact, the continuous capture of media images using the configuration compatible with multiple media recording modes is stopped.

7. The method according to claim 5, wherein: When the camera is in the first media recording mode, the termination of the input of the first contact is detected; and The first type of media includes a single image having a first set of media attributes.

8. The method according to claim 5, wherein: When the camera is in the second media recording mode, the termination of the input of the first contact is detected; and The first type of media includes a sequence of two or more images having a second set of media attributes.

9. The method of claim 8, wherein the sequence of two or more images includes at least one image that was captured before the camera entered the second media recording mode.

10. The method according to claim 5, wherein: When the camera is in the third media recording mode, the termination of the input of the first contact is detected; and The first type of media includes video with the attributes of the third group of media.

11. The method of claim 10, wherein the video includes at least one frame captured before the camera enters the third media recording mode.

12. The method according to any one of claims 1-3, wherein: The first indication that the camera is in the first media recording mode includes displaying a recording capability indication with a first appearance; The second indication that the camera is in the second media recording mode includes displaying the recording capability indication having a second appearance different from the first appearance; as well as The third indication that the camera is in the third media recording mode includes displaying the recording capability indication having a third appearance that is different from the first appearance and the second appearance.

13. The method according to any one of claims 1-3, wherein: The first indication that the camera is in the first media recording mode includes displaying the live view from the camera having a first visual feature; as well as The third indication that the camera is in the third media recording mode includes displaying the live view from the camera having a second visual feature that is different from the first visual feature.

14. The method according to any one of claims 1-3, wherein: The first indication that the camera is in the first media recording mode includes a mode indicator that displays the first state; The third indication that the camera is in the third media recording mode includes the mode indicator that displays the second state; as well as The method includes: When the input of the first contact is detected to continue, an animated transition is displayed from the mode indicator in the first state to the mode indicator in the second state.

15. The method of claim 4, comprising: When the third instruction corresponding to the third media recording mode is displayed, the termination of the input of the first contact is detected; as well as Termination in response to the detection of the input of the first contact: Based on the determination that the duration of the input meets a third predefined time threshold: Continue capturing media images using the configuration compatible with multiple media recording modes until a termination input different from the input of the first contact is detected; Upon detection of the termination input, a first video is acquired from the continuously captured media; as well as The first video obtained from persistent storage; as well as Based on the determination that the duration of the input does not meet the third predefined time threshold: Stop using the continuous capture of media images with the configuration that is compatible with multiple media recording modes; Acquire a second video from the continuously captured media; as well as The second video obtained from persistent storage.

16. An electronic device comprising: monitor; camera; One or more processors; Memory; and One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: A camera user interface is displayed on the monitor, the camera user interface including a live view from the camera; While displaying the live view from the camera, the start of input of the first contact is detected, the first contact activating the recording of media using the camera; In response to the detection of the start of the input that activates the first contact for recording media using the camera, a first indication is displayed that the camera is in a first media recording mode corresponding to the recording of a single image; When the first indication that the camera is in the first media recording mode is displayed, the input of the first contact is detected to continue; In response to the detection of the input of the first contact, the continuation, and before the detection of the termination of the input: Based on at least the determination that the input continues to satisfy a first predefined time threshold: A second indication shows that the camera is in a second media recording mode, which corresponds to the recording of an image sequence that continues simultaneously with the input from the first contact; While displaying the second indication that the camera is in the second media recording mode, further detection of the input of the first contact; In response to the further continuation of the input upon detecting the first contact, and prior to the termination of the input detection: Based on the further satisfaction of the second predefined time threshold determined at least by the input, a third indication is displayed that the camera is in a third media recording mode corresponding to video recording.

17. The electronic device of claim 16, wherein the one or more programs include instructions for performing the method of any one of claims 2-15.

18. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by an electronic device having a display and a camera, cause the electronic device to: A camera user interface is displayed on the monitor, the camera user interface including a live view from the camera; While displaying the live view from the camera, the start of input of the first contact is detected, the first contact activating the recording of media using the camera; In response to the detection of the start of the input that activates the first contact for recording media using the camera, a first indication is displayed that the camera is in a first media recording mode corresponding to the recording of a single image; When the first indication that the camera is in the first media recording mode is displayed, the input of the first contact is detected to continue; In response to the detection of the input of the first contact, the continuation, and before the detection of the termination of the input: Based on at least the determination that the input continues to satisfy a first predefined time threshold: A second indication shows that the camera is in a second media recording mode, which corresponds to the recording of an image sequence that continues simultaneously with the input from the first contact; While displaying the second indication that the camera is in the second media recording mode, further detection of the input of the first contact; In response to the further continuation of the input upon detecting the first contact, and prior to the termination of the input detection: Based on the further satisfaction of the second predefined time threshold determined at least by the input, a third indication is displayed that the camera is in a third media recording mode corresponding to video recording.

19. The computer-readable storage medium of claim 18, wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method of any one of claims 2-15.

Images