Device and method for accessing common device functions

Abstract

The present invention is entitled: "Device and method for accessing common device functions". The present invention discloses an electronic device that displays a first user interface, which includes multiple application icons corresponding to different applications among multiple applications installed on the device. The device detects a first input at a position corresponding to a first application icon of the multiple application icons on a touch-sensitive surface, and the first application icon corresponds to a first application of the multiple applications. In response to detecting the first input, the device displays a first mini-application object or a preview of the first mini-application object in an overlapping area, wherein the first mini-application object corresponds to a first application of the multiple applications; and the overlapping area includes an enabling representation for adding the first mini-application object to a second user interface that displays multiple mini-application objects. The device detects a second input at a position on the touch-sensitive surface corresponding to the enabling representation for adding the first mini-application object to the second user interface. In response to detecting the second input, the device adds the first mini-application object to the second user interface that displays the multiple mini-application objects.

Description

[0001] This application is a divisional application of the Chinese invention patent application with the application date of May 24, 2017, the national application number of 201710383083.0, and the invention name of “Device and method for accessing common device functions”. Technical Field

[0002] This document generally relates to electronic devices with touch-sensitive surfaces, including but not limited to electronic devices with touch-sensitive surfaces that include multiple user interfaces for accessing common device functions. Background Art

[0003] In recent years, the use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly. Exemplary touch-sensitive surfaces include touchpads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on displays.

[0004] Exemplary manipulations include accessing controls for controlling device functions and remote device functions. However, the methods for performing these manipulations are cumbersome and inefficient. As the use domain of remote devices expands, accessing specific controls and functions becomes more time-consuming. For example, it is time-consuming for a user to enter an input sequence to access a control through a drop-down menu, especially when the user does not know the location of the desired control in advance. It is also time-consuming to navigate between applications to access device functions. Summary of the invention

[0005] Therefore, there is a need for electronic devices with faster, more efficient methods and interfaces for accessing common device functions. Such methods and interfaces optionally supplement or replace conventional methods for accessing common device functions. Such methods and interfaces reduce the cognitive burden imposed on users and produce more efficient human-computer interfaces. For battery-powered devices, such methods and interfaces can save power and increase the time between battery charges.

[0006] The above-mentioned defects and other problems associated with the user interface of an electronic device with a touch-sensitive surface can be reduced or eliminated by means of the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, a tablet computer, or a handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also referred to as a "touch screen" or a "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, a memory, and one or more modules, programs, or instruction sets stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI on the touch-sensitive surface mainly through stylus and / or finger contact and gestures. In some embodiments, these functions optionally include image editing, drawing, presentation, word processing, website creation, disk editing, spreadsheet production, playing games, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transient computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0007] According to some embodiments, a method is performed at an electronic device having a display and a touch-sensitive surface. The method includes: when the device is in a display-off state, detecting a first input; and in response to detecting the first input: activating the display of the device and displaying a first user interface corresponding to the display-on state of the device on the display; while displaying the first user interface corresponding to the display-on state of the device, detecting a swipe gesture on the touch-sensitive surface; and in response to detecting the swipe gesture on the touch-sensitive surface: replacing the display of the first user interface with the display of a second user interface according to determining that the device is in a lock mode with the display on and the swipe gesture is along a first direction, the second user interface displays a first content; replacing the display of the first user interface with the display of a second user interface according to determining that the device is in an unlock mode with the display on and the swipe gesture is along a first direction, the second user interface displays a first content and a first additional content, the first additional content not being displayed when the device is in a lock mode with the display on.

[0008] According to some embodiments, a method is performed at an electronic device having a display and a touch-sensitive surface. The method includes: when the device is in a display-off state, detecting a first input to activate the display of the device; in response to detecting the first input: activating the display of the device and displaying a first user interface corresponding to the display-on state of the device on the display; while displaying the first user interface corresponding to the display-on state of the device, detecting a swipe gesture on the touch-sensitive surface; and in response to detecting the swipe gesture: replacing the display of the first user interface with the display of a camera application user interface based on determining that the swipe gesture is along a first direction; replacing the display of the first user interface with the display of a mini-application object user interface based on determining that the swipe gesture is along a second direction different from the first direction, the display of the first user interface is configured to include a plurality of mini-application objects, wherein a corresponding mini-application object in the plurality of mini-application objects has a corresponding application stored in the device; displaying a first page of a multi-page control panel user interface based on determining that the swipe gesture is along a third direction different from the first direction and the second direction; and displaying a notification user interface based on determining that the swipe gesture is along a fourth direction different from the first direction, the second direction, and the third direction, the notification user interface is configured to display a plurality of notifications.

[0009] According to some embodiments, a method is performed at an electronic device having a display and a touch-sensitive surface. The method includes: displaying a first user interface including a plurality of application icons corresponding to different applications among a plurality of applications installed on the device; detecting a first input at a position on the touch-sensitive surface corresponding to a first application icon among the plurality of application icons, the first application icon corresponding to a first application among the plurality of applications; in response to detecting the first input, displaying a first mini-application object or a preview of the first mini-application object in an overlay area, wherein: the first mini-application object corresponds to a first application among the plurality of applications; and the overlay area includes an enable representation for adding the first mini-application object to a second user interface displaying a plurality of mini-application objects; detecting a second input at a position on the touch-sensitive surface corresponding to an enable representation for adding the first mini-application object to the second user interface; and in response to detecting the second input, adding the first mini-application object to the second user interface displaying a plurality of mini-application objects.

[0010] According to some embodiments, a method is performed at an electronic device having a display and a touch-sensitive surface. The method includes: while displaying an initial user interface on the display, detecting a first gesture on the touch-sensitive surface; in response to detecting the first gesture, displaying a first page of a multi-page control panel on the display, wherein the first page of the multi-page control panel includes multiple device control affordances; while displaying the first page of the multi-page control panel, detecting a second gesture on the touch-sensitive surface; and in response to detecting the second gesture, displaying a second page of the multi-page control panel, wherein: the second page of the multi-page control panel includes multiple content playback control affordances; and the second page of the multi-page control panel replaces the first page of the multi-page control panel on the display.

[0011] According to some embodiments, an electronic device includes a display unit configured to display a user interface, a touch-sensitive surface unit configured to detect contact, and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: detect a first input when the device is in a display-off state. In response to detecting the first input, the processing unit is configured to: activate the display unit of the device, and enable display of a first user interface corresponding to the display-on state of the device on the display unit. When displaying the first user interface corresponding to the display-on state of the device, the processing unit is configured to detect a swipe gesture on the touch-sensitive surface unit. In response to detecting a swipe gesture on the touch-sensitive surface unit, the processing unit is configured to: replace the display of the first user interface with the display of a second user interface according to determining that the device is in a lock mode in which the display is on and the swipe gesture is along a first direction, and the second user interface displays a first content; and replace the display of the first user interface with the display of a second user interface according to determining that the device is in an unlock mode in which the display is on and the swipe gesture is along a first direction, and the second user interface displays a first content and a first additional content, and the first additional content is not displayed when the device is in a lock mode in which the display is on.

[0012] According to some embodiments, an electronic device includes a display unit configured to display a user interface, a touch-sensitive surface unit configured to detect contact, and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: when the device is in a display-off state, detect a first input to activate the display unit of the device. In response to detecting the first input, the processing unit is configured to: activate the display of the device and enable display of a first user interface corresponding to a display-on state of the device on the display unit. When displaying the first user interface corresponding to the display-on state of the device, the processing unit is configured to detect a swipe gesture on the touch-sensitive surface unit. In response to detecting a swipe gesture, the processing unit is configured to: based on determining that the swipe gesture is along a first direction, replace display of the first user interface with display of a camera application user interface; based on determining that the swipe gesture is along a second direction different from the first direction, replace display of the first user interface with display of a mini-application object user interface, the mini-application object user interface being configured to include multiple mini-application objects, wherein corresponding mini-application objects among the multiple mini-application objects have corresponding applications stored in the device; based on determining that the swipe gesture is along a third direction different from the first direction and the second direction, enable display of a first page of a multi-page control panel user interface; and based on determining that the swipe gesture is along a fourth direction different from the first direction, the second direction, and the third direction, enable display of a notification user interface, the notification user interface being configured to display multiple notifications.

[0013] According to some embodiments, an electronic device includes: a display unit configured to display a first user interface, the first user interface including a plurality of application icons corresponding to different applications among a plurality of applications installed on the device; a touch-sensitive surface unit configured to detect contact; and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: detect a first input at a position on the touch-sensitive surface unit corresponding to a first application icon among a plurality of application icons, the first application icon corresponding to a first application among a plurality of applications. In response to detecting the first input, the processing unit is configured to enable display of a first mini-application object or a preview of a first mini-application object in an overlay area, wherein the first mini-application object corresponds to a first application among a plurality of applications; and the overlay area includes an enabling representation for adding the first mini-application object to a second user interface displaying a plurality of mini-application objects. The processing unit is further configured to detect a second input at a position on the touch-sensitive surface unit corresponding to the enabling representation for adding the first mini-application object to the second user interface. In response to detecting the second input, the processing unit is configured to add the first mini-application object to the second user interface displaying a plurality of mini-application objects.

[0014] According to some embodiments, an electronic device includes a display unit configured to display a user interface, a touch-sensitive surface unit configured to detect contact, and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: detect a first gesture on the touch-sensitive surface unit when an initial user interface is displayed on the display unit. In response to detecting the first gesture, the processing unit is configured to enable display of a first page of a multi-page control panel on the display unit, wherein the first page of the multi-page control panel includes multiple device control enable representations. When displaying the first page of the multi-page control panel, the processing unit is configured to detect a second gesture on the touch-sensitive surface unit. In response to detecting the second gesture, the processing unit is configured to enable display of a second page of the multi-page control panel, wherein the second page of the multi-page control panel includes multiple content playback control enable representations, and the second page of the multi-page control panel replaces the first page of the multi-page control panel on the display unit.

[0015] According to some embodiments, an electronic device includes a display, a touch-sensitive surface, one or more sensors optionally for detecting the intensity of contact with the touch-sensitive surface, one or more processors, a memory, and one or more programs; the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the operations of any of the methods described herein or causing the execution of the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium stores instructions that, when executed by an electronic device having a display, a touch-sensitive surface, and one or more sensors optionally for detecting the intensity of contact with the touch-sensitive surface, cause the device to perform the operations of any of the methods described herein or cause the execution of the operations of any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device having a display, a touch-sensitive surface, one or more sensors optionally for detecting the intensity of contact with the touch-sensitive surface, a memory, and one or more processors for executing one or more programs stored in the memory includes one or more of the elements displayed in any of the above methods, and the one or more elements are updated in response to input, as described in any of the methods described herein. According to some embodiments, an electronic device includes: a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface; and components for performing the operations of any of the methods described herein or causing the execution of the operations of any of the methods described herein. According to some embodiments, an information processing device for use in an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface includes components for performing the operations of any of the methods described herein or causing the execution of the operations of any of the methods described herein.

[0016] Thus, electronic devices having a display, a touch-sensitive surface, and optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface are provided with faster, more efficient methods and interfaces for accessing commonly used device functions, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may supplement or replace conventional methods for accessing commonly used device functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the drawings.

[0018] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

[0019] Figure 1B is a block diagram illustrating example components for event processing according to some embodiments.

[0020] Figure 2 A portable multifunction device with a touch screen according to some embodiments is shown.

[0021] Figure 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0022] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some embodiments is shown.

[0023] Figure 4B An exemplary user interface is shown for a multifunction device having a touch-sensitive surface separate from the display according to some embodiments.

[0024] FIG. 4C to FIG. 4E Exemplary dynamic intensity thresholds are shown according to some embodiments.

[0025] Figure 5A to Figure 5BJ An exemplary user interface for accessing common device functions according to some embodiments is shown.

[0026] FIG. 6A to FIG. 6E is a flow chart illustrating a method of accessing controls from a user interface opened from a display according to some embodiments.

[0027] 7A to 7E is a flow chart illustrating a method of accessing controls from a user interface opened from a display according to some embodiments.

[0028] FIG. 8A to FIG. 8Cis a flow chart illustrating a method of adding a mini-app object to a mini-app user interface according to some embodiments.

[0029] 9A to 9E is a flowchart illustrating a method of using a multi-page control panel navigation control according to some embodiments.

[0030] Figures 10 to 13 is a functional block diagram of an electronic device according to some embodiments. DETAILED DESCRIPTION

[0031] Many electronic devices provide various interfaces for accessing application features and modifying device settings. Such interfaces may require the user to activate the device, provide authentication information, and / or drill down into multiple menus to access desired application features or device settings. The methods described herein provide interfaces for accessing common device functions such as application features and device settings. Such interfaces reduce the number of inputs required to access such features, which reduces power usage and extends the battery life of the device by enabling the user to use the device faster and more efficiently.

[0032] The following Figure 1A to Figure 1B , Figure 2 and Figure 3 A description of an exemplary apparatus is provided. FIG. 4A to FIG. 4B and Figure 5A to Figure 5BJ An exemplary user interface for providing tactile feedback is shown. FIG. 6A to FIG. 6E A flowchart is shown of a method for accessing controls from a user interface opened from a display according to some embodiments. 7A to 7E A flowchart is shown of a method for accessing controls from a user interface opened from a display according to some embodiments. FIG. 8A to FIG. 8C A flowchart illustrating a method of adding a mini-app object to a mini-app user interface according to some embodiments. 9A to 9E A flow chart illustrating a method of using a multi-page control panel navigation control according to some embodiments. Figure 5BJ The user interface in FIG. 6A to FIG. 6E , 7A to 7E , FIG. 8A to FIG. 8C and 9A to 9E process.

[0033] Exemplary Devices

[0034] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail, thereby not unnecessarily obscuring aspects of the embodiments.

[0035] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0036] The terms used in the description of the various embodiments described herein are for the purpose of describing specific embodiments only and are not intended to be limiting. As used in the description of the various embodiments described and in the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the items listed in association. It will also be understood that the terms "includes", "including", "comprises" and / or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groupings.

[0037] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that ..." or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that ..." or "in response to determining that ..." or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0038] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple Watch from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads) are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0039] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse and / or joystick.

[0040] The device typically supports a variety of applications, such as one or more of the following applications: a notepad application, a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a game application, a telephone application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0041] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied with different applications, and / or adjusted and / or varied within the corresponding applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports various applications with a user interface that is intuitive and clear to the user.

[0042] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A1 is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes simply referred to as a touch-sensitive display. The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0043] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a device component is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation, which corresponds to a perceived change in the physical characteristics of the device or device component. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when there is no change in the smoothness of the touch-sensitive surface, movement of the touch-sensitive surface is optionally interpreted or felt by the user as "roughness" of the touch-sensitive surface. Although such interpretation of touch by the user will be limited by the user's individualized sensory perceptions, there are many sensory perceptions of touch that are common to most users. Therefore, when tactile output is described as corresponding to a particular sensory perception of the user (e.g., "release click", "press click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or a component thereof that would generate the sensory perception of a typical (or average) user.

[0044] It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing and / or application specific integrated circuits.

[0045] The memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The access of other components of the device 100 (such as one or more CPUs 120 and peripheral device interface 118) to the memory 102 is optionally controlled by a memory controller 122.

[0046] Peripheral interface 118 may be used to couple the device's input and output peripherals to one or more CPUs 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0047] In some embodiments, peripherals interface 118, one or more CPUs 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0048] RF (radio frequency) circuit 108 receives and transmits RF signals, also called electromagnetic signals. RF circuit 108 converts electrical signals into / from electromagnetic signals and communicates with a communication network and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. RF circuit 108 optionally communicates with a network and other devices via wireless communications, such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). Wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilization Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)) and / or Short Message Service (SMS), or any other suitable communications protocol, including communications protocols that have not been developed as of the filing date of this document.

[0049] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into human-audible sound waves. The audio circuit 110 also receives electrical signals converted from sound waves through the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset having both output (e.g., single or dual ear headphones) and input (e.g., a microphone).

[0050] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., Figure 2 208) optionally includes an up / down button for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2 206).

[0051] The touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives an electrical signal from the touch-sensitive display system 112 and / or sends the electrical signal to the touch-sensitive display system 712. The touch-sensitive display system 112 displays a visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some visual outputs or all of the visual outputs correspond to user interface objects. As used herein, the term "indicator" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to an input pointing to a graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, or other user interface controls.

[0052] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from a user based on tactile and / or haptic contact. The touch-sensitive display system 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In an exemplary embodiment, the point of contact between the touch-sensitive display system 112 and the user corresponds to a finger or stylus of the user.

[0053] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as the . iPod and Technology found in.

[0054] The touch-sensitive display system 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi or greater). The user optionally uses any suitable object or attachment such as a stylus, finger, etc. to contact the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as accurate as stylus-based input due to the larger contact area of ​​the finger on the touch screen. In some embodiments, the device translates the rough finger-based input into a precise pointer / cursor position or command for performing the action desired by the user.

[0055] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touch screen.

[0056] The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0057] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. One or more optical sensors 164 optionally include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. One or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with an imaging module 143 (also referred to as a camera module), one or more optical sensors 164 optionally capture static images and / or videos. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to the touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to obtain an image of the user (e.g., for self-portraits, for video conferencing while the user is watching other video conference participants on the touch screen, etc.).

[0058] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor is shown coupled to a force sensor controller 159 in the I / O subsystem 106. One or more contact force sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact force sensors 165 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the rear of the device 100, opposite the touch-sensitive display system 112 located on the front of the device 100.

[0059] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0060] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1A A tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106 is shown. One or more tactile output generators 167 optionally include one or more electroacoustic devices such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). One or more tactile output generators 167 receive tactile feedback generation instructions from the tactile feedback module 133 and generate tactile outputs on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on the back of the device 100, opposite the touch-sensitive display system 112 located on the front of the device 100.

[0061] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on an analysis of data received from one or more accelerometers. In addition to the one or more accelerometers 168, the device 100 optionally also includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0062] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a tactile feedback module (or instruction set) 133, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, such as Figure 1A and Figure 3 . The device / global internal state 157 includes one or more of the following: an active application state indicating which applications, if any, are currently active; a display state indicating what applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state including information obtained from the device's various sensors and other input or control devices 116; and position and / or location information regarding the device's position and / or posture.

[0063] The operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.

[0064] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices, or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a device compatible with some of Apple Inc. (Cupertino, California). iPod A multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used in iPod devices. In some embodiments, the external port is compatible with some of Apple Inc. (Cupertino, California) iPod A Lightning connector that is the same as, similar to, and / or compatible with the Lightning connector used in iPod devices.

[0065] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to the detection of contact (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact or stylus contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, contact / motion module 130 and display controller 156 detect contact on a touch pad.

[0066] The contact / motion module 130 optionally detects gesture input from a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0067] In some embodiments, detecting a single-finger tap gesture depends on detecting the length of time between a finger press event and a finger lift event, but is independent of the intensity of the finger contact between the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between a finger press event and a finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of the finger contact during the tap reaches a given intensity threshold (greater than a nominal contact detection intensity threshold) (such as a light press or deep press intensity threshold). Therefore, a single-finger tap gesture can meet an input standard that is configured to be met even when the characteristic intensity of the contact does not meet a given intensity threshold. For clarity, the finger contact in a tap gesture typically needs to meet the nominal contact detection intensity threshold in order to detect a finger press event, and no contact is detected below the nominal contact detection intensity threshold. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In the case where the device is configured to detect contact with a finger or stylus hovering over a touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0068] In some embodiments, detecting a single-finger tap gesture depends on detecting the length of time between a finger press event and a finger lift event, but is independent of the intensity of the finger contact between the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between a finger press event and a finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of the finger contact during the tap reaches a given intensity threshold (greater than a nominal contact detection intensity threshold) (such as a light press or deep press intensity threshold). Therefore, a single-finger tap gesture may meet a specific input criterion that does not require the characteristic intensity of the contact to meet a given intensity threshold to meet the specific input criterion. For clarity, the finger contact in a tap gesture typically needs to meet the nominal contact detection intensity threshold in order to detect a finger press event, and no contact is detected below the nominal contact detection intensity threshold. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In the case where the device is capable of detecting contact of a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0069] The same concepts apply in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, an expand gesture, and / or a long press gesture are optionally detected based on satisfying criteria that are independent of the strength of the contacts included in the gesture or that do not require one or more contacts performing the gesture to reach a strength threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a pinch gesture is detected based on the movement of two or more contacts toward each other; an expand gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of contacts on the touch-sensitive surface that move an amount less than a threshold. Therefore, the statement that a particular gesture recognition criterion does not require the strength of one or more contacts to satisfy a corresponding strength threshold to satisfy the particular gesture recognition criterion means that the particular gesture recognition criterion can be satisfied when one or more contacts in the gesture do not reach the corresponding strength threshold, and can also be satisfied when one or more of the contacts in the gesture reaches or exceeds the corresponding strength threshold. In some embodiments, a tap gesture is detected based on determining that a finger down event and a finger up event are detected within a predefined time period, without regard to whether the contact is above or below a corresponding strength threshold during the predefined time period, and a swipe gesture is detected based on determining that the contact movement is greater than a predefined magnitude even if the contact is above the corresponding strength threshold at the end of the contact movement. Even in implementations where detection of gestures is affected by the strength of the contact performing the gesture (e.g., the device detects a long press more quickly when the strength of the contact is above the strength threshold, or the device delays detection of a tap input when the strength of the contact is higher), detection of these gestures does not require the contact to reach a particular strength threshold as long as the criteria for recognizing the gesture are met without the contact reaching the particular strength threshold (e.g., even if the amount of time required to recognize the gesture changes).

[0070] In some cases, the contact intensity threshold, duration threshold, and movement threshold are combined in various combinations to create heuristics for distinguishing two or more different gestures for the same input element or area, so that different interactions with the same input element can provide a richer set of user interactions and responses. The statement that a particular set of gesture recognition criteria does not require the intensity of one or more contacts to meet the corresponding intensity threshold to meet the particular gesture recognition criteria does not exclude the simultaneous evaluation of other gesture recognition criteria that depend on intensity to identify other gestures that do have criteria that are met when the gesture includes a contact with an intensity above the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture competes with a second gesture recognition criterion for a second gesture, where the first gesture recognition criterion does not require the intensity of one or more contacts to meet the corresponding intensity threshold to meet the first gesture recognition criterion, and the second gesture recognition criterion depends on one or more contacts reaching the corresponding intensity threshold. In such a competition, if a gesture first satisfies the second gesture recognition criterion for the second gesture, the gesture is optionally not recognized as satisfying the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before moving a predefined amount of movement, a deep press gesture is detected instead of a swipe gesture. In contrast, if the contact moves a predefined amount of movement before reaching the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in this case, the first gesture recognition criteria for the first gesture still does not require the intensity of one or more contacts to satisfy the corresponding intensity threshold to satisfy the first gesture recognition criteria, because if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with an intensity of the contact that does not increase to above the corresponding intensity threshold), the gesture will be recognized as a swipe gesture according to the first gesture recognition criteria. In this way, a specific gesture recognition criterion that does not require the intensity of one or more contacts to satisfy the corresponding intensity threshold to satisfy the specific gesture recognition criterion will (A) in some cases ignore the intensity of the contact associated with the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases still depend on the intensity of the contact associated with the intensity threshold, in a sense that if a set of competing intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) recognizes an input as corresponding to an intensity-dependent gesture (e.g., for a long press gesture that competes with the deep press gesture for recognition) before the specific gesture recognition criterion recognizes the gesture corresponding to the input, then the specific gesture recognition criterion (e.g., for the long press gesture) will fail.

[0071] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other displays, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0072] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application program, etc., and also receives coordinate data and other graphic attribute data together if necessary, and then generates screen image data to be output to the display controller 156.

[0073] Haptic feedback module 133 includes various software components for generating instructions used by tactile output generator 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

[0074] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0075] GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to phone 138 for location-based dialing, to camera 143 as photo / video metadata, and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and maps / navigation widgets).

[0076] Application 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0077] Contacts module 137 (sometimes referred to as address book or contact list);

[0078] Telephone module 138;

[0079] Video conferencing module 139;

[0080] Email client module 140;

[0081] Instant messaging (IM) module 141;

[0082] Fitness support module 142;

[0083] Camera module 143 for still images and / or video images;

[0084] Image management module 144;

[0085] Browser module 147;

[0086] Calendar module 148;

[0087] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, and a user-created widget 149-6;

[0088] A widget creator module 150 for forming a user-created widget 149-6;

[0089] Search module 151;

[0090] Video and music player module 152, which optionally consists of a video player module and a music player module;

[0091] Notepad module 153;

[0092] Map module 154; and / or

[0093] Online video module 155.

[0094] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0095] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 includes executable instructions for managing an address book or contact list (e.g., stored in the application internal state 192 of the contact module 137 in memory 102 or memory 370), including: adding one or more names to the address book; deleting one or more names from the address book; associating one or more phone numbers, one or more email addresses, one or more physical addresses, or other information with a name; associating images with names; categorizing and sorting names; providing phone numbers and / or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140, or IM 141; and the like.

[0096] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for entering a sequence of characters corresponding to a phone number, accessing one or more phone numbers in address book 137, modifying an entered phone number, dialing a corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communications optionally use any of a variety of communication standards, protocols, and technologies.

[0097] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137 and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0098] In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0099] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for inputting a sequence of characters corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs) or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0100] In combination with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, fitness support module 142 includes executable instructions for: creating a workout (e.g., including time, distance, and / or calorie burn goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for fitness; and displaying, storing, and transmitting fitness data.

[0101] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0102] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still images and / or video images.

[0103] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving and displaying web pages or portions thereof and attachments and other files linked to web pages) in accordance with user instructions.

[0104] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0105] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5) or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).

[0106] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget creator module 150 includes executable instructions for creating a desktop widget (e.g., transferring a user-specified portion of a web page to a desktop widget).

[0107] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0108] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and executable instructions for displaying, presenting, or otherwise playing back videos (e.g., on touch-sensitive display system 112, or on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

[0109] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. according to user instructions.

[0110] In combination with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with maps (e.g., driving routes; data relating to stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0111] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124), send an email containing a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0112] Each of the above modules and applications corresponds to an executable instruction set for performing one or more of the above functions and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0113] In some embodiments, device 100 is a device on which operation of a predefined set of functions is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operation of device 100, the number of physical input control devices (such as push buttons, dials, etc.) on device 100 is optionally reduced.

[0114] The predefined set of functions that are performed exclusively through the touch screen and / or trackpad optionally includes navigation between user interfaces. In some embodiments, the trackpad navigates the device 100 from any user interface displayed on the device 100 to a main menu, a home menu, or a root menu when touched by a user. In such embodiments, a "menu button" is implemented using a trackpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a trackpad.

[0115] Figure 1B 1 is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in an operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0116] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event distributor module 174. In some embodiments, the application 136-1 includes an application internal state 192, which indicates one or more current application views displayed on the touch-sensitive display system 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application or applications are currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0117] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating information being displayed by the application 136-1 or information ready to be displayed by the application, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0118] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0119] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or an input exceeding a predetermined duration).

[0120] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0121] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0122] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are considered to be correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0123] Hit view determination module 172 receives information related to sub-events of contact-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0124] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, views higher in the hierarchy will still remain actively participating views.

[0125] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments that include active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

[0126] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In another embodiment, event classifier 170 is a standalone module or is part of another module (such as contact / motion module 130) stored in memory 102.

[0127] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, the corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more event recognizers in event recognizers 180 are part of an independent module, such as a user interface toolkit (not shown) or a higher level object from which application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178 and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177 or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views in the application views 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in the corresponding application view 191.

[0128] A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0129] Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events (e.g., touch or touch movement). Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (e.g., from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0130] Event comparator 184 compares event information with predefined event or sub-event definitions, and based on the comparison result, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes definitions of events such as event 1 (187-1), event 2 (187-2), and other events (e.g., predefined sub-event sequences). In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined stage on a displayed object, a first lift-off (touch end) of a predetermined stage, a second touch (touch start) of a predetermined stage on a displayed object, and a second lift-off (touch end) of a predetermined stage. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined stage on the displayed object, movement of the touch on the touch-sensitive display system 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0131] In some embodiments, event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with the sub-event and the object that triggered the hit test.

[0132] In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays the delivery of the event information until it has been determined whether the sub-event sequence does or does not correspond to the event type of the event identifier.

[0133] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definition 186, the corresponding event recognizer 180 enters an event not possible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0134] In some embodiments, the corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0135] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag obtains the flag and performs a predefined process.

[0136] In some embodiments, the event delivery instructions 188 include a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or with an actively participating view receives the event information and performs a predetermined process.

[0137] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module 145. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on a touch-sensitive display.

[0138] In some embodiments, one or more event handlers 190 include or have access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0139] It should be understood that the above discussion of event processing for user touches on a touch-sensitive display also applies to other forms of user input utilizing input devices to operate the multifunction device 100, and not all user input is initiated on a touch screen. For example, mouse movement and mouse button presses, optionally in combination with single or multiple keyboard presses or holds; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof as input corresponding to a sub-event, define an event to be recognized.

[0140] Figure 2 A touch screen (e.g., Figure 1AThe portable multifunction device 100 of the present invention is a touch-sensitive display system 112 in the device 100. The touch screen optionally displays one or more graphics within the user interface (UI) 200. In this embodiment and in other embodiments described below, the user can select one or more of the graphics by, for example, making gestures on the graphics using one or more fingers 202 (not drawn to scale in the drawings) or one or more styluses 203 (not drawn to scale in the drawings). In some embodiments, the selection of one or more graphics will occur when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of the fingers that have been in contact with the device 100 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with the graphics will not select the graphics. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0141] The device 100 optionally also includes one or more physical buttons, such as a "home" button or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display 112.

[0142] In some embodiments, the device 100 includes a touch screen display, a menu button 204, a push button 206 for turning the device on / off and for locking the device, one or more volume adjustment buttons 208, a user identity module (SIM) card slot 210, an earphone jack 212, and a docking / charging external port 124. The push button 206 is optionally used to: turn the device on / off by pressing the button and keeping the button in a pressed state for a predefined time interval; lock the device by pressing the button and releasing the button before a predefined time interval; and / or unlock the device or initiate an unlocking process. In some embodiments, the device 100 also accepts voice input for activating or deactivating certain functions through a microphone 113. The device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0143] Figure 3300 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (e.g., a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuits (sometimes called chipsets) that interconnect system components and control communications between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touch screen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 167 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A Memory 370 includes high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located away from one or more CPUs 310. In some embodiments, memory 370 stores data related to portable multifunction device 100 ( Figure 1A ) stored in memory 102 of portable multifunction device 100, or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100 ( Figure 1A )'s memory 102 optionally does not store these modules.

[0144] Figure 3Each element in the above-mentioned identified element is optionally stored in one or more memory devices in the previously mentioned memory device.Each module in the above-mentioned identified module corresponds to a group of instructions for performing the above-mentioned functions.The above-mentioned identified module or program (that is, instruction set) need not be implemented as a separate software program, process or module, and therefore the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments.In some embodiments, memory 370 optionally stores a subset of the above-mentioned modules and data structures.In addition, memory 370 optionally stores other modules and data structures not described above.

[0145] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

[0146] Figure 4A An exemplary user interface of an application menu on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0147] one or more signal strength indicators 402 of one or more wireless communications, such as cellular signals and Wi-Fi signals;

[0148] Time 404;

[0149] Bluetooth indicator 405;

[0150] Battery status indicator 406;

[0151] A tray 408 with commonly used application icons such as:

[0152] an icon 416 of the phone module 138 labeled "Phone," which optionally includes an indicator 414 of the number of missed calls or voicemail messages;

[0153] An icon 418 of the email client module 140 labeled “Mail”, which optionally includes an indicator 410 of the number of unread emails;

[0154] An icon 420 labeled "Browser" of the browser module 147; and

[0155] An icon 422 labeled “iPod” of the video and music player module 152 (also referred to as the iPod (trademark of Apple Inc.) module 152); and

[0156] Icons for other applications, such as:

[0157] An icon 424 labeled “Message” of the IM module 141;

[0158] An icon 426 labeled “Calendar” of the calendar module 148 ;

[0159] Icon 428 labeled “Photos” of image management module 144;

[0160] An icon 430 labeled as “Camera” of the camera module 143;

[0161] An icon 432 labeled “Online Video” of the online video module 155;

[0162] Icon 434 labeled “Stock Market” of the Stock Market widget 149 - 2 ;

[0163] An icon 436 labeled “MAP” of the map module 154 ;

[0164] Icon 438 labeled “Weather” of the weather widget 149 - 1 ;

[0165] An icon 440 labeled as “clock” of the alarm clock desktop applet 149 - 4 ;

[0166] An icon 442 labeled “Fitness Support” of the fitness support module 142 ;

[0167] An icon 444 labeled "Notepad" of the notepad module 153; and

[0168] An icon 446 of a settings application or module that provides access to settings for the device 100 and its various applications 136 .

[0169] It should be noted that Figure 4A The icon labels shown in are exemplary only. For example, in some embodiments, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0170] Figure 4B A touch-sensitive surface 451 (eg, Figure 3 A device (e.g., a tablet or touch pad 355) Figure 3Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357) for detecting intensity of contact on touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating tactile output for a user of device 300.

[0171] Figure 4B A touch-sensitive surface 451 (eg, Figure 3 A device (e.g., a tablet or touch pad 355) Figure 3 Although some of the subsequent embodiments will be given with reference to input on a touch screen display 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with touch-sensitive surface 451 at a location corresponding to a corresponding location on the display (e.g., Figure 4B 460 and 462) (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g. Figure 4B When the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is separated, the device is used to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0172] In addition, although the following examples are primarily given with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures, etc.), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact) and subsequent movement of the cursor along the swipe path (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detection of contact and subsequent cessation of detection of contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or that a mouse and finger contacts are optionally used simultaneously.

[0173] As used herein, the term "focus selector" is an input element that indicates the current portion of a user interface that a user is interacting with. In some implementations that include a cursor or other positional marker, the cursor acts as a "focus selector" such that when the cursor rests over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 3 Touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 in the display, a specific user interface element is adjusted according to the detected input. In a touch screen display (e.g., Figure 1A A touch-sensitive display system 112 or Figure 4AIn some implementations of a touch screen in a touch screen display, a detected contact on the touch screen acts as a "focus selector" such that when an input (e.g., a press input by the contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted according to the detected input. In some implementations, the focus moves from one area of ​​the user interface to another area of ​​the user interface without a corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user in order to convey the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user intends to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch screen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).

[0174] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated force of the contact. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the alternative measurement of the contact force or pressure is used directly to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the alternative measurement). In some embodiments, the alternative measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is measured in pressure units). The intensity of the contact is used as an attribute of the user input, thereby allowing the user to access additional device functions that the user cannot easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical controls / mechanical controls such as knobs or buttons).

[0175] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether a user has performed an action (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0176] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact intensity is detected to increase, and / or before or after contact intensity is detected to decrease). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of contact intensity, the mean value of contact intensity, the average value of contact intensity, the top 10% value of contact intensity, the half-maximum value of contact intensity, the 90% maximum value of contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the intensity of the contact over time). In some embodiments, the characteristic strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this example, a contact whose characteristic strength does not exceed the first threshold results in a first operation, a contact whose characteristic strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose characteristic strength exceeds the second threshold results in a third operation. In some embodiments, a comparison between the characteristic strength and one or more strength thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0177] In some embodiments, a portion of a gesture is identified for determining a characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an end position (e.g., a drag gesture), where the strength of the contact increases. In this example, the characteristic strength of the contact at the end position may be based only on a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only the swipe contact portion at the end position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the strength of the swipe contact to achieve the purpose of determining the characteristic strength.

[0178] The user interface figures described herein optionally include various intensity graphs that show the intensity of contacts on the touch-sensitive surface relative to one or more intensity thresholds (e.g., contact detection intensity threshold IT0, light press intensity threshold IT1, and light touch intensity threshold IT2). L , (e.g., at least initially above I L Deep compression intensity threshold IT D and / or one or more other intensity thresholds (e.g., below I L The intensity threshold I H )). This intensity graph is not typically part of the displayed user interface, but is provided to help explain the graph. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, when a contact having a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold IT0, contacts below which are no longer detected) is detected, the device will move the focus selector in accordance with the movement of the contact on the touch-sensitive surface without performing an operation associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface illustrations.

[0179] In some embodiments, the response of the device to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, the strength of the contact exceeding the first strength threshold during the input triggers the first response. In some embodiments, the response of the device to the input detected by the device depends on a criterion that includes both the contact strength during the input and a time-based criterion. For example, for some "deep press" inputs, the contact strength exceeding the second strength threshold (greater than the first strength threshold for light press) during the input triggers the second response only when a delay time passes between meeting the first strength threshold and meeting the second strength threshold. The delay time is typically less than 200ms in duration (e.g., 40ms, 100ms, or 120ms, depending on the size of the second strength threshold, and the delay time increases as the second strength threshold increases). This delay time helps avoid accidental deep press inputs. For another example, for some "deep press" inputs, a period of reduced sensitivity occurs after reaching the first strength threshold. During this period of reduced sensitivity, the second strength threshold is increased. This temporary increase in the second strength threshold also helps avoid accidental deep press inputs. For other deep press inputs, the response to detecting the deep press input is not dependent on time-based criteria.

[0180] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of concurrent inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Exemplary factors are described in U.S. patent applications 14 / 399,606 and 14 / 624,296, which are hereby incorporated by reference in their entirety.

[0181] For example, Figure 4C A dynamic intensity threshold 480 is shown that changes over time based in part on the intensity of touch input 476 over time. Dynamic intensity threshold 480 is the sum of two components, a first component 474 that decays over time after a predefined delay time p1 from the initial detection of touch input 476, and a second component 478 that tracks the intensity of touch input 476 over time. The initial high intensity threshold of first component 474 reduces accidental triggering of a "deep press" response while still allowing a "deep press" response to occur immediately if touch input 476 provides sufficient intensity. Second component 478 reduces unintentional triggering of a "deep press" response by gradually fluctuating in intensity in the touch input. In some embodiments, when touch input 476 satisfies dynamic intensity threshold 480 (e.g., within a certain period of time), the touch input 476 may be triggered. Figure 4C ), triggering a “deep press” response.

[0182] Figure 4D Another dynamic intensity threshold 486 (eg, intensity threshold I D ). Figure 4D Two other intensity thresholds are also shown: a first intensity threshold I H and the second intensity threshold I L .exist Figure 4D Although the touch input 484 satisfies the first intensity threshold I before time p2, H and the second intensity threshold I L , but no response is provided until after a delay time p2 at time 482. Figure 4D In the example, the dynamic intensity threshold 486 decays over time, and at the time 482 (when the dynamic intensity threshold 486 is equal to the second intensity threshold I L The decay begins at time 488 after a predefined delay time p1 has elapsed since the associated response was triggered. This type of dynamic intensity threshold reduces the delay between triggering and a lower intensity threshold (such as the first intensity threshold I H Or the second intensity threshold I L ) is associated with a response, or simultaneously with the dynamic intensity threshold I D Unexpected triggering of the associated response.

[0183] Figure 4E Another dynamic intensity threshold 492 (eg, intensity threshold I D ).exist Figure 4E In the example, after a delay time p2 has passed since the initial detection of the touch input 490, the trigger and the intensity threshold I L At the same time, after a predefined delay time p1 from the initial detection of the touch input 490, the dynamic intensity threshold 492 decays. Therefore, without releasing the touch input 490, the dynamic intensity threshold 492 decays after the trigger and intensity threshold I L The intensity of touch input 490 decreases after the associated response, and then the intensity of touch input 490 increases, which can trigger the intensity threshold I D The associated response (e.g., at time 494) is performed even when the intensity of touch input 490 is below another intensity threshold (e.g., intensity threshold I L ) is also the case.

[0184] The contact characteristic strength is below the light press strength threshold IT L The intensity increases to between the light pressure intensity threshold IT L and deep compression intensity threshold IT D The intensity between IT and the deep press intensity threshold is sometimes referred to as a "light press" input. D The intensity of the deep compression increases to above the deep compression intensity threshold IT D The intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. L A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact lifted from the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some illustrations, shaded circles or ellipses are used to represent the intensity of contacts on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0185] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the strength of the contact (or multiple contacts) increases to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting that the strength of the corresponding contact increases to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and the strength of the contact subsequently decreases to below the press input strength threshold, and the corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases to below the press input threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0186] In some embodiments, the device employs intensity hysteresis to avoid unexpected input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the strength of the contact increases from an strength equal to or below a hysteresis strength threshold to an strength equal to or above a press input strength threshold and, optionally, the strength of the contact subsequently decreases to an strength equal to or below the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (for example, depending on the circumstances, the strength of the contact increases or the strength of the contact decreases).

[0187] For ease of explanation, optionally, descriptions of operations performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input are triggered in response to detecting that the intensity of the contact increases above the press input intensity threshold, the intensity of the contact increases from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreases below the press input intensity threshold, or the intensity of the contact decreases below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting that the intensity of the contact decreases below the press input intensity threshold, the operations are optionally performed in response to detecting that the intensity of the contact decreases below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, the triggering of these responses is also dependent on satisfying time-based criteria (e.g., a delay time elapses between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0188] User interface and associated processes

[0189] Attention now turns to embodiments of a user interface ("UI") and associated processes that can be implemented on an electronic device such as portable multifunction device 100 or device 300 having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contact with the touch-sensitive surface.

[0190] Figure 5A to Figure 5BJ An exemplary user interface for accessing common device functions according to some embodiments is shown. The user interface in these figures is used to illustrate the following process, including FIG. 6A to FIG. 6E , 7A to 7E , FIG. 8A to FIG. 8C and 9A to 9E Although some of the subsequent examples will be given with reference to input on a touch screen display (in which a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface 451 that is separate from the display 450, such as Figure 4B as shown in .

[0191] Figure 5A-1 to Figure 5A-4 The display (e.g., touch-sensitive display 112) is shown activated in response to changing the orientation of device 100. In some embodiments, the device uses one or more sensors (e.g., accelerometers, gyroscopes, audio sensors, thermal sensors, and / or light sensors) to determine whether the orientation of the device has changed. For example, the device determines whether the device has been rotated beyond a threshold angle (e.g., rotated along an axis of the device, such as tilting from a generally horizontal position of the device to a generally vertical position of the device). Figure 5A-1 , the device is held flat in the user's hand 502 so that the device display is generally horizontal. Figure 5A-2, the device is tilted so that the display is Figure 5A-1 is more vertical. Figure 5A-2 In , the display is not activated because the tilt angle of the device does not increase above the threshold tilt angle. Figure 5A-3 , the device is tilted so that the display is Figure 5A-2 is more vertical. Figure 5A-3 In FIG. 5 , as the tilt angle of the device increases above a threshold tilt angle, the display is activated (e.g., display content 504 is displayed by the display). Display content 504 includes, for example, time, date, signal indicator, battery charge level indicator, lock indicator, and / or one or more notifications (such as notification 506). In FIG. Figure 5A-4 In some embodiments, when the display is activated (e.g., Figure 5A-3 As shown in FIG. 5 , part or all of the displayed content 504 is shown in a smaller size (e.g., narrower width) than the default size of the displayed content 504. As the device continues to tilt after the display is activated, the size of the displayed content 504 gradually increases to the default size, such as Figure 5A-4 shown.

[0192] FIG. 5C to FIG. 5E 5 shows a user interface displayed in response to input provided by contact 508 at a physical button 204 (e.g., a "home" or menu button). Figure 5B In the embodiment, the physical button 204 detects a contact 508 that activates but does not actuate the physical button (e.g., when the contact 508 rests on the surface of the physical button without pressing the physical button). For example, according to detecting a contact 508 at the physical button 204 that does not meet the criteria for actuating the physical button 204 (e.g., the characteristic intensity of the contact 508 exceeds the contact detection intensity threshold IT0 but does not exceed the actuation threshold IT0), A , as shown by the actuation meter 510), the physical button is activated (but not actuated). In some embodiments, the activation of the physical button 204 wakes up the device and / or the touch screen display 112. For example, in response to the activation of the physical button 204, the device activates the display and displays the wake-up screen user interface 512, as shown in FIG. Figure 5C For example, in response to an input that activates but does not cause a physical button to move, the device is in an awake (display on) state, but is not fully activated.

[0193] In some embodiments, in response to a contact that satisfies the criteria for actuating the physical button (e.g., when the intensity of contact 508 with physical button 204 exceeds an actuation threshold IT A When Figure 5D), the device 100 displays a user interface (e.g., application springboard user interface 400) including a plurality of application icons (e.g., application icons 416 to 446), as shown in FIG. Figure 5E For example, in response to input from actuating a physical button, the device is fully activated.

[0194] In some embodiments, physical button 204 includes a biometric sensor, such as a fingerprint sensor for user authentication scanning (e.g., TouchID scanning). In some embodiments, activation and / or actuation of the physical button unlocks the device based on successful authentication of the user by the biometric sensor of the physical button recognizing a fingerprint corresponding to contact 508. For example, in response to activation of physical button 204 in which successful user authentication occurs, the unlocked state of lock icon 514 is displayed at Figure 5C In the wake-up screen interface 512.

[0195] In some embodiments, the device 100 includes a virtual button (e.g., instead of a physical button) that functions as a home or menu button 204. When describing the button 204 herein, it should be understood that a mechanical button, a virtual button, or another type of button may be used. For example, the virtual button is activated based on a determination that a characteristic intensity of contact with the virtual button does not exceed an actuation threshold, and the virtual button is actuated based on a determination that a characteristic intensity of contact with the virtual button exceeds an actuation threshold. In some embodiments, the virtual button includes a biometric sensor for user authentication.

[0196] Figure 5F-1 to Figure 5F-4 5 shows a sequence of user interfaces displayed in response to a detected input when the device 100 is in a locked mode (e.g., as shown by the locked state of the lock icon 514). Figure 5F-1 , wake-up screen interface 512 is displayed. In some embodiments, for example, in response to input at push button 206 for turning the device on / off and locking the device, in response to input at physical button 204 that does not unlock the device (e.g., input by a user whose fingerprint is not recognized by the fingerprint scanner of physical button 204), and / or in response to input at a virtual button that activates the device display but does not unlock the device, the lock mode of wake-up screen interface 512 is displayed.

[0197] Figure 5F-1 A downward swipe gesture is shown, wherein contact 516 moves along the path indicated by arrow 518. In response to the downward swipe gesture, a locked mode of notification user interface 520 is displayed, such as Figure 5F-2 In some embodiments, when a downward swipe occurs, an animation is displayed in which notification user interface 520 is displayed as gradually descending from the top of the display.

[0198] In response to Figure 5F-2 The input of the contact 522 is detected at the position corresponding to the search input area 524, and a user interface 526 with a keyboard 528 is displayed, such as Figure 5F-3 shown.

[0199] exist Figure 5F-3 , a search term ("movies") is entered in search input area 524 (e.g., using keyboard 528). In response to input initiating the search (e.g., input of contact 530 detected at an affordance corresponding to a "return" key of keyboard 528), a locked mode of search results user interface 532 is displayed, as shown in FIG. Figure 5F-4 shown.

[0200] The user may wish to limit the information displayed when the device is locked so that others cannot see private and / or sensitive information. In some embodiments, the device is configurable so that specific types of notifications, notification content, and / or search results displayed when the device is unlocked are not visible when the device is locked. Figure 5F-5 to Figure 5F-8 Additional content is shown displayed in the unlocked mode of notification user interface 520 and search results user interface 532 .

[0201] Figure 5F-5 to Figure 5F-8 A sequence of user interfaces is shown that is displayed in response to detected input when device 100 is in unlocked mode (e.g., as indicated by the unlocked state of lock icon 514). In some embodiments, the unlocked mode of wake screen interface 512 is displayed, for example, in response to input at physical button 204 that unlocks the device (e.g., input by a user whose fingerprint is not recognized by the fingerprint scanner of physical button 204).

[0202] Figure 5F-5 A downward swipe gesture is shown, wherein contact 534 moves along the path indicated by arrow 536. In response to the downward swipe gesture, the unlocked mode of notification user interface 520 is displayed, such as Figure 5F-6 shown.

[0203] In response to Figure 5F-6 The input of contact 538 is detected at a position corresponding to the search input area 524, and a user interface 526 having a keyboard 528 is displayed, such as Figure 5F-7 shown.

[0204] exist Figure 5F-7 , a search term ("movies") is entered in search input area 524 (e.g., using keyboard 528). In response to input initiating the search (e.g., input of contact 540 detected at a "return" key affordance of keyboard 528), an unlocked mode of search results user interface 532 is displayed, such as Figure 5F-8 shown.

[0205] Figure 5G The locked mode of notification user interface 520 is shown (eg, as shown by the locked state of lock icon 514). The locked mode of message notification 542 and the locked mode of voicemail notification 544 are displayed in the locked mode of notification user interface 520.

[0206] Figure 5H An unlocked mode of notification user interface 520 is shown (e.g., as indicated by the unlocked state of lock icon 514). An unlocked mode of message notification 542 and an unlocked mode of voicemail notification 544 are displayed in the unlocked mode of notification user interface 520. The unlocked mode of message notification 542 includes additional information, such as message content (e.g., text 546 and / or image 548 from one or more messages), compared to the locked mode of message notification 542. The unlocked mode of notification user interface 520 displays an invitation notification 550 that is not displayed in the locked mode of notification user interface 520.

[0207] Fig.5I A locked mode of search results user interface 532 is shown (e.g., as indicated by the locked state of lock icon 514). Search results displayed in the locked mode of search results user interface 532 for search term “movies” entered in search input area 524 include, for example, location-based results (e.g., suggested movies playing near the device's location, location suggestions for movie theaters near the device's location), application results (iMovie), and suggested website results.

[0208] Figure 5J An unlocked mode of search results user interface 532 is shown (eg, as indicated by the unlocked state of lock icon 514 ). The unlocked mode of notification user interface 532 displays messages 552 and 554 and email 556 that are not displayed in the locked mode of search results user interface 532 .

[0209] Figure 5K to Figure 5N 542 is shown in which a user is prompted to provide a password to view an expanded view of the message notification 542 in response to input received in the locked mode of the message notification 542. Figure 5K 542, an input for expanding message notification 542 is provided through contact 557. Because the device is in lock mode (as shown by the locked state of lock icon 514), user authentication is required before the expanded view of message notification 542 is displayed. Figure 5L A user authentication prompt user interface 558 including a password input interface 560 is shown. Based on determining that the user authentication information (e.g., input through contact 562 at the password input interface 560 and / or provided by the fingerprint sensor of the physical button 204) is valid, an expanded view of the notification 542 is displayed, such as Figure 5M shown.

[0210] In some embodiments, the expanded view of notification 542 includes, for example, a message sender avatar 564 (and / or the name of the message sender), an enable indication 566 for dismissing the notification (e.g., an "X" enable indication), message content (e.g., a message including image 548 and a message including text 546), and / or contextual conversation information (e.g., message 568 preceding message 548 in the conversation transcript). In some embodiments, when a new message is received in the conversation, the expanded view of message notification 542 is updated to display the new message (e.g., displaying new message 570 in message notification 542). In some embodiments, the expanded view of message notification 542 includes an enable indication 572 for displaying a reply view of message notification 542 (e.g., in response to input received by contact 574, such as a tap input at a position corresponding to enable indication 572 for displaying a reply view).

[0211] Figure 5N A reply view of a message notification 542 is shown that enables a user to reply to a message within the notification (e.g., without navigating from a notification user interface to an instant messaging application user interface). When the reply view of the message notification 542 is displayed, a keyboard 576 is displayed. Input received at the keyboard 576 is displayed in a message input area 578 of the message notification 542.

[0212] Figure 5O to Figure 5Q Navigation from calendar invitation message notification 550 to an expanded view of calendar invitation message notification 550, and navigation from the expanded view of calendar invitation message notification 550 to calendar application user interface 594 are shown. Fig.5O In the embodiment, an input of contact 580 is received at a location corresponding to the invitation message notification 550 in the unlocked view of the notification user interface 520. In response to detecting the input, the device displays an expanded view of the calendar invitation message notification 550, such as Figure 5P As shown. The expanded view of the calendar invitation message notification 550 includes contextual calendar information 582 corresponding to the invited event 584 and an action item menu including action items 586 ("Accept"), 588 ("Maybe"), and 590 ("Decline"). An input of a contact 592 is received at a location corresponding to the calendar invitation message notification 550. In response to detecting the input, the device displays a calendar application user interface 594 (e.g., an event details page corresponding to the invited event 584, such as Figure 5Q shown).

[0213] Figure 5R-1 A voicemail notification 544 is shown. Figure 5R-2Shown is an expanded view of voicemail notification 544. Expanded view of voicemail notification 544 includes, for example, playback controls for playing back the voicemail audio (e.g., play / pause control 596, playback position slider 598, and / or volume control 5100), a transcript of the voicemail 5102 (e.g., automatically generated by the device based on the voicemail audio), and / or an action item menu including action items 5104 ("Call Back") and 5106 ("Delete").

[0214] Figure 5S-1 Notification 5108 is shown for another exemplary application (eg, a driving service application). Figure 5S-2 An expanded view of notification 5108 is shown. The expanded view of notification 5108 includes, for example, a map 5112 and / or an action item 5118 ("Call Driver"). In some embodiments, map 5112 includes content that updates in real time and / or near real time. For example, a representation 5114 of the real-time location of the car is displayed on map 5112 relative to a representation 5116 of the car's destination. As the car moves, the representation 5114 of the car is updated to reflect the movement of the car.

[0215] Figure 5T-1 to Figure 5T-2 1 shows a sequence of user interfaces displayed in response to detected input when the device 100 is in the locked mode. Figure 5T-1 , the wake-up screen interface 512 is displayed in a locked mode, as indicated by the locked state of the lock icon 514.

[0216] Figure 5T-1 A right swipe gesture is shown, wherein contact 5117 moves along the path indicated by arrow 5120. In response to the right swipe gesture, a locked mode of mini-app user interface 5122 is displayed, as shown in FIG. Figure 5T-2 In some embodiments, when a swipe to the right is performed, an animation is displayed in which the notification user interface 520 is shown sliding gradually from the left side of the display to the right.

[0217] In some embodiments, the mini-app user interface 5122 includes a search input area 5124. In response to input received at the search input area 5124, a locked mode of the search results user interface (e.g., similar to Figure 5F-4 and Fig.5I Search results user interface 532 is shown).

[0218] Figure 5T-3 to Figure 5T-4 1 shows a sequence of user interfaces displayed in response to detected input when the device 100 is in unlocked mode. Figure 5T-3 , the wake-up screen interface 512 is displayed in an unlocked mode, as shown by the unlocked state of the lock icon 514.

[0219] Figure 5T-3 A right swipe gesture is shown, wherein contact 5119 moves along the path indicated by arrow 5121. In response to the right swipe gesture, an unlocked mode of mini-app user interface 5122 is displayed, as shown in FIG. Figure 5T-2 In response to input received at search input area 5124, an unlocked mode of the search results user interface is displayed (e.g., similar to Figure 5F-8 and Figure 5J Search results user interface 532 is shown).

[0220] Figure 5U A locked mode of mini-app user interface 5122 is shown. The locked mode of the mini-app user interface displays, for example, a locked mode of a calendar mini-app 5125 and / or a locked mode of a weather mini-app user interface 5126. In some embodiments, the locked and / or unlocked modes of mini-app user interface 5122 include information 5123, such as information about points of interest near device 100 (e.g., as determined using GPS and / or Wi-Fi data received by device 100 and / or points of interest data stored and / or accessed by device 100). In some embodiments, calendar mini-app 5125 includes an identifier (e.g., text 5131 identifying the corresponding application and / or an icon 5135 identifying the corresponding application).

[0221] exist Figure 5U , contact 5128 provides input at “show more” affordance 5127 for displaying an expanded view of calendar mini-app 5125. Because the device is in lock mode (as indicated by lock icon 514), user authentication is required before the expanded view of mini-app 5125 is displayed. Figure 5V 51 is a user authentication prompt user interface 558 including a password input interface 560. Based on determining that the user authentication information (e.g., input through contact 5129 at the password input interface 560 and / or provided by the fingerprint sensor of the physical button 204) is valid, an expanded view of the mini application 5125 is displayed, such as Figure 5W In some embodiments, the expanded view of mini-app 5125 includes, for example, an expanded time range (compared to the locked view and / or unlocked view of calendar mini-app 5125) and calendar event information that is not displayed in the locked view of calendar mini-app 5125 (e.g., "Coffee with Jon," "Team Meeting," "Meeting with Shelby at the Shellfish Market"). Weather mini-app 5126 is adjusted downward in mini-app user interface 5122 to accommodate the expansion of calendar mini-app 5125.

[0222] exist Figure 5W, providing an expanded view (such as that shown in FIG. 5A ) for calendar mini-application 5125 by contact 5137 at a location corresponding to enable indication 5130. Figure 5W ) is reduced to an input of an unexpanded view of the calendar mini-application 5125, as shown Figure 5X The mini-app user interface 5122 is shown in unlock mode.

[0223] In the unlocked mode of mini-app user interface 5122, an unlocked view of calendar mini-app 5125 is displayed (not expanded). Compared to the locked view of calendar mini-app 5125, the locked mode of calendar mini-app 5125 includes additional information, such as calendar event information (e.g., "Coffee with Jon" and / or "Team Meeting") that is not displayed in the unlocked view of calendar mini-app 5125. The unlocked mode of mini-app user interface 5122 displays the next mini-app 5132 that is not displayed in the locked mode of mini-app user interface 5122.

[0224] exist Figure 5X , at a location corresponding to the “Show More” enable indication 5141 , contact 5139 provides input for displaying an expanded view of the next mini-application 5132 .

[0225] Figure 5Y An expanded view of next mini-app 5132 is shown. The expanded view of next mini-app 5132 includes a map 5134 indicating the location of the next event and an action item menu including action items 5136 ("Navigate"), 5138 ("Snooze"), 5140 ("Message Invitees"), and 5142 ("Delete Event"). An input of contact 5143 is detected at a location corresponding to next mini-app 5132. In response to detecting the input, the device displays a calendar application user interface 5144 that includes an indication 5146 of the next event, as shown in FIG. Figure 5Z shown.

[0226] Figures 5AA to 5AD 4 shows changes that occur in the application springboard user interface 400 when input is received to display the quick action menu. Figure 5AA , an input of contact 5150 is detected at a location corresponding to mail application icon 418. Mail application icon 418 corresponds to a mail application (e.g., a mail application that is launched in response to a tap input detected at mail application icon 418). A characteristic intensity of the contact is indicated by intensity level meter 5148. Figure 5AB In the example, as indicated by the intensity level meter 5148, the characteristic intensity of the contact increases to exceed the prompt intensity threshold level IT HIn some embodiments, based on determining that the characteristic intensity of contact 5149 has increased to exceed the prompt intensity threshold level IT H , applying a blur effect to at least a portion of the user interface 400. Figures 5AB to 5AD As shown, as the characteristic strength of the contact increases, the blurring effect increases. In some embodiments, based on determining that the characteristic strength of the contact has increased to exceed the prompt strength threshold level IT H , a leading view 5152 of a quick action menu 5154 corresponding to the mail icon 418 is displayed. Figures 5AB to 5AC As shown, the size of the leading view 5152 increases as the characteristic strength of the contact increases.

[0227] exist Figure 5AD The characteristic intensity of contact 5150 has increased to exceed the deep press intensity threshold IT D In some embodiments, based on determining that the characteristic strength of the contact increases to exceed the deep press strength threshold IT D , one or more tactile output generators of device 100 output tactile output as shown at 5150. In some embodiments, quick action menu 5154 corresponding to mail application icon 418 is displayed. In some embodiments, mini-app preview 5156 corresponding to mail mini-app 5162 is displayed (e.g., in a location adjacent to quick action menu 5154 and / or on a disk that includes quick action menu 5154 and / or mini-app preview 5156). In some embodiments, quick action menu 5154 and / or mini-app preview 5156 are displayed at least partially covering application springboard user interface 400.

[0228] exist Figure 5AE , an input of contact 5158 is detected at “Add widget” affordance 5160 of mini-app preview 5156. In response to detecting the input, the device adds a mail mini-app 5162 corresponding to mini-app preview 5156 to mini-app user interface 5122, as shown in FIG. Figure 5AF In some embodiments, in response to detecting an input of contact 5158 at “Add Widget” affordance 5160 , the device displays mini-app user interface 5122 including mini-app preview 5156 .

[0229] exist Figure 5AF, a mail mini-app 5162 is displayed in the mini-app user interface 5122. In some embodiments, the mail mini-app 5162 includes avatars 5164 (e.g., 5164a, 5164b, 5164c, and / or 5164d) corresponding to one or more email senders who recently and / or frequently send emails to and / or receive emails from the device 100. An indicator 5166 at a location corresponding to the avatar 5164a indicates, for example, the number of unread emails received from the email sender corresponding to the avatar 5164a. An input of a contact 5170 is detected at a location corresponding to an enable indication 5168 for displaying an expanded view of the mail mini-app 5162. In response to detecting the input, the device displays the expanded view of the mail mini-app 5162, as shown in FIG. Figure 5AG shown.

[0230] Figure 5AG An expanded view of mail mini-app 5162 is shown displayed in mini-app user interface 5122. Additional avatars 5164e, 5164f, 5164g, and / or 5164h that are not shown in the unexpanded view of mail mini-app 5162 are shown in the expanded view of mail mini-app 5162.

[0231] Figure 5AH A transition from a state displaying wake screen interface 512 to a state displaying a locked mode of camera application user interface 5172 is shown. At a first time t1, a left swipe gesture is initiated, where contact 5176 begins to move along a first portion of the swipe gesture, as indicated by arrow 5178. At a second time t2, after time t1, a first portion of camera application user interface 5172 is displayed through the first portion of the swipe gesture, and contact 5176 continues to move along a second portion of the swipe gesture indicated by arrow 5180. At a third time t3, after time t2, camera application user interface 5172 is further displayed through the second portion of the swipe gesture, and contact 5176 continues to move along a third portion of the swipe gesture indicated by arrow 5182. At a fourth time t4, after time t3, camera application user interface 5172 is further displayed through the third portion of the swipe gesture, and contact 5176 continues to move along a fifth portion of the swipe gesture indicated by arrow 5184. At a fifth time t5, after time t4, the locked mode of camera application user interface 5172 is fully displayed.

[0232] In some embodiments, one or more camera controls 5186 are not initially displayed in the locked mode of the camera application user interface 5172. For example, after a delay from time t1 when the left swipe gesture is initiated (e.g., a delay of a period of time between time t1 and time t3), one or more camera controls 5186 are displayed in the camera application user interface 5172. In some embodiments, one or more camera controls 5186 are displayed in the camera application user interface 5172 after a threshold portion of the camera application user interface 5172 is displayed.

[0233] In some embodiments, in the locked mode (e.g., as shown in FIG. Figure 5AH 5), one or more images in the image library of device 100 are inaccessible. For example, when the unlock mode of camera application user interface 5172 is displayed (e.g., as in Figure 5AI 5), the most recently captured image will be displayed in area 5188, and when the lock mode of the camera application user interface 5172 is displayed (e.g., as shown in FIG. Figure 5AH When the image is displayed in area 5188 at time t5 as shown in the figure, no image will be displayed in area 5188.

[0234] Figure 5AI A transition from a state displaying wake screen interface 512 to a state displaying an unlocked mode of camera application user interface 5172 is shown. At a first time t1, a left swipe gesture is initiated, where contact 5190 begins to move along a first portion of the swipe gesture, as indicated by arrow 5192. At a second time t2, after time t1, a first portion of camera application user interface 5172 is displayed through the first portion of the swipe gesture, and contact 5190 continues to move along a second portion of the swipe gesture indicated by arrow 5194. At a third time t3, after time t2, camera application user interface 5172 is further displayed through the second portion of the swipe gesture, and contact 5190 continues to move along a third portion of the swipe gesture indicated by arrow 5196. At a fourth time t4, after time t3, camera application user interface 5172 is further displayed through the third portion of the swipe gesture, and contact 5190 continues to move along a fourth portion of the swipe gesture indicated by arrow 5198. At a fifth time t5, after time t4, the unlocked mode of camera application user interface 5172 is fully displayed.

[0235] In some embodiments, one or more camera controls 5186 are not initially displayed in the unlocked mode of the camera application user interface 5172. For example, after a delay from time t1 when the left swipe gesture is initiated (e.g., a delay of a period of time between time t1 and time t3), one or more camera controls 5186 are displayed in the camera application user interface 5172. In some embodiments, one or more camera controls 5186 are displayed in the camera application user interface 5172 after a threshold portion of the camera application user interface 5172 is displayed. This can avoid accidental operation of the camera application user interface.

[0236] exist Figure 5AJ , the input for partially displaying the camera application user interface 5172 is followed by the input for displaying the wake screen interface 512 again. In some embodiments, the input for displaying the camera application user interface 5172 is canceled by stopping the left swipe gesture (e.g., completing only the first part of the gesture before all of the camera application user interface 5172 or a threshold portion of the camera application user interface 5172 is displayed). In some embodiments, the input for displaying the camera application user interface 5172 is canceled by providing an input (e.g., a right swipe gesture) before all of the camera application user interface 5172 is displayed or before a threshold portion of the camera application user interface 5172 is displayed. For example, an input including the first part 5178 and the second part 5180 of the left gesture is detected, and then a right gesture is performed along the path indicated by the arrow 5202. The right gesture cancels the display of the camera application user interface 5172. This can avoid accidental display and / or operation of the camera application interface.

[0237] exist Figure 5AK , the wake-up screen interface 512 is displayed in a locked mode, as indicated by the locked state of the lock icon 514. Figure 5AK An upward swipe gesture is shown (e.g., from the bottom edge of the touch screen 112), where contact 5204 moves along the path indicated by arrow 5206. In response to the upward swipe gesture, the first page of a multi-page control panel user interface 5208 is displayed, as shown in FIG. Figure 5AL In some embodiments, when an upward swipe is performed, an animation is displayed in which the multi-page control panel user interface 5208 is shown gradually sliding upward from the bottom edge of the display.

[0238] Figure 5ALA first page of a multi-page control panel user interface 5208 is shown. Multi-page control panel user interface 5208 includes multiple control affordances, such as 5210 (airplane mode), 5212 (Wi-Fi), 5214 (Bluetooth), 5216 (Do Not Disturb mode), 5218 (rotation lock), 5220 (flashlight), 5222 (timer), 5224 (night mode), 5226 (calculator), 5228 (camera), 5230 (Apple TV mirroring), and / or 5232 (brightness). In some embodiments, one or more control affordances of the first page of multi-page control panel user interface 5208 are unavailable in the locked mode of the first page of multi-page control panel user interface 5208. In some embodiments, the appearance of one or more of the control affordances of the first page of multi-page control panel user interface 5208 indicates the state of the control affordance. For example, Wi-Fi control enable indication 5212 is shown without shading to indicate that Wi-Fi is enabled, and Bluetooth control enable indication 5214 is shown with shading to indicate that Bluetooth is disabled.

[0239] In some embodiments, the multi-page control panel user interface 5208 includes a control user interface disabling indication 5234. In some embodiments, a page indicator 5236 is used to indicate the page of the multi-page control panel currently displayed. In some embodiments, the multi-page control panel user interface 5208 partially or completely covers another user interface (e.g., Figure 5AL In some embodiments, the user interface appearance (e.g., wake screen user interface 512) partially covered by multi-page control panel user interface 5208 is changed, as shown in 5238. For example, the partially covered user interface 512 is blurred and / or has a lower brightness than the previous user interface appearance.

[0240] In response to an input of contact 5241 with movement along the path indicated by arrow 5240 (eg, swiping left), a second page of multi-page control panel user interface 5208 is displayed, as shown in FIG. Figure 5AMAs shown. In some embodiments, the second page of the multi-page control panel user interface 5208 includes multiple content playback control affordances, such as a playback locator 5242, a previous track control 5246, a pause / play control 5248, a next track control 5250, and / or a volume adjustment control 5252. In some embodiments, the second page of the multi-page control panel user interface 5208 includes content routing destination information 5260 and / or affordances 5262 for displaying / stopping display of the routing destination menu. In some embodiments, the second page of the multi-page control panel user interface 5208 includes content information of the currently playing and / or recently played content, such as an image 5264 representing the content (e.g., an album cover) and / or identification information 5266 of the content (e.g., a track name, an album name, and / or an artist). In some embodiments, one or more control affordances of the first page of the multi-page control panel user interface 5208 are not available in the locked mode of the second page of the multi-page control panel user interface 5208.

[0241] In response to input including movement of contact 5268 along the path indicated by arrow 5270 (eg, swiping left), a third page of multi-page control panel user interface 5208 is displayed, as shown in FIG. Figure 5AN As shown. In some embodiments, the third page of the multi-page control panel user interface 5208 includes multiple remote device control enable representations, such as temperature controls 5272, fan controls 5274, shade controls 5276, light controls 5278, door controls 5280, camera controls 5282, smoke alarm controls 5284, and / or sleep controls 5286. In some embodiments, one or more remote device controls are not available in the third page of the locked mode of the multi-page control panel user interface 5208. For example, the remote device control enable representations 5272-5286 are shown as visually changed (e.g., shaded) from the default state of the control enable representations 5272-5286 displayed in the unlocked mode of the multi-page user interface 5208 to indicate that one or more remote device controls corresponding to the visually changed remote device control enable representations are not available in the locked mode. In this way, access rights to the remote device space are limited to adjustment by a user who has provided verification information to unlock the device. In some embodiments, the third page of the multi-page control panel user interface 5208 includes identification information (e.g., the name of the defined area 5288 and / or an image 5290 corresponding to the area) for a defined area (e.g., a scene, such as a "living room") where one or more remote devices controlled by the control indications 5272-5286 are located.

[0242] In some embodiments, the remote device control enable indications 5272-5286 displayed in the third page of the multi-page control panel user interface 5208 are remote device control enable indications corresponding to (e.g., located in) a defined area (e.g., a room) in which the device 100 is currently located (e.g., determined by GPS and / or Wi-Fi signals received by the device). In some embodiments, the remote device control enable indications 5272-5286 displayed in the third page of the multi-page control panel user interface 5208 are remote device control enable indications corresponding to a defined area that was most recently visited. In some embodiments, the third page of the multi-page control panel user interface 5208 includes an indication 5292 for displaying / stopping display of multiple defined area identifiers (e.g., a menu 5294 for displaying multiple defined area identifiers, such as Figure 5AQ shown).

[0243] exist Figure 5AN , input for operating light control enable indication 5278 is provided by contact 5296. Because the device is in lock mode (as indicated by lock icon 514), user verification is required before light control enable indication 5278 can be operated. In some embodiments, in response to detecting input at control enable indication 5278 while lock mode is active, the device displays verification prompt user interface 558. Figure 5AO 5 is a user verification prompt user interface 558 including a password input interface 560. Based on determining that the user verification information (e.g., input through contact 5295 at the password input interface 560 and / or provided by the fingerprint sensor of the physical button 204) is valid, an unlock view of the third page of the multi-page control panel user interface 5208 is displayed, as shown in FIG. Figure 5AP shown.

[0244] exist Figure 5AP , remote device control enable representations 5272-5286 are shown without shading because control enable representations 5272-5286 can be used in the unlocked mode of the device. Input (e.g., a tap gesture) is provided by contact 5296 at a location corresponding to enable representation 5292 for displaying / stopping display of multiple defined area identifiers. In response to the input, menu 5294 including multiple defined area identifiers is displayed, such as Figure 5AQ In some embodiments, the visual appearance of the enable representation 5292 for displaying / stopping display of multiple defined area identifiers changes in response to input provided at a location corresponding to the enable representation 5292. For example, the orientation of the chevron icon of the enable representation 5292 changes from Figure 5AP becomes Figure 5AQ .

[0245] exist Figure 5AQ, menu 5294 of multiple defined area identifiers (e.g., "Favorites" 5300, "Current Room" 5302, "Master Bedroom" 5304, "Office" 5306, and / or "Kitchen" 5308) is displayed. Input (e.g., a tap gesture) is provided by contact 5298 at a location corresponding to affordance 5292 for displaying / stopping display of multiple defined area identifiers. In response to the input, menu 5294 of multiple defined area identifiers 5300-5308 stops being displayed, and a third page of multi-page control panel user interface 5208 is redisplayed, for example, as shown in FIG. Figure 5AR In some embodiments, in response to receiving an input (e.g., a tap input) of a corresponding defined area identifier of menu 5294, a set of control affordances corresponding to the selected corresponding defined area identifier is displayed.

[0246] like Figure 5AR to Figure 5AS As shown, in response to an input of contact 5310 having movement along the path indicated by arrow 5312 (e.g., swiping downward), the third page of the multi-page control panel user interface 5208 is stopped from being displayed, and the lower user interface 512 is redisplayed, as shown in FIG. Figure 5AS (In some embodiments, input received at a location corresponding to the control user interface disabling indication 5234 causes the multi-page control panel user interface 5208 to stop being displayed and the underlying wake screen interface 512 to be redisplayed, as shown in FIG. Figure 5AS shown.)

[0247] like Figure 5AS to Figure 5AT As shown, in response to an input of contact 5314 having movement along the path indicated by arrow 5316 (e.g., swiping upward), the multi-page control panel user interface 5208 is redisplayed. In some embodiments, in response to the input, the most recently displayed page of the multi-page control panel user interface 5208 (e.g., the third page of the multi-page control panel user interface 5208) is redisplayed, such as Figure 5AT shown.

[0248] Figure 5AU Input is shown received at an application user interface (e.g., calendar application user interface 5318) via contact 5322 (e.g., an upward swipe along the path indicated by arrow 5320) to display the first page of multi-page dashboard user interface 5208, as shown in FIG. Figure 5AV shown.

[0249] exist Figure 5AV , receiving input of contact 5235 (e.g., swiping left along the path indicated by arrow 5327) to display a second page of multi-page user interface 5208, such as Figure 5AW shown.

[0250] exist Figure 5AW , the second page of multi-page control panel user interface 5208 is displayed partially covering calendar application user interface 5318. In response to an input for stopping display of multi-page control panel user interface 5208 (e.g., a downward swipe of contact 5329 along the path indicated by arrow 5331), the second page of multi-page control panel user interface stops being displayed, and calendar application user interface 5318 is redisplayed, as shown in FIG. Figure 5AX shown.

[0251] exist Figure 5AX , an input for redisplaying multi-page control panel user interface 5208 (e.g., an upward swipe of contact 5333 along the path indicated by arrow 5335) is detected at calendar application user interface 5318. In response to detecting the input, the device redisplays the second page of multi-page control panel user interface 5208, as shown in FIG. Figure 5AY shown.

[0252] exist Figure 5AY , an input for displaying a third page of multi-page control panel user interface 5208 (e.g., a horizontal swipe of touch 5339 along the path indicated by arrow 5337) is detected at the second page of multi-page control panel user interface 5208. In response to detecting the input, the device displays the third page of multi-page control panel user interface 5208, as shown in FIG. Figure 5AZ shown.

[0253] exist Figure 5AZ , an input for displaying a second page of multi-page control panel user interface 5208 (e.g., a horizontal swipe of touch 5343 along the path indicated by arrow 5341) is detected at the third page of multi-page control panel user interface 5208. In response to detecting the input, the device displays the second page of multi-page control panel user interface 5208, as shown in FIG. Figure 5BA shown.

[0254] exist Figure 5BA , in response to an input (e.g., a tap input) of contact 5324 at a position corresponding to affordance 5262 for displaying / stopping display of a routing destination menu, routing destination menu 5326 is shown, as shown in FIG. Figure 5BB shown.

[0255] exist Figure 5BB, a second page of the multi-page control panel user interface 5208 includes a routing destination menu 5326 containing routing destinations 5328 (“iPhone”), 5260 (“Airpods”), and 5332 (“Bluetooth speaker”). For example, the routing destination menu expands from the content routing destination information area 5260, and the size of the multi-page control panel user interface 5208 expands (vertically) to accommodate the routing destination menu 5326. In some embodiments, one or more routing destinations include information corresponding to the routing destinations, such as an indication 5334 of the current routing destination (e.g., a check mark icon) and / or a battery level indication 5336. In some embodiments, the visual appearance of the enable indication 5262 changes in response to input provided at a location corresponding to the enable indication 5262. For example, the orientation of the chevron icon of the enable indication 5262 changes from: Figure 5BA becomes Figure 5BB .

[0256] exist Figure 5BB , in response to an input (e.g., a tap input) of contact 5338 at a position corresponding to routing destination 5332 (“Bluetooth speaker”), the routing destination of content (e.g., currently playing content) is changed from routing destination 5260 (“Airpods”) to routing destination 5332 (“Bluetooth speaker”), as shown in FIG. Figure 5BC shown.

[0257] exist Figure 5BC In the example, the content has been routed to destination 5332 (“Bluetooth speaker”) and / or has stopped being routed to routing destination 5260 (“Airpods”), for example, as indicated by the absence of indication 5334 of the current routing destination in the area corresponding to routing destination 5260 (“Airpods”) and the presence of indication 5334 of the current routing destination in the area corresponding to routing destination 5332 (“Bluetooth speaker”). In response to an input (e.g., a tap input) of contact 5340 at a location corresponding to affordance 5262 for displaying / stopping display of routing destination menu 5326, routing destination menu 5326 is dismissed, as shown in FIG. Figure 5BD shown.

[0258] exist Figure 5BD , an input for changing the content routing destination is shown (e.g., a horizontal swipe of touch 5323 along the path indicated by arrow 5345). In response to detecting the input for changing the content routing destination, the device updates content routing destination information area 5260 to indicate the changed content routing destination (e.g., "Airpods"), as shown. Figure 5BE shown.

[0259] Figures 5BF to 5BJThe functionality of modifying the control affordances of the first page of a multi-page control panel user interface 5208 is shown according to some embodiments. Fig. 5BF , an input of contact 5342 is detected at a location corresponding to Wi-Fi control affordance 5212 of the first page of multi-page control panel user interface 5208. As indicated by intensity meter 5344, the characteristic intensity of the contact is below prompt press intensity threshold IT. H , Light press intensity threshold IT L and deep compression intensity threshold IT D . According to determining that the characteristic strength of the contact meets the control switching standard (for example, the characteristic strength of the contact is lower than the prompt pressing strength threshold IT H ), the input of contact 5342 will correspond to the Wi-Fi control indication 5212 switching the Wi-Fi control from a disabled state to an enabled state.

[0260] Figure 5BG Shows the Wi-Fi controls from Fig. 5BF The disabled state shown is switched to Figure 5BG 5208. To indicate that the Wi-Fi control has been switched to the enabled state, the appearance of Wi-Fi control indicator 5212 changes (e.g., changes visually, such as from a shaded state to an unshaded state), and / or notification 5346 is displayed (e.g., “Wi-Fi: On”).

[0261] Figure 5BH to Figure 5BI An input for displaying modification options for Wi-Fi controls is shown. Figure 5BH , the input of contact 5348 at the location corresponding to Wi-Fi control enable indication 5212 satisfies the enhanced control criteria (e.g., the characteristic intensity of contact 5348 increases to exceed the light press intensity threshold level IT as indicated by intensity meter 5344). L , and / or the characteristic intensity of contact 5348 increases to exceed the deep press intensity threshold level IT D In response to the input, modification options menu 5350 includes modification options 5354 ("Disconnect from home network"), 5356 ("Turn off for 1 hour"), 5358 ("Always turn off before I leave"), 5360 ("Connect to other network"), and 5362 ("Wi-Fi settings"). An input of contact 5352 is detected at a location corresponding to modification option 5356 ("Turn off for 1 hour").

[0262] exist Figure 5BJ In response to Figure 5BIAs shown, the Wi-Fi controls are (temporarily) disabled, e.g., as indicated by the shaded state of Wi-Fi control indicator 5212 and notification 5346 (e.g., “Wi-Fi: Off”).

[0263] FIG. 6A to FIG. 6E A flowchart of a method 600 for accessing a control from a user interface opened from a display according to some embodiments is shown. The method 600 is performed on an electronic device (e.g., Figure 3 Device 300 in, or Figure 1A The method 600 is performed on a portable multifunction device 100 in FIG. 1 . In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display and the touch-sensitive surface are separate. Some operations in method 600 are optionally combined, and / or the order of some operations is optionally changed.

[0264] As described below, method 600 provides an intuitive way to access controls from a user interface opened by a display. The method reduces the cognitive burden on a user when accessing controls from a user interface opened by a display, thereby creating a more efficient human-machine interface. For battery-powered electronic devices, users are enabled to access controls faster and more efficiently, which saves power and increases the time between battery charges.

[0265] In the display off state (for example, Figure 5B and Figure 5D ), the device detects a first input (602). In some embodiments, the display-off state is a state in which the display is turned off or dimmed while the touch-sensitive surface in the touch screen remains active, a state in which the touch screen is turned off, and / or a state in which the display function of the touch screen is turned off, for example, in a sleep / hibernation mode, a power saving mode, or another mode with limited functionality. In some embodiments, the first input is an input on a display activation enable indication (e.g., a physical button or a virtual button) that meets display activation criteria for activating the device display (e.g., waking up the device and / or waking up the touch screen display). For example, the first input is an input of contact 508 at physical button 204, as shown in FIG. Figure 5B and Figure 5DAs shown. In some embodiments, the display activation criteria do not require that the characteristic intensity of the contact in the detected input satisfy the corresponding intensity threshold to satisfy the display activation criteria (for example, the display activation criteria is satisfied by a tap gesture). In some embodiments, the display activation criteria is satisfied when the contact is detected to be lifted off before the characteristic intensity of the contact exceeds a predetermined intensity threshold (if the display activation enable representation is a virtual button), or when the contact does not physically actuate the display activation enable representation (if the display activation enable representation is a physical button, such as physical button 204). In some embodiments, the display activation criteria are satisfied by other types of input (for example, voice input, user picking up and tilting the device, etc.) without the need to detect contact.

[0266] In response to detecting the first input, the device (604) activates the display of the device (e.g., touch screen display 112) and displays a first user interface corresponding to the display on state of the device (e.g., the device displays a first user interface such as Figure 5C As shown in the wake-up screen 512, the device displays a user interface that is immediately displayed when the device is woken up, or the device activates the device's display).

[0267] While displaying a first user interface corresponding to a display-on state of the device, the device detects (606) a swipe gesture on the touch-sensitive surface (e.g., swiping along the Figure 5F-1 The downward gesture along the path indicated by the middle arrow 518, along Figure 5F-5 The downward gesture along the path indicated by the middle arrow 536, Figure 5T-1 The rightward gesture along the path indicated by the middle arrow 5120, along Figure 5T-3 The rightward gesture of the path indicated by the middle arrow 5121, such as Figure 5AH The left gesture shown, such as Figure 5AI The left gesture shown, along Figure 5AK An upward gesture along the path shown by arrow 5206, and / or along the path shown by arrow 5206. Figure 5AS Upward gesture along the path indicated by arrow 5314).

[0268] In response to detecting a swipe gesture on the touch-sensitive surface (608): based on determining that the device is in a lock mode with the display on (e.g., as indicated by the lock state of lock icon 514) and the swipe gesture is in a first direction (e.g., downward and / or right), the device replaces the first user interface with display of the second user interface (e.g., as indicated by the lock state of lock icon 514). Figure 5C The wake-up screen 512 shown in FIG. 5 is displayed, and the second user interface displays the first content (for example, Figure 5F-2 Notification screen as shown and / or Figure 5T-2Based on determining that the device is in unlocked mode with the display turned on (e.g., as shown by the unlocked state of lock icon 514) and the swipe gesture is in a first direction (e.g., downward and / or right), the device replaces display of the first user interface with display of a second user interface, the second user interface displaying first content and first additional content (e.g., restricted notification content and / or restricted mini-app object content) that are not displayed when the device is in locked mode with the display turned on. For example, Figure 5F-6 and Figure 5H As shown, the unlocked mode of notification user interface 520 includes additional content not shown in the locked mode of notification user interface 520, such as notification 550, text 546 of notification 542, and / or image 548 of notification 542. Figure 5T-4 and Figure 5X As shown, the unlocked mode of the mini-app user interface 5122 includes additional content, such as the next mini-app 5132 and / or calendar event information not displayed in the locked view of the calendar mini-app 5125 (e.g., “Coffee with Jon,” “Team Meeting”).

[0269] Displaying different content depending on whether the device is in a locked mode or an unlocked mode provides efficient access to information available in the device while maintaining the security of sensitive information available in the device. Providing secure access to information stored in the device enhances the operability of the device and makes the user-device interface more efficient (e.g., by allowing the user to access information in a display-on state of the device without fully activating the device), and additionally reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0270] In some embodiments, the notification includes information corresponding to a communication received by the device, such as a phone call, video call, voicemail, email, SMS, MMS, and / or calendar invitation. In some embodiments, the notification is a calendar appointment notification, calendar appointment invitation, reminder, and / or another type of notification generated by an application installed on the device.

[0271] In some embodiments, the first content includes (610) one or more notifications (e.g., excluding restricted notification information), and the first additional content includes restricted notification information (e.g., notification content and / or other notification information that is subject to restricted access through device settings, such as privacy and / or security settings). For example, the first additional content includes contextual information, such as earlier communication content in a thread that includes the first content in the conversation transcript (such as message 568 prior to received message 548, for which notification 542 was generated, as shown in FIG5M) and / or calendar information corresponding to a time range associated with a calendar appointment notification in the first content (e.g., such as Figure 5P In some embodiments, when the device is in unlocked mode with the display on, the second user interface (e.g., Figure 5F-6 , Figure 5G , Figure 5M , Figure 5N , Figure 5P , Figure 5R-2 and Figure 5S-2 ) including when the device is in a lock mode with the display on (e.g., Figure 5F-2 and Figure 5H In some embodiments, the expanded notification content includes a more complete version of the notification content shown in the short version of the notification. In some embodiments, the expanded notification content includes images, interactive controls, and selectable options for performing actions on the notification that are not included in the short version of the notification.

[0272] In some embodiments, the first content includes (612) one or more mini-app objects (e.g., one or more "widgets"), and the first additional content includes restricted mini-app object information (e.g., mini-app object information and / or other information that is subject to restricted access through device settings, such as privacy and / or security settings). For example, calendar mini-app 5125 as shown in unlocked mode of mini-app user interface 5122 (e.g., as shown in FIG. 5125 ). Figure 5T-4 , Figure 5W and Figure 5X ) included in the calendar mini-application 5125 (as shown Figure 5T-2 and Figure 5U Mini-app object information not shown).

[0273] In some embodiments, the restricted mini-app object information and / or restricted notification information includes, for example: contextual content (such as an additional message in a conversation transcript or an expanded view of a calendar), photo content, video, audio, real-time updated content (such as a traffic app map view showing real-time vehicle locations), controls (e.g., action buttons), a user's list of frequently used contacts, and / or a keyboard for inline text input (e.g., when operating an "answer" control in a communication notification). For example, additional content includes, for example: Figure 5M Contextual content (e.g., messages 568 and 570) and / or reply affordance 572 in an expanded view of message notification 542 as shown; Figure 5N The message input area 578 and / or keyboard 576 in the expanded view of the message notification 542 shown; Fig.5O and Figure 5P The invitation message notification 550 shown (but not presented in the locked mode of the notification user interface 520); Figure 5R-2 5100), a transcribed text 5102 of the voicemail in an expanded view of the voicemail notification 544, and / or action items 5104 ("Call Back") and 5106 ("Delete"); Figure 5S-2 Map 5112 and / or action item 5118 (“Call Driver”) in an expanded view of notification 5108 as shown; Figure 5W and Figure 5Y The expanded view of the calendar mini-app 5125 shown has calendar event information (e.g., "Coffee with Jon," "Team Meeting," "Meeting with Shelby at the Shellfish Market") and the expanded calendar time range, the next mini-app 5132, and / or Figure 5AG Additional avatars 5164e through 5164h in the expanded view of the mail mini-app 5162 shown.

[0274] Displaying mini-application objects in an interface accessible while the device's display is on allows a user to view information (e.g., commonly used information, such as frequently accessed, user-specified, and / or otherwise specified) from an application without fully activating the device and / or activating the application. Providing access to application information while the device's display is on enhances the device's operability and makes the user-device interface more efficient (e.g., by allowing the user to access information while the device's display is on without fully activating the device), which in addition reduces power usage and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0275] In some embodiments, in response to detecting a swipe gesture on the touch-sensitive surface (614): determining that the device is in a lock mode with the display on (e.g., Figure 5F-1 Shown and / or Figure 5T-1 ) and the swipe gesture is in a second direction different from the first direction (e.g., Figure 5T-1 Rightward and / or as shown Figure 5F-1 ), the device replaces the display of the first user interface with the display of the third user interface, and the third user interface displays the second content (for example, Figure 5T-2 Mini-App Objects screen as shown and / or as Figure 5F-2 and, based on determining that the device is in an unlocked mode in which the display is on (e.g., Figure 5F-5 and Figure 5T-3 ) and the swipe gesture is along a second direction (e.g., as Figure 5T-3 Rightward and / or as shown Figure 5F-5 The device replaces display of the first user interface with display of a third user interface, the third user interface displays second content and second additional content (e.g., limited mini-app object content (e.g., as shown in FIG. 1 ) that are not displayed when the device is in the locked mode with the display turned on. Figure 5T-4 ), and / or limited notification content or other notification information that is subject to limited access through privacy and / or security settings on the electronic device (e.g., Figure 5F-6 shown)).

[0276] In some embodiments, the swipe gesture is a swipe gesture along a third direction different from the first direction and the second direction (e.g., Figure 5AH and 5AI Based on determining that the device is in unlocked mode (e.g., Figure 5AI ), replacing the first user interface with a display of a fourth user interface associated with a camera application (e.g., camera application user interface 5172). Based on determining that the device is in a locked mode (e.g., Figure 5AH As shown), the first user interface is replaced with a display of a fourth user interface associated with the camera application, and access to an image library associated with the camera application is limited (e.g., only images captured since the first input was received are accessible).

[0277] In some embodiments, the swipe gesture is a swipe gesture along a fourth direction different from the first direction, the second direction, and the third direction (e.g., along Figure 5AK The path shown by arrow 5206 and / or Figure 5AS 5314 in FIG. 5 is an upward path). Based on determining that the device is in unlocked mode (e.g., Figure 5AS), replacing the first user interface 512 with a display of a control panel user interface (e.g., multi-page control panel user interface 5208). In some embodiments, based on determining that the device is in a locked mode (e.g., Figure 5AK ), replacing the first user interface 512 with a display of the multi-page control panel user interface 5208, and at least one panel and / or control of the control panel user interface 5208 is inaccessible (e.g., as Figure 5AN For example, when the device is in locked mode (e.g. Figure 5AN In some embodiments, based on determining that the device is in locked mode, the device displays a locked mode mark (e.g., a closed padlock icon 514) on the first user interface 512, the second user interface (e.g., the notification user interface 520 and / or the mini-application user interface 5122), the third user interface (e.g., the notification user interface 520 and / or the mini-application user interface 5122), the fourth user interface (e.g., the camera application user interface 5172), and / or the control panel user interface 5208. In some embodiments, based on determining that the device is in unlocked mode, the device displays an unlocked mode mark (e.g., an open padlock icon, such as the unlocked state of the lock icon 514) on the first user interface, the second user interface, the third user interface, the fourth user interface, and / or the control panel user interface.

[0278] In some embodiments, the second content includes (616) one or more mini-app objects (e.g., a calendar mini-app 5125, a weather mini-app 5126, a next mini-app 5132, and / or a mail mini-app 5162); and the second additional content includes restricted mini-app object information.

[0279] In some embodiments, the second content includes (618) one or more notifications (e.g., calendar invitation notification 550, message notification 542, voicemail notification 544, and / or notification 5108); and the second additional content includes restricted notification information.

[0280] In some embodiments, the second user interface includes (620) a first search input area 524 (e.g., Figure 5F-2 to Figure 5F-4 and Figure 5F-6 to Figure 5F-8 ), and the third user interface includes a second search input area 5124 (for example, as Figure 5T-2 and Figure 5T-4 shown).

[0281] In some embodiments, the device detects (622) input (e.g., text input for filtering a search) in corresponding search input areas of the first search input area and the second search input area (e.g., search input area 524, which includes the input word "movie," and / or search input area 5124).

[0282] In response to detecting an input in the corresponding search area: based on determining that the device is in the lock mode with the display turned on, the device displays (624) a first set of search results (e.g., Figure 5F-4 and Fig.5I According to the determination that the device is in the unlocked mode in which the display is turned on, the device displays a search result different from the first group (for example, Figure 5F-6 and Figure 5J For example, message 552, message 554, and email 556 are displayed in a second set of search results (e.g., a set of search results that includes additional search results that are limited due to the security and / or privacy policy of the device and are not included in the first set of search results). Figure 5J , but in the Fig.5I Not shown in FIG.

[0283] In some embodiments, in response to detecting a first input (e.g., upon display activation), such as Figure 5D Input via contact 508 is shown determining whether the device is in an unlocked mode with the display on or in a locked mode (626).

[0284] In some embodiments, in response to detecting a swipe gesture on the touch-sensitive surface (e.g., after display activation), a determination is made as to whether the device is in a locked mode with the display on or an unlocked mode with the display on (628).

[0285] In some embodiments, the electronic device includes (630) one or more sensors (including, for example, an accelerometer, a gyroscope, a microphone, a vibration sensor, a thermal sensor, a touch sensor, and / or a light sensor), and detecting a first input (to activate a display of the device) includes: using the one or more sensors to detect a change in the device environment (e.g., the device is picked up, such as Figure 5A-1 to Figure 5A-4). In some embodiments, the change in the device environment is, for example, a change in the internal and / or external state of the device, such as a change in the orientation of the device and / or a change in the environment (e.g., light level). For example, the environment of the device changes when the user picks up the device from a table and / or takes the device out of a pocket to view the device held in hand 502, and / or in response to detecting a trigger phrase or keyword spoken by the user. In response to detecting a change in the device environment, the device determines whether the change in the device environment meets the display activation criteria. In some embodiments, the display activation criteria include one or more of a tilt criterion (e.g., a threshold deviation from an initial position and / or a threshold deviation from a horizontal axis), an acceleration criterion, and / or a light level change criterion. In some embodiments, the output of one or more sensors is used to prevent false positives. For example, if the light level change criterion is met, but the acceleration amount is below a threshold acceleration level (e.g., different from the acceleration criterion), the display activation criterion is not met. In some embodiments, as the orientation of the device changes, the wallpaper and / or notification expands as the orientation changes (e.g., as Figure 5A-3 to Figure 5A-4 shown).

[0286] In some embodiments, detecting a first input (to activate a display of the device) includes (632) detecting activation of a display activation enable representation (e.g., a physical button 204 and / or a virtual button on touch-sensitive surface 112). In some embodiments, detecting activation of a display activation enable representation includes detecting operation of a physical button 204 (such as sleep / wake button 206). In some embodiments, detecting activation of a display activation enable representation includes detecting, by one or more sensors for detecting intensity of contact with a touch-sensitive surface (e.g., at a location corresponding to a virtual button, such as a virtual home button configured to detect force, contact area, and / or fingerprint), that satisfies a corresponding threshold intensity (e.g., such as a touch screen). Figure 5D The intensity meter 510 indicates the IT A ) increases the contact feature strength.

[0287] In some embodiments, detecting a first input (to activate the display of the device) includes (634) detecting contact with a display activation indicator (e.g., a physical sleep / wake button 206 and / or a physical home button 204) that activates the display activation indicator (e.g., to activate the display of the device) without actuating the display activation indicator. In some embodiments, the contact with the display activation indicator that does not actuate the display activation indicator is contact with the physical button 204 that does not move and / or press the physical button (e.g., a tap on a button that includes a fingerprint sensor, such as a physical home button, for Touch ID scanning), such as Figure 5B shown.

[0288] In some embodiments, detecting a first input (to activate the display of the device) includes (636) detecting contact with a display activation enable indication (e.g., a virtual button and / or physical button 204) that activates the display activation enable indication for activating the device display (e.g., waking up the device or waking up a touch screen display) without activating the display activation enable indication for performing at least one additional function associated with the display activation enable indication. In some embodiments, the display activation enable indication is a virtual button or physical button 204 that triggers execution of different functions in response to contact at different intensity levels. For example, contact with a first intensity threshold (e.g., such as a touch screen display) may be triggered by contact with a first intensity threshold. Figure 5B The intensity meter 510 indicates the IT A ) activates a first function associated with the virtual button (e.g., to activate a display of the device), and has a characteristic intensity equal to or exceeding a first intensity threshold (e.g., as Figure 5D ) activates a second function associated with the virtual button (e.g., to unlock the device (if the device is locked) and display the home screen (e.g., application springboard user interface 400, as shown) Figure 5E shown)).

[0289] In some embodiments, while displaying first user interface 512, the device detects (638) a second input, and in response to detecting the second input, the device replaces display of the first user interface with display of a user interface that includes multiple application icons corresponding to different applications in multiple applications at a moment after a second swipe gesture in a direction opposite to the first direction is detected on the touch-sensitive surface. For example, the device detects a second input that includes contact 508 on a display activation enable representation (e.g., physical button 204 or a virtual button), e.g., that satisfies a device unlock criterion for unlocking the device (if the device is locked), and in response to the second input, the device displays a home screen (e.g., application springboard user interface 400, as shown in FIG. 1 ). Figure 5E In some embodiments, the device unlocking criteria requires that the characteristic strength of the contact in the detected input reaches or exceeds a corresponding strength threshold (e.g., if the display activation indication is a virtual button). For example, in some embodiments, the device unlocking criteria includes when the characteristic strength of the contact increases to the deep press strength threshold IT D In some embodiments, the device unlocking criteria require contact to physically actuate the display activation indication (if the display activation indication is a physical button 204), for example, Figure 5DIn some embodiments, in response to detecting the second input, the device replaces display of the first user interface (e.g., wake screen user interface 512) with display of a user interface that includes multiple application icons corresponding to different applications of the multiple applications (e.g., the user interface including the multiple application icons is application springboard user interface 400, such as Figure 5E As shown). In some embodiments, the second input includes pressing a physical button (e.g., a home button). In some embodiments, the second input includes detecting an increase in the characteristic intensity of the contact that satisfies the device unlocking criteria for unlocking the device (if the device is locked) and displaying the home screen by one or more sensors for detecting the intensity of contact with the touch-sensitive surface (e.g., at a location corresponding to a virtual button, such as a virtual home button configured to detect force and / or fingerprints). In some embodiments, when the input detected on a corresponding application icon in a user interface including multiple application icons satisfies the application launch criteria, the device launches the application corresponding to the corresponding application icon. In some embodiments, the application launch criteria are satisfied by a tap input, for example, without requiring the characteristic intensity of the tap input to satisfy the corresponding intensity threshold.

[0290] In some embodiments, in response to detecting the second input: the device determines (640) whether the device is in a locked mode with the display on or in an unlocked mode with the display on; and based on determining that the device is in a locked mode with the display on (e.g., the device is not unlocked by a Touch ID scan that occurs when the second input is provided, and / or the device is not unlocked by a previous input provided before the second input is provided), the device displays (e.g., prompts the user to provide an unlock input to unlock the device) an authentication user interface, such as a user verification prompt user interface 558 including a password input interface 560, as shown in FIG. Figure 5L shown.

[0291] In some embodiments, detecting the first input includes (642) detecting activation of a display activation affordance. For example, the first input includes contact 508 on a display activation affordance (e.g., a physical button 204 or a virtual button).

[0292] In some embodiments, the corresponding mini-app object is configured (644) to perform less than a subset of the full functionality of the corresponding application of the corresponding mini-app object.

[0293] In some embodiments, a corresponding mini-app object (e.g., Figure 5UThe mini-app object 5125 shown in FIG. 6 shows (646) an identifier of a corresponding application for the corresponding mini-app object (e.g., text 5131 identifying the corresponding application and / or an icon 5135 identifying the corresponding application, such as Figure 5U shown).

[0294] In some embodiments, the corresponding mini-app object displays (648) a portion of the content from the corresponding application of the corresponding mini-app object. For example, the calendar mini-app 5125 includes a portion of the content from the calendar application, such as Figure 5Z , as shown in calendar application user interface 5144 in (e.g., this portion of the content includes a subset of hours, such as hours close to the current time, from a day's schedule in the calendar application).

[0295] In some embodiments, predefined input (650) on the corresponding mini-app object (e.g., including input corresponding to contact 5143 at the location of the next mini-app object 5132, such as Figure 5Y ) to launch the application corresponding to the corresponding mini-application object (e.g., a calendar application, such as Figure 5Z 5144 in the calendar application user interface).

[0296] In some embodiments, the corresponding mini-app object runs (652) as a standalone application resident in the memory of the device that is distinct from an associated application that also resides in the memory of the device.

[0297] In some embodiments, the corresponding mini-app object runs as an extension or component of the associated application on the device (654).

[0298] In some embodiments, the corresponding mini-app object has (656) a dedicated memory portion for temporarily storing information.

[0299] In some embodiments, the memory portion is accessible (658) by the corresponding full-function application of the corresponding mini-app object.

[0300] In some embodiments, a notification is a data object published by an operating system (or a notification management module of the operating system) by an application (or a server associated with the application) for display outside the normal user interface of the application. A notification may include data retrieved from a data repository that is accessible by the notification and the application associated with the notification. In some embodiments, a notification may include a programmable component (e.g., a mini-application object or extension) that can dynamically load or generate data for display on a device. In some embodiments, a notification received from an application (or a server associated with an application) includes data that is used to generate a shorter version of the notification for display on a device that is more abbreviated and a longer version of the notification for display on a device that is more complex.

[0301] In some embodiments, a mini-app object is configured to perform less than a subset of the full functionality of the corresponding application.

[0302] In some embodiments, the mini-app object displays an identifier of the corresponding application.

[0303] In some embodiments, a mini-app object displays a portion of content from a corresponding application. For example, a map mini-app object displays a portion of a map displayed in a map application corresponding to the map mini-app object. For example, a calendar mini-app object displays a portion of a calendar displayed in a corresponding calendar application.

[0304] In some embodiments, a predefined input on a mini-app object launches the corresponding application.

[0305] In some embodiments, the mini-app object runs as a separate application resident in the memory of the device that is distinct from the associated application that also resides in the memory of the device. For example, a mini-app object corresponding to a social networking application runs as a single-purpose or simplified application that has less than a subset of the full functionality of the corresponding application, but is associated with the full-featured social networking application. In this embodiment, the mini-app object runs independently of the social networking application, and in scenarios where the social networking application is not running, the mini-app object continues to run.

[0306] In some embodiments, the mini-app object runs as an extension or component of an associated application on the device. For example, a mini-app object for a calendar application is a single feature or operating component of a full-featured calendar application. In this embodiment, if the calendar application is not running (e.g., in the background), the calendar mini-app object does not run either.

[0307] In some embodiments, the mini-app object has a dedicated memory portion for temporarily storing information. In some embodiments, this memory portion is accessible by the corresponding full-function application. For example, a mini-app object for an instant messaging application has a memory portion for temporarily storing a partially written reply message. In this embodiment, if the user opens the corresponding application in the middle of writing a reply message, the content of the reply message is retrieved from the temporary storage location and used by the full-function application to allow the user to complete his reply message.

[0308] In some embodiments, the mini-app is a combination of any of the features described in paragraphs

[00299] to

[00305] .

[0309] It should be understood that FIG. 6A to FIG. 6E The specific order in which the operations in the description are described is merely exemplary and is not intended to represent that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with reference to other methods described herein (e.g., methods 700, 800, and 900) are also applicable in a similar manner to the details of the processes described above with reference to FIG. 6A to FIG. 6E Method 600 described herein. For example, the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described above with reference to method 600 optionally have one or more of the features of the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described herein with reference to other methods described herein (e.g., methods 700, 800, and 900). For the sake of brevity, these details are not repeated here.

[0310] 7A to 7E A flowchart of a method 700 for accessing controls from a user interface opened from a display according to some embodiments is shown. Method 700 is performed on an electronic device (e.g., Figure 3 Device 300 in, or Figure 1A The method 700 is performed on a portable multifunction device 100 in FIG. 10A . In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 700 are optionally combined, and / or the order of some operations is optionally changed.

[0311] As described below, method 700 provides an intuitive way to access controls from a user interface opened from a display. The method reduces the cognitive burden on the user when accessing controls, thereby creating a more efficient human-machine interface. For battery-powered electronic devices, users are enabled to access controls faster and more efficiently, which saves power and increases the time between battery charges.

[0312] With the device in a display-off state (e.g., a state in which the display is turned off or dimmed while a touch-sensitive surface in the touch screen remains active, a state in which the touch screen is turned off, and / or a state in which display functionality of the touch screen is turned off, e.g., implemented in a sleep / hibernation mode, a power saving mode, or another mode with limited functionality), the device detects (702) a first input, e.g., Figure 5B Input is shown via contact 508 at physical button 204 to activate the display of the device (eg, wake up the device or wake up a touch screen display).

[0313] In response to detecting the first input (704): the device activates the display of the device and displays a first user interface 512 (eg, as shown in FIG. 5 ) on the display corresponding to the display on state of the device. Figure 5C The wake-up screen 512 shown is a user interface that is displayed immediately when waking up the device or activating the display of the device).

[0314] While displaying a first user interface corresponding to a display-on state of the device, the device detects ( 706 ) a swipe gesture on the touch-sensitive surface.

[0315] In response to detecting a swipe gesture (708): determining that the swipe gesture is in a first direction (e.g., Figure 5AH and Figure 5AI 51), the device replaces the display of the first user interface 512 with the display of the camera application user interface 5172. Based on determining that the swipe gesture is in a second direction different from the first direction (e.g., along Figure 5T-1 The path indicated by arrow 5120 and / or along Figure 5T-3 The device replaces the display of first user interface 512 with the display of mini-app object user interface 5122, which is configured to include multiple mini-app objects (e.g., calendar mini-app 5125, weather mini-app 5126, next mini-app 5132, and / or mail mini-app 5162). The corresponding mini-app objects of the multiple mini-app objects have corresponding applications (e.g., calendar application, weather application, and / or mail application) stored in the device. Based on determining that the swipe gesture is along a third direction different from the first direction and the second direction (e.g., along Figure 5AK Upwards along the path indicated by arrow 5206, and / or along the path indicated by arrow 5206 Figure 5AS The device displays the first page of the multi-page control panel user interface 5208. For example, the device displays the multi-page control panel user interface 5208, which includes: Figure 5AL The first page shown has control indicators (e.g., 5210 to 5232) for adjusting device settings; Figure 5AM The second page shown has a second control user interface, which includes media player control enable representations (e.g., 5242 to 5252); and / or as Figure 5AN The third page shown has a third control user interface, which includes remote device control enable representations (e.g., 5272 to 5286). According to determining that the swipe gesture is along a fourth direction different from the first direction, the second direction, and the third direction (e.g., along Figure 5F-1 Downwards along the path indicated by arrow 518, and / or along the path indicated by arrow 518 Figure 5F-5 540 , the device displays a notification user interface 520 configured to display multiple notifications (e.g., calendar invitation notification 550, message notification 542, voicemail notification 544, and / or notification 5108). In some embodiments, the notification includes information corresponding to a communication received by the device, such as a phone call, video call, voicemail, email, SMS, MMS, and / or calendar invitation. In some embodiments, the notification is a calendar appointment notification, a calendar appointment invitation, a reminder, and / or another type of notification generated by an application installed on the device.

[0316] Providing a camera application user interface, a mini-application object user interface, a multi-page control panel user interface, and a notification user interface accessible via input received in a user interface corresponding to a display-on state of a device enables a user to view information (e.g., commonly used information, such as application information and device settings) on a device display without fully activating the device. Providing access to such information in a display-on state of a device enhances the operability of the device and makes the user-device interface more efficient (e.g., by allowing the user to access information in a display-on state of the device without fully activating the device), which in addition reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0317] In some embodiments, replacing the display of the first user interface 512 with the display of the camera application user interface 5172 includes (710) delaying display of one or more control enable representations associated with the camera application (e.g., such as Figure 5AH and Figure 5AI The control indication 5186 shown in the figure is displayed until the control display interval has passed. For example, until the left gesture is initiated at time t1. Figure 5AI and Figure 5AH Control enable indication 5186 is not displayed until time t3 in the display. In some embodiments, replacing the display of the first user interface 512 with the display of the camera application user interface 5172 includes gradually displaying the camera application user interface when a swipe gesture is received, and displaying one or more control enable indications based on determining that the displayed portion of the camera application user interface 5172 increases to above a threshold portion. In some embodiments, by delaying the display of one or more control enable indications 5186 associated with the camera application until a control display interval has passed or until a threshold portion of the camera application user interface 5172 is displayed, the user is given some time and opportunity to return to a previous user interface (e.g., a home screen, a wake-up screen, etc.) without lifting contact 5176. In other words, navigation of the user interface becomes smoother and more efficient, and the user may change his / her mind after seeing a prompt of the result of a continued swipe gesture in the current swipe direction (e.g., continuing to swipe in the current direction will enter the camera application user interface). In some embodiments, some camera control functions associated with the control enable indication (such as some auxiliary light sensors and / or some front-end control functions) and some control functions not associated with the control enable indication (for example, some back-end control functions) require time and power to activate, and activation of these control functions is delayed until the user's intention to activate the camera application is confirmed by the user continuing to sweep the gesture across the relevant threshold.

[0318] In some embodiments, the device detects a gesture to return to the first user interface during a control display interval (e.g., a swipe gesture in the opposite direction of a swipe gesture that caused display of the camera application user interface) (712); and in response to detecting the gesture to return to the first user interface during the control display interval, the device replaces display of the camera application user interface (e.g., the portion of the camera application user interface that was already displayed) with display of the first user interface (e.g., restoring the first user interface) (714).

[0319] In some embodiments, before displaying a multi-page control panel user interface 5208 (e.g., Figure 5ALIn some embodiments, in response to detecting a gesture at a location corresponding to the first page of the multi-page control panel, the device displays (718) a second page of the multi-page control panel user interface (e.g., including controls for media playback and / or controls for accessory devices (such as home device accessories)), such as a touch-sensitive surface. Figure 5AM The second page of the multi-page control panel user interface 5208 is shown.

[0320] In some embodiments, while displaying the second page of the multi-page control panel user interface 5208, the device detects (720) a gesture (e.g., a left gesture along the path indicated by arrow 5270) at a location on the touch-sensitive surface that corresponds to the second page of the multi-page control panel user interface (e.g., within and / or adjacent to the first page of the multi-page control panel user interface); and in response to detecting the gesture at the location that corresponds to the second page of the multi-page control panel user interface, the device displays (722) a third page of the multi-page control panel user interface (e.g., including controls for one or more accessory devices (such as home devices) that are communicatively coupled to the electronic device), such as Figure 5AN The third page of the multi-page control panel user interface 5208 is shown.

[0321] In some embodiments, while notification user interface 520 is displayed, the device detects (724) a dismiss gesture (e.g., a swipe left, or a dismiss gesture on an enable indication such as an “X” enable indication (e.g., a swipe left on an enable indication (e.g., an “X” enable indication (e.g., a “z” on an ... Figure 5M In some embodiments, when the notification user interface is subsequently displayed, the corresponding notification is not displayed.

[0322] In some embodiments, while the notification user interface is displayed, the device detects (728) a launch gesture (e.g., a tap gesture, such as by tapping a finger at a location corresponding to notification 550 (e.g., a tap gesture) at a location on the touch-sensitive surface that corresponds to the first notification included in the notification user interface. Figure 5P) or a tap gesture made by contact 592 at a location corresponding to the expanded version of the notification); and in response to detecting the launch gesture, the device launches (730) an application corresponding to the first notification (e.g., Figure 5Q shown).

[0323] In some embodiments, while the mini-app object user interface is displayed, the device detects (732) a launch gesture (e.g., a tap gesture) at a location on the touch-sensitive surface corresponding to a first mini-app object included in the mini-app object user interface. For example, the launch gesture is performed by Figure 5Y Input by contact 5143 is shown. In some embodiments, the launch gesture is detected at the expanded version of the mini-app object. In some embodiments, the launch gesture is detected at the non-expanded version of the mini-app object.

[0324] In response to detecting the launch gesture, the device launches (734) the application corresponding to the first mini-application object. For example, in response to the launch gesture performed by contact 5143 at a location corresponding to the next mini-application object 5132, as shown in FIG. Figure 5Y As shown, the corresponding calendar application 5144 is displayed, such as Figure 5Z shown.

[0325] In some embodiments, while displaying mini-app object user interface 5122, the device detects (736) an information expansion gesture (e.g., a tap gesture at a "show more" affordance on a mini-app object tray) on the touch-sensitive surface at a location corresponding to a second mini-app object included in the mini-app object user interface, where the second mini-app object includes mini-app object information (e.g., content, functionality, and / or input devices such as control affordances and / or a keyboard). For example, the information expansion gesture is input via contact 5128 at a location corresponding to "show more" affordance 5127 of calendar mini-app 5125, as shown in FIG. Figure 5U as shown, and / or input made by contact 5139 at a location corresponding to the "show more" indication 5141 of the next mini-application 5132, as shown Figure 5X In response to detecting the information expansion gesture (738): the device expands the second mini-application object (eg, increases the vertical size of the mini-application object, such as Figure 5W An expanded view of the calendar mini-application object 5125 in Figure 5Y The device displays the mini-app object information and the additional mini-app object information in the expanded second mini-app object. Figure 5Y As shown, the expanded view of the next mini-application 5132 includes Figure 5X , as well as additional information such as a map 5134 and / or action items 5136 ("Navigate"), 5138 ("Snooze"), 5140 ("Message Invitees"), and / or 5142 ("Delete Event"). In some embodiments, other mini-app objects move downward and / or scroll away from the bottom of the mini-app object user interface (e.g., as Figure 5Y ), thereby making room for the expanded mini-app object.

[0326] In some embodiments, the respective mini-app object is configured to perform (740) less than a subset of the full functionality of the respective mini-app object's corresponding application.

[0327] In some embodiments, the corresponding mini-app object displays (742) an identifier of the corresponding application for the corresponding mini-app object (e.g., text 5131 identifying the corresponding application and / or an icon 5135 identifying the corresponding application, such as Figure 5U shown).

[0328] In some embodiments, the corresponding mini-app object displays (744) a portion of the content from the corresponding application of the corresponding mini-app object. For example, the calendar mini-app 5125 includes a portion of the content from the calendar application, such as Figure 5Z , as shown in calendar application user interface 5144 in (e.g., a subset of hours, such as hours close to the current time, from a day's schedule in the calendar application).

[0329] In some embodiments, a predefined input on the corresponding mini-app object (e.g., input including contact 5143 at a position corresponding to the next mini-app object 5132, such as Figure 5Y ) launches (746) the corresponding application (e.g., a calendar application, such as Figure 5Z 5144 in the calendar application user interface).

[0330] In some embodiments, the corresponding mini-app object runs (748) as a standalone application resident in the device's memory that is distinct from an associated application that also resides in the device's memory.

[0331] In some embodiments, the corresponding mini-app object runs as an extension or component of the associated application on the device (750).

[0332] In some embodiments, the corresponding mini-app object has (752) a dedicated memory portion for temporarily storing information.

[0333] In some embodiments, the memory portion is accessible by a corresponding full-function application of the corresponding mini-app object (754).

[0334] It should be understood that 7A to 7E The specific order in which the operations in the description are described is merely exemplary and is not intended to represent that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with reference to other methods described herein (e.g., methods 600, 800, and 900) are also applicable in a similar manner to the details of the processes described above with reference to 7A to 7E For example, the contacts, gestures, user interface objects, and animations described above with reference to method 700 optionally have one or more of the features of the contacts, gestures, user interface objects, and animations described herein with reference to other methods described herein (e.g., methods 600, 800, and 900). For the sake of brevity, these details are not repeated here.

[0335] FIG. 8A to FIG. 8C A flowchart of method 800 of adding a mini-app object to a mini-app user interface according to some embodiments is shown. Method 800 is performed on an electronic device (e.g., Figure 3 Device 300 in, or Figure 1A The method 800 is performed on a portable multifunction device 100 in FIG. 1 . In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display and the touch-sensitive surface are separate. Some operations in method 800 are optionally combined, and / or the order of some operations is optionally changed.

[0336] As described below, method 800 provides an intuitive way to add mini-app objects to a mini-app user interface. The method reduces the cognitive burden on a user when adding a mini-app object to a mini-app user interface, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, enabling a user to add mini-app objects to a mini-app user interface more quickly and efficiently saves power and increases the time between battery charges.

[0337] The device displays (802) a first user interface 400 including a plurality of application icons corresponding to different applications of a plurality of applications installed on the device (e.g., the device displays a home screen or a Figure 5AA Application springboard 400 shown).

[0338] The device detects (804) a first input (e.g., a light press or deep press input by first contact 5150) at a location on the touch-sensitive surface that corresponds to a first application icon (e.g., mail application icon 418) in the plurality of application icons, the first application icon corresponding to the first application in the plurality of applications. For example, the first input is an input by contact 5150, such as Figures 5AB to 5AD shown.

[0339] In response to detecting the first input, the device displays (806) the first mini-app object or a preview of the first mini-app object (e.g., a representation or collapsed version of the first mini-app object, such as, for example, a preview of the first mini-app object) in an overlay area (e.g., overlaying application springboard user interface 400). Figure 5AF A preview 5156 of the mini-app object 5162 is shown, as Figure 5AD In some embodiments, one or more functional items corresponding to a mini-app object are not activated in the preview of the mini-app object. Figure 5AD When receiving input at the avatar 5164a of the mini-application preview 5156, no response occurs, but in response to Figure 5AF 5164a of the mini-app object 5162, responds (e.g., displays emails received from and / or sent to the user corresponding to the avatar 5164a). For example, the preview 5156 of the mini-app displays, for example, four avatars 5164a to 5164d. The mini-app object 5162 includes a function for expanding the mini-app object 5162 to display additional avatars (e.g., Figure 5AG5164e through 5164h shown). The first mini-app object (e.g., preview 5156) corresponds to a first application (e.g., a mail application) of the plurality of applications; and the overlay area includes an affordance ("Add widget" affordance 5160) for adding the first mini-app object (e.g., mini-app object 5162 corresponding to preview 5156) to a second user interface (e.g., mini-app user interface 5122), the second user interface displaying a plurality of mini-app objects (e.g., including one or more user-selected mini-app objects and / or one or more default mini-app objects).

[0340] The device detects (808) a second input (e.g., input via second contact 5150, such as a tap input, wherein the characteristic intensity of the contact increases to the light press intensity threshold IT) at a location on the touch-sensitive surface that corresponds to affordance 5160 for adding first mini-app object 5162 to the second user interface. L A light press input above, or where the characteristic intensity of the contact increases to the deep press intensity threshold IT D above deep press input).

[0341] In response to detecting the second input, the device adds (810) the first mini-app object 5162 to the second user interface displaying the plurality of mini-app objects. In some embodiments, the plurality of mini-app objects (e.g., mail mini-app object 5162, calendar mini-app object 5125, and / or next mini-app object 5132, such as the second user interface 5122) are added to the second user interface 5122. Figure 5AF In some embodiments, the corresponding trays of the vertical tray stack correspond to corresponding mini-app objects in the plurality of mini-app objects, and each tray has the same initial tray height.

[0342] Adding a mini-app object to a mini-app object user interface in response to an input corresponding to an application icon is an intuitive and efficient way to facilitate user selection of a desired mini-app program object. Providing the ability to add a mini-app object to a collection of mini-app objects from an application icon enhances the operability of the device and makes the user-device interface more efficient, which in addition reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0343] In some embodiments, while displaying a first user interface (e.g., application springboard user interface 400), the device detects (812) a third input on the touch-sensitive surface 112 that satisfies first criteria, such as navigation criteria for navigating from the first user interface to a second user interface (mini-app user interface 5122) (e.g., the third input is one or more right swipe gestures).

[0344] In some embodiments, in response to detecting the third input, the device replaces display of the first user interface (e.g., application springboard user interface 400) with a second user interface (mini-app user interface 5122) (814), wherein replacing display of the first user interface with the second user interface includes: ceasing to display the first user interface (e.g., application springboard user interface 400); and displaying the second user interface (mini-app user interface 5122), including simultaneously displaying in the second user interface first mini-app object 5162 (e.g., having updated content from the first application) and at least one second mini-app object corresponding to a second application different from the first application among the multiple applications (e.g., calendar mini-app object 5125 and / or next mini-app object 5132, as shown in FIG. 1 ). Figure 5AF ). In some embodiments, the second user interface includes an "edit" affordance for rearranging the mini-app objects in the mini-app object stack and / or adding new mini-app objects to the mini-app object stack.

[0345] In some embodiments, the first mini-application object includes (816) at least one optional information item (e.g., avatar 5164a, 5164b, 5164c, and / or 5164d). In some embodiments, the optional information item is an item that displays information related to a particular type of object, such as a stock list item that displays stock information (e.g., opening price, highest price, price change), a weather forecast item that displays weather forecast information for a location (e.g., a city), avatars 5164 corresponding to one or more email senders who recently and / or frequently sent emails to device 100 and / or received emails from device 100 and / or calendar events.

[0346] While displaying the second user interface, the device detects (818) a fourth input at a location on the touch-sensitive surface that corresponds to the at least one selectable information item.

[0347] In response to detecting the fourth input, the device displays (820) additional information associated with the selectable information item (e.g., in a mini-app object that displays multiple stock lists and corresponding stock prices, each stock list is a selectable information item). In response to detecting a tap input at one of the stock lists in the mini-app object, additional information about the stock list (such as a stock value history) is displayed in the mini-app object. In some embodiments, the information associated with the selectable information item replaces display of at least one previously displayed portion of the mini-app object. In some embodiments, the mini-app object expands and the additional information is displayed in the expanded area of ​​the mini-app object.

[0348] In some embodiments, first mini-app object 5162 is added (822) to the highest position (e.g., as shown in FIG. 1 ) in a vertical stack of mini-app objects (e.g., mini-app objects 5162, 5125, and / or 5132) in the second user interface. Figure 5AF In some embodiments, first mini-app object 5162 is added to another position (e.g., the bottom, leftmost, and / or rightmost) in the stack, grid, or other collection of mini-app objects. In some embodiments, a user can change the position of one or more mini-app objects in the collection of mini-app objects.

[0349] In some embodiments, the first mini-application object includes (824) identification information that identifies the first application (e.g., text 5131 that identifies the corresponding application and / or an icon 5135 that identifies the corresponding application, such as Figure 5U shown).

[0350] In some embodiments, the first mini-app object is configured to provide (826) a subset of the functionality provided by the first application (e.g., obtaining data from a remote device (such as weather data, stock data, traffic data, location data for the remote device, and / or map data), determining upcoming calendar appointments, and / or determining travel times to predefined locations and / or points of interest).

[0351] In some embodiments, the first mini-app object includes (828) a subset of content from the first application (e.g., text, an image, a portion of a calendar, a map (such as a real-time updating map showing the location of the electronic device and / or the location of a vehicle relative to the location of the electronic device), travel times to predefined locations and / or points of interest, weather data, and / or stock data). In some embodiments, the content in the mini-app object is updated at periodic intervals and / or in response to user input.

[0352] In some embodiments, in response to detecting the first input, the device display (830) (e.g., in a panel displayed adjacent to the mini-app object) displays a menu 5154 of selectable options (e.g., one or more quick action menu items) corresponding to the first application.

[0353] In some embodiments, a predefined input on the first mini-app object 5162 launches (832) the first application (e.g., launches a mail application).

[0354] In some embodiments, the first mini-app object runs as a separate application residing in memory of the device that is distinct from the first application (834).

[0355] In some embodiments, the first mini-app object runs as an extension or component of the first application (836).

[0356] In some embodiments, the first mini-app object has (838) a dedicated memory portion for temporarily storing information.

[0357] In some embodiments, the memory portion is accessible by the first application (840).

[0358] It should be understood that FIG. 8A to FIG. 8C The specific order in which the operations in the description are described is merely exemplary and is not intended to represent that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with reference to other methods described herein (e.g., methods 600, 700, and 900) are also applicable in a similar manner to the details of the processes described above with reference to FIG. 8A to FIG. 8C Method 800 described herein. For example, the contacts, gestures, user interface objects, and intensity thresholds described above with reference to method 800 optionally have one or more of the features of the contacts, gestures, user interface objects, and intensity thresholds described herein with reference to other methods described herein (e.g., methods 600, 700, and 900). For the sake of brevity, these details are not repeated here.

[0359] 9A to 9E A flowchart of method 900 for using a multi-page control panel navigation control according to some embodiments is shown. Method 900 is performed on an electronic device (e.g., Figure 3 Device 300 in, or Figure 1AThe method 900 is performed on a portable multifunction device 100 in FIG. 1 . In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 900 are optionally combined, and / or the order of some operations is optionally changed.

[0360] As described below, method 900 provides an intuitive way to use a multi-page control panel navigation control. The method reduces the cognitive burden on a user when using a multi-page control panel navigation control, thereby creating a more efficient human-machine interface. For battery-powered electronic devices, enabling a user to use a multi-page control panel navigation control faster and more efficiently saves power and increases the time between battery charges.

[0361] After displaying an initial user interface on the display (e.g., application user interface 5318, springboard user interface 400 including multiple application icons, or wake screen user interface 512 displayed when the electronic device is awakened from a display off state), the device detects (902) a first gesture on the touch-sensitive surface (e.g., a first swipe gesture, such as an upward swipe made by contact 5322 along the path indicated by arrow 5320, as shown in FIG. 5). Figure 5AU shown).

[0362] In response to detecting the first gesture, the device displays (904) a first page of multi-page control panel 5208 on the display, such as Figure 5AV In some embodiments, the first page of the multi-page control panel is displayed as overlaying a portion of the initial user interface (e.g., overlaying the calendar application user interface 5318). In some embodiments, in accordance with the upward movement of the contact in the first gesture, the first page of the multi-page control panel 5208 gradually rises from the bottom edge of the display. The first page of the multi-page control panel includes multiple device control affordances (e.g., reference Figure 5AL The control affordances are represented by 5210 to 5228. In some embodiments, the control affordances represent controls corresponding to one or more features of the device and / or controls for initiating display of an application program interface (such as an application program interface of an application program run by the device).

[0363] While displaying the first page of the multi-page control panel, the device detects (906) a second gesture on the touch-sensitive surface. For example, the second gesture is a second swipe gesture, such as a horizontal swipe in a first direction orthogonal to the direction of the first swipe gesture (e.g., performed by contact 5325 along the path indicated by arrow 5327). In some embodiments, the second gesture is not associated with a particular control affordance, e.g., the second gesture has the same effect at any location (within and / or adjacent to the multi-page control panel user interface) where the second gesture is received.

[0364] In response to detecting the second gesture, the device displays (908) a second page of multi-page control panel 5208 (eg, Figure 5AW ), wherein: the second page of the multi-page control panel includes a plurality of content playback control affordances (e.g., a playback position control slider, a volume control slider, a play / pause, fast forward and / or rewind controls, such as reference Figure 5AM The control affordances 5242 to 5252 are described. In some embodiments, the content playback control affordances are different from the device control affordances. The second page of the multi-page control panel 5208 replaces the first page of the multi-page control panel 5208 on the display. In some embodiments, during the transition of the first page of the multi-page control panel user interface 5208 replacing the first page of the multi-page control panel user interface 5208, an animation of sliding the first page (e.g., a horizontal slide based on a detected gesture) is displayed.

[0365] Compared to a single-page control panel, a multi-page control panel provides efficient access to commonly used application functions and / or device settings to a greater extent. Typically, the control panel can be accessed via input detected when the device is in any of a variety of device states (e.g., a display-on state, a fully activated state displaying a home screen user interface including multiple application icons, and / or a fully activated state displaying an application user interface). Providing a multi-page control panel increases the number of commonly used features and settings accessible to the user without, for example, exiting an application, fully activating the device, and / or accessing a settings menu to access these features, which enhances the operability of the device and makes the user-device interface more efficient, which in addition reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0366] In some embodiments, while displaying the second page of the multi-page control panel, the device detects (910) a third gesture on the touch-sensitive surface (e.g., a gesture performed by contact 5329 moving along the path indicated by arrow 5331, as shown in FIG. Figure 5AW , or a gesture performed by contact 5339 moving along the path shown by arrow 5337, as Figure 5AYshown).

[0367] In some embodiments, in response to detecting the third gesture, based on determining that the third gesture is of the first gesture type (e.g., the gesture is a downward swipe, such as Figure 5AW 5234; the gesture is a tap outside the second control user interface, and / or the gesture is a tap at a location corresponding to the control user interface disarm indication 5234), the device stops (912) displaying the second page of the multi-page control panel 5208 on the display (and displays an overlay portion of the initial user interface, such as Figure 5AX For example, as the contact 5329 in the third gesture moves downward, the vertical length of the multi-page control panel 5208 decreases, and the multi-page control panel 5208 appears to gradually descend.

[0368] In some embodiments, based on determining that the third gesture is a second gesture type that is different from the first gesture type (for example, the third gesture is a horizontal swipe in the first direction, such as input made by contact 5339 moving along the path indicated by arrow 5337, as shown in FIG. Figure 5AY As shown), the device displays (914) a third page of the multi-page control panel, wherein the third page of the multi-page control panel includes a plurality of remote device control affordances (e.g., Figure 5AZ In some embodiments, the third page of the multi-page control panel includes a virtual remote control enable representation (e.g., 5272 to 5286) for controlling one or more home automation devices, such as reference Figure 5AN In some embodiments, the remote device control affordances are different from the content playback control affordances in the second page of the multi-page control panel and the device control affordances in the first page.

[0369] In some embodiments, the multiple device control enable representations, multiple content playback control enable representations, and / or multiple remote device control enable representations are arranged according to a predefined layout. In some embodiments, the third gesture is not associated with a specific control enable representation, for example, the third gesture has the same effect anywhere the third gesture is detected (within and / or adjacent to the multi-page control panel user interface).

[0370] In some embodiments, the plurality of remote device control affordances included in the third page of the multi-page control panel 5208 correspond to respective defined areas (e.g., “scenes,” such as rooms and / or buildings, for example) in a plurality of defined areas (e.g., house rooms and / or campus buildings) that can be selected by the user (916). Figure 5AZ The different devices in the "living room" shown in FIG. Figure 5AP to Figure 5AQ shown).

[0371] In some embodiments, displaying the third page 5208 of the multi-page control panel includes (918) redisplaying one or more remote device controls included in the third page of the multi-page control panel when the third page of the multi-page control panel is displayed.

[0372] In some embodiments, displaying a third page of a multi-page control panel includes displaying (920) in the third page of the multi-page control panel one or more remote device controls corresponding to a defined area in which the device is currently located (e.g., as determined using GPS and / or Wi-Fi signals received by device 100).

[0373] In some embodiments, the third page of the multi-page control panel includes (922) a limited area column indication enable indication (e.g., an indication 5292 for displaying / stopping display of multiple limited area identifiers), which, when activated, is configured to enable multiple limited area identifiers that can be selected by the user (e.g., a list of "scenes", such as reference Figure 5AP to Figure 5AQ as described) display.

[0374] In some embodiments, the first page of the multi-page control panel 5208 overlays (924) a portion of the initial user interface 5318 (e.g., Figure 5AV ). For example, the first page of the multi-page control panel 5208 and the initial user interface 5318 are displayed simultaneously, and a portion of the initial user interface 5318 is visually obscured by the first page of the multi-page control panel 5208. In some embodiments, the first page of the multi-page control panel is translucent, and a portion of the initial user interface 5318 that exists after the first page of the multi-page control panel 5208 is partially visible through the first page of the multi-page control panel 5208. In some embodiments, the second page of the multi-page control panel 5208 also covers the portion of the initial user interface 5318. In some embodiments, one or more pages of the multi-page control panel user interface 5208 have a fixed size that is smaller than the size of the display and the size of the initial user interface 5318.

[0375] In some embodiments, the initial user interface is (926) an application user interface (e.g., calendar application user interface 5318); and the multi-page control panel 5208 and the application user interface 5318 are displayed simultaneously, for example, as Figure 5AW In some embodiments, a multi-page control panel user interface may be accessed without exiting the application.

[0376] In some embodiments, after stopping displaying the second page of the multi-page control panel, the device detects (928) a fourth gesture on the touch-sensitive surface (e.g., an upward swipe to redisplay the control user interface, such as input by moving contact 5333 along the path indicated by arrow 5335, as shown in FIG. Figure 5AX shown).

[0377] In some embodiments, in response to detecting the fourth gesture, the device redisplays (930) the second page of the multi-page control panel 5208 (e.g., Figure 5AY In some embodiments, when input is received to dismiss a multi-page control panel user interface, and input is subsequently received to redisplay the multi-page control panel user interface, the most recently displayed page of the multi-page control panel user interface (displayed before the input to dismiss the multi-page control panel user interface) is displayed in response to the subsequent input to redisplay the multi-page control panel user interface. For example, if the third page of the multi-page control panel was displayed immediately before the input to dismiss the multi-page control panel was received, the third page of the multi-page control panel (for example, as shown) is displayed again in response to the subsequent input to redisplay the multi-page control panel user interface. Figure 5AR to Figure 5AT shown).

[0378] In some embodiments, the plurality of device control indicators (e.g., reference Figure 5AM The control indicators (5242 to 5252) include (932) at least one toggle control. A toggle control is, for example, a multi-state control, such as a control that sets a component state of an electronic device (e.g., a flashlight control that turns a device flashlight on and off, a Wi-Fi control, a Bluetooth control) and / or a control that sets a mode state of an electronic device (e.g., an enabled mode and a disabled mode) (e.g., a night mode control, an airplane mode control, a do not disturb mode control, a rotation lock control, and / or a mirror mode control (e.g., "Apple TV mirroring: on")). For example, Figure 5BF to Figure 5BG As shown, Wi-Fi control 5212 switches from a disabled state to an enabled state in response to input made through contact 5342.

[0379] In some embodiments, such as Figure 5BA As shown, the second page of the multi-page control panel 5208 includes (934) an indicia (e.g., content routing destination information area 5260) of the routing destination of the currently playing media (e.g., the device to which the media is being routed or sent, such as “Playing on AirPods”).

[0380] In some embodiments, when displaying the second page of a multi-page control panel (e.g., Figure 5BDAs shown), the device detects (936) input at a location on the touch-sensitive surface corresponding to the mark 5260 of the routing destination of the currently playing media (e.g., a swipe gesture performed by moving contact 5323 along the path shown by arrow 5345), where the routing destination of the currently playing media includes a first device corresponding to the first zone (“Bluetooth speaker”). In some embodiments, the routing destination corresponding to the first zone is, for example, headphones connected to the electronic device, speakers of the electronic device, and / or one or more remote devices (e.g., remote speakers and / or remote displays in the first room). In response to detecting input at a location on the touch-sensitive surface corresponding to the mark of the routing destination of the currently playing media, the device routes (938) the media output to a second device corresponding to the second zone (e.g., “Airpods”, such as Figure 5BE In some embodiments, the second zone includes, for example, a remote speaker and / or a remote display in a second room. In some embodiments, the marker 5260 changes to reflect the change in the routing destination (e.g., Figure 5BE shown).

[0381] In some embodiments, the second page of the multi-page control panel 5208 includes (940) a media routing destination column indicating an enable indication (e.g., a chevron adjacent to an indication of the current destination of the currently playing media, such as an enable indication 5262 for showing / stopping display of a routing destination menu).

[0382] In some embodiments, the device detects (942) input at a location on the touch-sensitive surface that corresponds to media routing destination column representation 5262 (e.g., input made by contact 5324, such as Figure 5BA shown).

[0383] In some embodiments, in response to detecting input at a location corresponding to a media routing destination column representation, the device displays (944) a list 5236 of media routing destination options, such as Figure 5BB as shown (eg, replacing the display of at least some of the content in the second page with a list of media routing destination options and / or moving the display position of the at least some of the content).

[0384] In some embodiments, the device populates a list 5326 of media routing destination options with an identifier of the electronic device and at least one additional device. In some embodiments, populating the list of media routing destination options with at least one additional device includes (946): based on determining that the electronic device is currently routing media output to one or more remote devices, populating the list of media routing destination options with the one or more remote devices to which the electronic device is currently routing media output; and based on determining that the electronic device has been previously paired with one or more pairable devices, populating the list of media routing destination options with the one or more pairable devices. In some embodiments, if the number of pairable devices with which the electronic device has been most recently paired exceeds a threshold number, the list is populated with a threshold number of most recent pairable devices. In some embodiments, a pairable device that is not currently detected is not included in the one or more pairable devices. In some embodiments, an electronic device (e.g., such as Figure 5BB The iPhone 5328 shown is the first media routing destination in the list of media routing destination options, followed by the device (if any) to which the electronic device is currently routing media (e.g., Figure 5BB Airpods 5260 shown), followed by the pairable devices (if any) that the electronic device has previously paired with (e.g., Bluetooth speakers 5332, such as Figure 5BB as shown), followed by the signal sending device (if any) detected by the electronic device.

[0385] In some embodiments, based on determining that one or more signaling (e.g., Bluetooth signaling) devices are detected near the electronic device, the device populates (948) a list of media routing destination options with the one or more signaling devices.

[0386] In some embodiments, the electronic device includes (950) one or more sensors for detecting intensity of contact with touch-sensitive surface 112. The device detects a first input, including detecting (952) a first contact 5342 at a location on touch-sensitive surface 112 corresponding to a first control enable representation 5212 in a plurality of device control enable representations, such as Figure 5BF to Figure 5BI In response to detecting the first input (954): determining that the first input satisfies a second criterion (e.g., an enhanced control criterion, which includes that when the characteristic intensity of the contact increases to a deep press intensity threshold IT D The first contact is touched when the touch is touched, and the second contact is touched when the touch is touched, as shown in the strength level meter 5344, and the second standard (e.g., enhanced control standard) requires that the characteristic strength of the first contact meets the first strength threshold to meet the second standard, and the device displays one or more modification options (e.g., 5354, 5356, 5358, 5360, 5362, as shown in the strength level meter 5344) corresponding to the control of the first control enable indication 5212. Figure 5BI 53). For example, the modification options are mode modification options (e.g., enabling Wi-Fi using the selected network) and / or duration modification options (e.g., enabling Wi-Fi for the next hour). In some embodiments, the modification options are displayed in menu 5350. Based on determining that the first input satisfies a third criterion (e.g., a control switching criterion) (e.g., the first input is a tap input), where the third criterion (e.g., the control switching criterion) does not require that the characteristic intensity of the first contact satisfies the first intensity threshold, the device switches the function of the control corresponding to the first control affordance (e.g., enabling / disabling Wi-Fi, as shown). Figure 5BF to Figure 5BG shown).

[0387] In some embodiments, while displaying one or more modification options for a control corresponding to a first control affordance, the device detects (956) activation of a first modification option (e.g., modification option 5356) of the one or more modification options (e.g., by Figure 5BI In some embodiments, the second input is a continuation of the first input made by contact. In some embodiments, the second input is a separate tap input on the modification option. In response to detecting the second input, the device modifies (958) the control corresponding to the first control affordance (e.g., as shown) according to the activated first modification option. Figure 5BJ shown).

[0388] In some embodiments, a corresponding page of a multi-page control panel includes (960) a marker 5236 of the total number of pages the multi-page control panel has (e.g., a dot corresponding to each page), and an indicator corresponding to the currently displayed page of the multi-page control panel is highlighted in the corresponding page (e.g., as shown in FIG. Figure 5AL , Figure 5AM and Figure 5AN For example, the mark 5236 indicates the position of the current page among the multiple pages.

[0389] It should be understood that 9A to 9E The specific order in which the operations in the description are described is merely exemplary and is not intended to represent that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with reference to other methods described herein (e.g., methods 600, 700, and 800) are also applicable in a similar manner to the details of the processes described above with reference to 9A to 9EMethod 900 described herein. For example, the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described above with reference to method 900 optionally have one or more of the features of the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described herein with reference to other methods described herein (e.g., methods 600, 700, and 800). For the sake of brevity, these details are not repeated here.

[0390] According to some embodiments, Fig.10 1 shows a functional block diagram of an electronic device 1000 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of various described embodiments. It should be understood by those skilled in the art that Fig.10 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0391] like Fig.10 As shown in , the electronic device 1000 includes a display unit 1002, a touch-sensitive surface unit 1004, and a processing unit 1008 coupled to the display unit 1002 and the touch-sensitive surface unit 1004. In some embodiments, the electronic device includes one or more sensor units 1006, and the processing unit 1008 is coupled to the display unit 1002, the touch-sensitive surface unit 1004, and the one or more sensor units 1006. In some embodiments, the processing unit 1008 includes a detection unit 1010, an activation unit 1012, a display enabling unit 1014, a replacement unit 1016, and a determination unit 1018.

[0392] When the device is in a display-off state, the processing unit 1008 is configured to detect a first input (e.g., using the detection unit 1010). In response to detecting the first input, the processing unit 1008 is configured to: activate the display unit 1002 of the device (e.g., using the activation unit 1012), and enable display of a first user interface corresponding to the display-on state of the device on the display unit 1002 (e.g., using the display enabling unit 1014). While displaying the first user interface corresponding to the display-on state of the device, the processing unit 1008 is configured to detect a swipe gesture on the touch-sensitive surface unit 1004 (e.g., using the detection unit 1010). In response to detecting a swipe gesture on the touch-sensitive surface unit 1004, the processing unit 1008 is configured to: based on determining that the device is in a locked mode with the display turned on and the swipe gesture is along a first direction, replace (for example, using a replacement unit 1016) the display of the first user interface with the display of a second user interface, the second user interface displaying the first content; and based on determining that the device is in an unlocked mode with the display turned on and the swipe gesture is along the first direction, the processing unit 1008 is configured to replace (for example, using a replacement unit 1016) the display of the first user interface with the display of the second user interface, the second user interface displaying the first content and first additional content, the first additional content not being displayed when the device is in the locked mode with the display turned on.

[0393] In some embodiments, the first content includes one or more notifications; and the first additional content includes restricted notification information.

[0394] In some embodiments, the first content includes one or more mini-app objects; and the first additional content includes restricted mini-app object information.

[0395] In some embodiments, the processing unit 1008 is also configured to: in response to detecting a swipe gesture on the touch-sensitive surface unit 1004: based on determining that the device is in a locked mode with the display on and the swipe gesture is along a second direction different from the first direction, replace the display of the first user interface with a display of a third user interface (for example, using a replacement unit 1016), and the third user interface displays the second content; and based on determining that the device is in an unlocked mode with the display on and the swipe gesture is along the second direction, replace the display of the first user interface with a display of a third user interface (for example, using a replacement unit 1016), and the third user interface displays the second content and second additional content, and the second additional content is not displayed when the device is in the locked mode with the display on.

[0396] In some embodiments, the second content includes one or more mini-app objects; and the second additional content includes restricted mini-app object information.

[0397] In some embodiments, the second content includes one or more notifications; and the second additional content includes restricted notification information.

[0398] In some embodiments, the second user interface includes a first search input area, and the third user interface includes a second search input area.

[0399] In some embodiments, the processing unit 1008 is further configured to: detect input in a corresponding search input area in the first search input area and the second search input area (for example, using the detection unit 1010). In response to detecting the input in the corresponding search area, the processing unit 1008 is configured to: enable display of the first set of search results (for example, using the display enabling unit 1014) based on determining that the device is in a locked mode in which the display is turned on; and enable display of a second set of search results that is different from the first set of search results (for example, using the display enabling unit 1014) based on determining that the device is in an unlocked mode in which the display is turned on.

[0400] In some embodiments, a determination is made as to whether the device is in an unlocked mode or a locked mode with the display on in response to detecting a first input.

[0401] In some embodiments, a determination is made as to whether the device is in a locked mode with the display on or an unlocked mode with the display on in response to detecting a swipe gesture on the touch-sensitive surface unit 1004 .

[0402] In some embodiments, the electronic device includes one or more sensor units 1006, and detecting a first input includes: detecting a change in the environment of the device using one or more sensors 1006 (for example, using detection unit 1010); and determining whether the change in the environment of the device meets a display activation criterion in response to detecting the change in the environment of the device.

[0403] In some embodiments, detecting the first input includes detecting (e.g., using detection unit 1010) activation of a display activation indicator.

[0404] In some embodiments, detecting the first input includes (e.g., using detection unit 1010) detecting contact with the display activation enable indication, which activates the display activation enable indication without actuating the display activation enable indication.

[0405] In some embodiments, detecting the first input includes detecting contact with a display activation indicator, the contact activating the display activation indicator to activate the display of the device, and activating the display activation indicator to perform at least one additional function associated with the display activation indicator.

[0406] In some embodiments, processing unit 1008 is configured to: while displaying the first user interface, detect a second input (e.g., using detection unit 1010). In response to detecting the second input, processing unit 1008 is configured to replace (e.g., using replacement unit 1016) the display of the first user interface with a display of a user interface including a plurality of application icons, the plurality of application icons corresponding to different applications from the plurality of applications.

[0407] In some embodiments, in response to detecting a second input, the processing unit 1008 is configured to: determine whether the device is in a locked mode with the display on or an unlocked mode with the display on (e.g., using the determination unit 1018); and based on determining that the device is in a locked mode with the display on, enable display of the authentication user interface (e.g., using the display enable unit 1014).

[0408] In some embodiments, a corresponding mini-app object is configured to perform less than a subset of the full functionality of the corresponding application of the corresponding mini-app object.

[0409] In some embodiments, a corresponding mini-app object displays an identifier of the corresponding application of the corresponding mini-app object.

[0410] In some embodiments, a respective mini-app object displays a portion of the content from the respective mini-app object's corresponding application.

[0411] In some embodiments, a predefined input on a corresponding mini-app object launches a corresponding application of the corresponding mini-app object.

[0412] In some embodiments, the corresponding mini-app object runs as a standalone application that resides in the device's memory, which is different from the associated application that also resides in the device's memory.

[0413] In some embodiments, the corresponding mini-app object runs as an extension or component of the associated application on the device.

[0414] In some embodiments, the corresponding mini-app object has a dedicated memory portion for temporarily storing information.

[0415] In some embodiments, the memory portion is accessible by a corresponding full-function application of the corresponding mini-app object.

[0416] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (for example, as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0417] References Figures 6A-6E The operation is optionally performed by Figure 1A-1B or Fig.10 . For example, detection operations 602 and 606 and display operations 604 and 608 are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 transmits event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a corresponding predefined event or sub-event is detected, event identifier 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Figure 1A to Figure 1B How the components shown in FIG. 1 can implement other processes.

[0418] According to some embodiments, Fig.11 1 shows a functional block diagram of an electronic device 1100 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of various described embodiments. It should be understood by those skilled in the art that Fig.11 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0419] like Fig.11As shown in , the electronic device 1100 includes a display unit 1102, a touch-sensitive surface unit 1104, and a processing unit 1108 coupled to the display unit 1102 and the touch-sensitive surface unit 1104. In some embodiments, the processing unit 1108 includes: a detection unit 1110, a display enabling unit 1112, an activation unit 1114, and a replacement unit 1116.

[0420] When the device is in a display-off state, the processing unit 1108 is configured to detect a first input (e.g., using the detection unit 1110) to activate the display of the device (e.g., using the activation unit 1114). In response to detecting the first input, the processing unit 1108 is configured to: activate the display unit 1102 of the device (e.g., using the activation unit 1114), and enable display of a first user interface corresponding to the display-on state of the device on the display unit 1102 (e.g., using the display enabling unit 1112). While displaying the first user interface corresponding to the display-on state of the device, the processing unit 1108 is configured to detect a swipe gesture on the touch-sensitive surface unit 1104 (e.g., using the detection unit 1110). In response to detecting a swipe gesture, processing unit 1108 is configured to: based on determining that the swipe gesture is along a first direction, replace (e.g., using replacement unit 1116) display of the first user interface with display of a camera application user interface; based on determining that the swipe gesture is along a second direction different from the first direction, replace display of the first user interface with display of a mini-application object user interface (e.g., using replacement unit 1116), the mini-application object user interface being configured to include multiple mini-application objects, wherein corresponding mini-application objects among the multiple mini-application objects have corresponding applications stored in the device; based on determining that the swipe gesture is in a third direction different from the first direction and the second direction, enable display of a first page of a multi-page control panel user interface (e.g., using display enabling unit 1112); and based on determining that the swipe gesture is in a fourth direction different from the first direction, the second direction, and the third direction, enable display of a notification user interface configured to display multiple notifications (e.g., using display enabling unit 1112).

[0421] In some embodiments, replacing display of the first user interface with display of the camera application user interface includes delaying display of one or more control enable indications associated with the camera application until a control display interval has passed.

[0422] In some embodiments, processing unit 1108 is further configured to detect a gesture to return to the first user interface during the control display interval (e.g., using detection unit 1110). In response to detecting the gesture to return to the first user interface during the control display interval, processing unit 1108 is configured to replace (e.g., using replacement unit 1116) the display of the camera application user interface with the display of the first user interface.

[0423] In some embodiments, the processing unit 1108 is further configured to: when displaying a multi-page control panel user interface, detect a gesture at a location on the touch-sensitive surface unit 1104 corresponding to a first page of the multi-page control panel user interface (for example, using the detection unit 1110); and in response to detecting a gesture at a location corresponding to the first page of the multi-page control panel, enable display of a second page of the multi-page control panel user interface (for example, using the display enabling unit 1112).

[0424] In some embodiments, the processing unit 1108 is further configured to: when displaying the second page of the multi-page control panel user interface, detect a gesture at a location on the touch-sensitive surface unit 1104 corresponding to the second page of the multi-page control panel user interface (for example, using the detection unit 1110); and in response to detecting a gesture at a location corresponding to the second page of the multi-page control panel user interface, enable display of a third page of the multi-page control panel user interface (for example, using the display enabling unit 1112).

[0425] In some embodiments, the processing unit 1108 is further configured to: when displaying the notification user interface, detect (for example, using the detection unit 1110) a release gesture at a location on the touch-sensitive surface unit 1104 corresponding to a corresponding notification included in the notification user interface; and in response to detecting the release gesture, stop displaying the corresponding notification in the notification user interface.

[0426] In some embodiments, the processing unit 1108 is further configured to: when displaying a notification user interface, detect (for example, using the detection unit 1110) a launch gesture at a location on the touch-sensitive surface unit 1104 corresponding to a first notification included in the notification user interface; and in response to detecting the launch gesture, launch an application corresponding to the first notification.

[0427] In some embodiments, processing unit 1108 is further configured to: when displaying the mini-app object user interface, detect (for example, using detection unit 1110) a launch gesture at a location on touch-sensitive surface unit 1104 corresponding to a first mini-app object included in the mini-app object user interface; and in response to detecting the launch gesture, launch an application corresponding to the first mini-app object.

[0428] In some embodiments, processing unit 1108 is configured to: while displaying the mini-app object user interface, detect (e.g., using detection unit 1110) an information expansion gesture at a location on touch-sensitive surface unit 1104 corresponding to a second mini-app object included in the mini-app object user interface, the second mini-app object including mini-app object information; and in response to detecting the information expansion gesture, expand the second mini-app object; and enable (e.g., using display enabling unit 1112) display of the mini-app object information and additional mini-app object information in the expanded second mini-app object.

[0429] In some embodiments, a corresponding mini-app object is configured to perform less than a subset of the full functionality of the corresponding application of the corresponding mini-app object.

[0430] In some embodiments, a corresponding mini-app object displays an identifier of the corresponding application of the corresponding mini-app object.

[0431] In some embodiments, a respective mini-app object displays a portion of the content from the respective mini-app object's corresponding application.

[0432] In some embodiments, a predefined input on a corresponding mini-app object launches a corresponding application of the corresponding mini-app object.

[0433] In some embodiments, the corresponding mini-app object runs as a standalone application that resides in the device's memory, which is different from the associated application that also resides in the device's memory.

[0434] In some embodiments, the corresponding mini-app object runs as an extension or component of the associated application on the device.

[0435] In some embodiments, the corresponding mini-app object has a dedicated memory portion for temporarily storing information.

[0436] In some embodiments, the memory portion is accessible by a corresponding full-function application of the corresponding mini-app object.

[0437] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (for example, as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0438] References 7A to 7E The operation is optionally performed by Figure 1A to Figure 1B or Fig.11 . For example, detection operations 702 and 706 and display operations 704 and 708 are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 transmits event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a corresponding predefined event or sub-event is detected, event identifier 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Figure 1A to Figure 1B How the components shown in FIG. 1 can implement other processes.

[0439] According to some embodiments, Fig.12 1 shows a functional block diagram of an electronic device 1200 configured according to the principles of the various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of the various described embodiments. It should be understood by those skilled in the art that Fig.12 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0440] like Fig.12As shown, the electronic device 1200 includes: a display unit 1202 configured to display a first user interface, the first user interface including a plurality of application icons corresponding to different applications among a plurality of applications installed on the device; a touch-sensitive surface unit 1204; and a processing unit 1208 coupled to the display unit 1202 and the touch-sensitive surface unit 1204. In some embodiments, the processing unit 1208 includes: a detection unit 1210, a display enabling unit 1212, an adding unit 1214, and a replacing unit 1216.

[0441] Processing unit 1208 is configured to detect (e.g., using detection unit 1210) a first input at a location on touch-sensitive surface unit 1204 corresponding to a first application icon in a plurality of application icons, the first application icon corresponding to a first application in a plurality of applications. In response to detecting the first input, processing unit 1208 is configured to enable display of a first mini-application object or a preview of the first mini-application object in an overlay area (e.g., using display enabling unit 1212). In some embodiments, the first mini-application object corresponds to a first application in a plurality of applications; and the overlay area includes an affordance for adding the first mini-application object to a second user interface that displays a plurality of mini-application objects. Processing unit 1208 is further configured to detect (e.g., using detection unit 1210) a second input at a location on touch-sensitive surface unit 1204 corresponding to an affordance for adding the first mini-application object to a second user interface. In response to detecting the second input, processing unit 1208 is configured to add the first mini-application object to a second user interface that displays a plurality of mini-application objects (e.g., using adding unit 1214).

[0442] In some embodiments, while displaying the first user interface, processing unit 1208 is configured to detect (e.g., using detection unit 1210) a third input on touch-sensitive surface unit 1204 that satisfies the first criterion for navigating from the first user interface to the second user interface. In response to detecting the third input, processing unit 1208 is configured to replace the display of the first user interface with the second user interface (e.g., using replacement unit 1216), and replacing the display of the first user interface with the second user interface includes: stopping (e.g., using display enable unit 1212) displaying the first user interface; and enabling (e.g., using display enable unit 1212) displaying the second user interface, including (e.g., using display enable unit 1212) simultaneously displaying the first mini-application object and at least one second mini-application object corresponding to a second application different from the first application in the plurality of applications in the second user interface.

[0443] In some embodiments, the first mini-application object includes at least one optional information item; and the processing unit 1208 is configured to: when displaying the second user interface, detect a fourth input at a position on the touch-sensitive surface unit 1204 corresponding to the at least one optional information item (for example, using the detection unit 1210); and in response to detecting the fourth input, enable the display of additional information associated with the optional information item (for example, using the display enabling unit 1212).

[0444] In some embodiments, the first mini-app object is added at the highest position in a vertical stack of mini-app objects in the second user interface.

[0445] In some embodiments, the first mini-app object includes identification information that identifies the first application.

[0446] In some embodiments, the first mini-app object is configured to provide a subset of the functionality provided by the first application.

[0447] In some embodiments, the first mini-app object includes a subset of the content from the first application.

[0448] In some embodiments, processing unit 1208 is further configured to enable display of a menu of selectable options corresponding to the first application (e.g., using display enabling unit 1212) in response to detecting the first input.

[0449] In some embodiments, a predefined input on the first mini-app object launches the first application.

[0450] In some embodiments, the first mini-app object runs as a separate application that resides in device memory and is distinct from the first application.

[0451] In some embodiments, the first mini-app object runs as an extension or component of the first application.

[0452] In some embodiments, the first mini-app object has a dedicated memory portion for temporarily storing information.

[0453] In some embodiments, the memory portion is accessible by the first application.

[0454] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (for example, as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0455] References FIG. 8A to FIG. 8C The operation is optionally performed by Figure 1A to Figure 1B or Fig.12 . For example, display operations 802 and 806 and detection operations 804 and 808 are optionally implemented by event classifier 170, event identifier 180, and event handler 190. Event monitor 171 in event classifier 170 detects contact on touch-sensitive display 112, and event distributor module 174 transmits event information to application 136-1. The corresponding event identifier 180 of application 136-1 compares the event information with the corresponding event definition 186 and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a corresponding predefined event or sub-event is detected, event identifier 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Figure 1A to Figure 1B How the components shown in FIG. 1 can implement other processes.

[0456] According to some embodiments, Fig.13 1 shows a functional block diagram of an electronic device 1300 configured according to the principles of the various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that implements the principles of the various described embodiments. It should be understood by those skilled in the art that Fig.13 The functional blocks described in the present invention are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further limitation of the functional blocks described herein.

[0457] like Fig.13As shown, the electronic device 1300 includes a display unit 1302, a touch-sensitive surface unit 1304, one or more sensor units 1306, and a processing unit 1308 coupled to the display unit 1302, the touch-sensitive surface unit 1304, and the one or more sensor units 1306. In some embodiments, the electronic device includes one or more sensor units 1006, and the processing unit 1008 is coupled to the display unit 1002, the touch-sensitive surface unit 1004, and the one or more sensor units 1006. In some embodiments, the processing unit 1308 includes: a detection unit 1310, a display enabling unit 1312, a routing unit 1314, a filling unit 1316, a modification unit 1318, and a switching unit 1320.

[0458] When the initial user interface is displayed on the display unit 1302, the processing unit 1308 is configured to detect (e.g., using the detection unit 1310) a first gesture on the touch-sensitive surface unit 1304. In response to detecting the first gesture, the processing unit 1308 is configured to enable (e.g., using the display enabling unit 1312) the display of the first page of the multi-page control panel on the display unit 1302. In some embodiments, the first page of the multi-page control panel includes multiple device control enable representations. While displaying the first page of the multi-page control panel, the processing unit 1308 is configured to detect (e.g., using the detection unit 1310) a second gesture on the touch-sensitive surface unit 1304. In response to detecting the second gesture, the processing unit 1308 is configured to enable (e.g., using the display enabling unit 1312) the display of the second page of the multi-page control panel. In some embodiments, the second page of the multi-page control panel includes multiple content playback control enable representations; and the second page of the multi-page control panel replaces the first page of the multi-page control panel on the display unit 1302.

[0459] In some embodiments, while displaying the second page of the multi-page control panel, the processing unit 1308 is configured to detect (for example, using the detection unit 1310) a third gesture on the touch-sensitive surface unit 1304; and in response to detecting the third gesture, based on determining that the third gesture is the first gesture type, stop displaying (for example, using the display enabling unit 1312) the second page of the multi-page control panel on the display unit 1302.

[0460] In some embodiments, processing unit 1308 is configured to: based on determining that the third gesture is a second gesture type different from the first gesture type, enable display of a third page of the multi-page control panel (e.g., using display enabling unit 1312). In some embodiments, the third page of the multi-page control panel includes multiple remote device control enable representations.

[0461] In some embodiments, multiple remote device control indicators included in a third page of a multi-page control panel represent different devices in corresponding defined areas corresponding to multiple defined areas that can be selected by a user.

[0462] In some embodiments, displaying the third page of the multi-page control panel includes (e.g., using display enable unit 1312) redisplaying one or more remote device controls that were included in the third page of the multi-page control panel when the third page of the multi-page control was displayed.

[0463] In some embodiments, displaying a third page of a multi-page control panel includes (e.g., using display enable unit 1312) displaying one or more remote device controls in the third page of the multi-page control panel, wherein the one or more remote device controls correspond to a defined area where the device is currently located.

[0464] In some embodiments, a third page of a multi-page control panel includes a limited area column indicator that, when activated, is configured to display a plurality of limited area identifiers that are selectable by a user.

[0465] In some embodiments, the first page of a multi-page control panel covers a portion of the initial user interface.

[0466] In some embodiments, the initial user interface is an application user interface; and the multi-page control panel and the application user interface are displayed simultaneously.

[0467] In some embodiments, after stopping displaying the second page of the multi-page control panel, the processing unit 1308 is configured to detect a fourth gesture on the touch-sensitive surface unit 1304 (e.g., using the detection unit 1310); and in response to detecting the fourth gesture, redisplay the second page of the multi-page control panel (e.g., using the display enabling unit 1312).

[0468] In some embodiments, multiple device control indicators include at least one toggle control.

[0469] In some embodiments, the second page of the multi-page control panel includes an indicia of the routing destination of the currently playing media.

[0470] In some embodiments, while displaying the second page of the multi-page control panel, the processing unit 1308 is configured to detect (e.g., using the detection unit 1310) an input at a location on the touch-sensitive surface unit 1304 that corresponds to a mark of a routing destination for the currently playing media. In some embodiments, the routing destination for the currently playing media includes a first device corresponding to the first zone. In response to detecting the input at a location on the touch-sensitive surface unit 1304 that corresponds to the mark of the routing destination for the currently playing media, the processing unit 1308 is configured to (e.g., using the routing unit 1314) route the media output to a second device corresponding to the second zone.

[0471] In some embodiments, the second page of the multi-page control panel includes a media routing destination column indication; and the processing unit 1308 is configured to detect (e.g., using the detection unit 1310) an input at a location on the touch-sensitive surface unit 1304 corresponding to the media routing destination column indication; and in response to detecting the input at a location corresponding to the media routing destination column indication, enable display of a list of media routing destination options (e.g., using the display enable unit 1312).

[0472] In some embodiments, processing unit 1308 is configured to populate a list of media routing destination options with an identifier of the electronic device and at least one additional device (e.g., using populating unit 1316), wherein populating the list of media routing destination options with at least one additional device includes, based on determining that the electronic device is currently routing media output to one or more remote devices, populating the list of media routing destination options with one or more remote devices to which the electronic device is currently routing media output (e.g., using populating unit 1316); and based on determining that the electronic device has been previously paired with one or more pairable devices, populating the list of media routing destination options with one or more pairable devices (e.g., using populating unit 1316).

[0473] In some embodiments, the processing unit 1308 is further configured to populate the list of media routing destination options with one or more signal sending devices (e.g., using the filling unit 1316) based on determining that one or more signal sending devices are detected near the electronic device.

[0474] In some embodiments, the electronic device includes one or more sensor units 1306 for detecting intensity of contact with a touch-sensitive surface unit 1304 (for example, using detection unit 1310); and the processing unit 1308 is configured to detect a first input (for example, using detection unit 1310), including detecting a first contact at a position on touch-sensitive surface unit 1304 corresponding to a first control enable indication among multiple device control enable indications; in response to detecting the first input, based on determining that the first input satisfies a second criterion, wherein the second criterion requires that a characteristic intensity of the first contact satisfies a first intensity threshold to satisfy the second criterion, enable display of one or more modification options of a control corresponding to the first control enable indication (for example, using display enabl...

Claims

1. A method for displaying a user interface, comprising: At an electronic device having a display and one or more input devices: detecting a first input while the electronic device is in a power save mode; as well as In response to detecting the first input: activating the display of the electronic device from the power saving mode, and displaying a first user interface on the display, wherein the first user interface is an initial user interface displayed when the electronic device exits the power saving mode in response to the first input; while displaying the first user interface, detecting a swipe gesture; as well as, In response to detecting the swipe gesture: In response to determining that the swipe gesture is in a first direction and that the electronic device is in an unlocked mode when the swipe gesture is detected, replacing display of the first user interface with a second user interface while maintaining the unlocked mode of the electronic device, and In response to determining that the swipe gesture is in a first direction and that the electronic device is in a locked mode when the swipe gesture is detected, replacing display of the first user interface with a second user interface while maintaining the locked mode of the electronic device, wherein: The first user interface and the second user interface are different from a home screen user interface, the home screen user interface including a plurality of application launch icons corresponding to a plurality of applications; and The second user interface includes a first search input area; detecting an input in the first search input area in the second user interface; and In response to detecting the input in the first search input area in the second user interface: Based on determining that the electronic device is in the locked mode, displaying a first set of search results on the second user interface; and Based on determining that the electronic device is in the unlock mode, a second group of search results different from the first group of search results are displayed on the second user interface.

2. The method according to claim 1, comprising: In response to detecting the swipe gesture: replacing display of the first user interface with display of a third user interface based on determining that the electronic device is in the lock mode of the electronic device and the swipe gesture is in a second direction different from the first direction, the third user interface displaying second content; as well as, Based on determining that the electronic device is in the unlocked mode of the electronic device and the swipe gesture is in the second direction, the display of the first user interface is replaced by the display of the third user interface, and the third user interface displays the second content and second additional content that are not displayed when the electronic device is in the locked mode.

3. The method according to claim 2, wherein: The second content includes one or more mini-application objects; and The second additional content includes restricted mini-application object information.

4. The method according to claim 2, wherein: The second content includes one or more notifications; and The second additional content includes restricted notification information. The method of claim 2 , wherein the third user interface comprises a second search input area.

6. The method according to claim 5, comprising: detecting an input in the second search input area; In response to detecting the input in the second search input area: Based on determining that the electronic device is in the locked mode, displaying a third set of search results; as well as Based on determining that the electronic device is in the unlock mode, a fourth group of search results different from the third group of search results are displayed. 7 . The method of claim 1 , wherein determining whether the electronic device is in the unlocked mode or the locked mode is performed in response to detecting the first input. 8 . The method of claim 1 , wherein determining whether the electronic device is in the locked mode or the unlocked mode is performed in response to detecting the swipe gesture.

9. The method of any one of claims 1-6, wherein detecting the first input comprises: Detecting changes in the environment of the electronic device using one or more sensors; as well as In response to detecting the change in the environment of the electronic device, it is determined whether the change in the environment of the electronic device satisfies a display activation criterion.

10. The method of any one of claims 1-6, wherein detecting the first input comprises detecting activation of a display activation indicator.

11. The method of any one of claims 1-6, wherein detecting the first input comprises detecting contact with a display activation enable indication, the contact activating the display activation enable indication without actuating the display activation enable indication.

12. A method according to any one of claims 1-6, wherein detecting the first input comprises detecting contact with a display activation enable indication, the contact activating the display activation enable indication to activate the display of the electronic device and causing the first user interface to be displayed.

13. The method according to any one of claims 1 to 6, comprising: While displaying the first user interface, detecting a second input; as well as In response to detecting the second input, display of the first user interface is replaced with display of a user interface including a plurality of application icons corresponding to different applications of a plurality of applications.

14. The method of claim 13, comprising: In response to detecting the second input: determining whether the electronic device is in the locked mode or the unlocked mode of the electronic device; as well as Based on determining that the electronic device is in the locked mode, an authentication user interface is displayed.

15. The method according to any one of claims 1 to 6, comprising: In response to detecting the corresponding input: Based on determining that the corresponding input satisfies the device unlocking criteria, the display of the first user interface is replaced with the home screen user interface, and the home screen user interface includes the multiple application launch icons corresponding to the multiple applications.

16. The method according to any one of claims 1 to 6, wherein: Displaying the first set of search results on the second user interface includes displaying at least a first application icon corresponding to a first application on the second user interface in response to the input in the first search input area; as well as Displaying the second set of search results on the second user interface includes displaying the first application icon and at least a representation of first content within a second application on the second user interface, wherein the representation of the first content within the second application is not in the first set of search results.

17. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device having a display and one or more input devices, cause the electronic device to perform the method according to any one of claims 1 to 16.

18. An electronic device, comprising: monitor; one or more input devices; one or more processors; Memory; and One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 16.

Images