Systems and methods for interacting with multiple applications concurrent display on an electronic device with a touch-sensitive display

Abstract

The present disclosure relates to systems and methods for interacting with multiple applications simultaneously displayed on an electronic device having a touch-sensitive display. In one embodiment, an electronic device detects movement of a contact in a direction substantially perpendicular to an edge of a touch-sensitive display while displaying content on the touch-sensitive display, and upon determining that the movement of the contact satisfies a first movement-based threshold, displays a taskbar overlying at least a portion of the content, the taskbar comprising a plurality of affordance representations, the plurality of affordance representations, when selected, cause the electronic device to open respective applications available on the electronic device. While displaying the taskbar and in accordance with a determination that a movement of the contact satisfies a second movement-based threshold greater than the first movement-based threshold, the electronic device activates an application switcher user interface that includes respective selectable representations of at least some applications recently used on the electronic device.

Description

[0001] This application is a divisional application of an invention patent application with an application date of May 25, 2018, application number 201880031922.5, and invention name “System and method for interacting with multiple applications displayed simultaneously on an electronic device with a touch-sensitive display”. Technical Field

[0002] Embodiments herein relate generally to electronic devices with touch-sensitive displays and, more particularly, to systems and methods for multitasking on electronic devices with touch-sensitive displays (e.g., portable multifunction devices with touch-sensitive displays). Background Art

[0003] Handheld electronic devices with touch-sensitive displays are ubiquitous. Although these devices were originally designed for information consumption (e.g., web browsing) and communication (e.g., email), they are rapidly replacing desktop and laptop computers as users' primary computing devices. When using a desktop or laptop computer, these users are able to perform daily multitasking by accessing and using different running applications (e.g., cutting and pasting text from a document into an email). Although the range of new features and applications for handheld electronic devices has grown tremendously, the ability to multitask and switch between applications on a handheld electronic device requires an input mechanism that is completely different from that of a desktop or laptop computer.

[0004] In addition, because the screens of handheld electronic devices are smaller than those of conventional desktop and laptop computers, the need for multitasking is particularly pressing. Some conventional handheld electronic devices attempt to meet this need by recreating a desktop computer interface on a handheld electronic device. However, these attempted solutions do not take into account the following factors: (i) the significant difference in screen size between desktop computers and handheld electronic devices, and (ii) the significant difference between the keyboard and mouse interaction of a desktop computer and the touch input and gesture input of a handheld electronic device with a touch-sensitive display. Other attempted solutions require complex input sequences and menu hierarchies that are even less user-friendly than those provided by a desktop or laptop computer. Therefore, it is desirable to provide an intuitive and easy-to-use system and method for simultaneously accessing multiple functions or applications on a handheld electronic device. Summary of the invention

[0005] Embodiments described herein address the need for systems, methods, and graphical user interfaces that provide intuitive and seamless interaction for multitasking on handheld electronic devices. Such methods and systems optionally supplement or replace conventional touch input or gestures.

[0006] (A1) According to some embodiments, in a device having a touch-sensitive display (e.g., Figure 1C An electronic device (e.g., a touch screen 112) Figure 1A A method is performed at a portable multifunction device 100 of the type described herein. The method includes displaying one or more applications of a first set in a display area that occupies substantially the entire touch-sensitive display. When the one or more applications of the first set are displayed, the method further includes: (i) displaying a taskbar that covers a first portion of the display area, (ii) detecting a contact on a first affordance displayed within the taskbar on the touch-sensitive display, and the first affordance is associated with one or more applications of a second set, and (iii) detecting movement of the contact away from the taskbar. When the contact is detected to be lifted from the touch-sensitive display while the contact is located on a second portion of the display area that is different from the first portion, the method further includes: displaying (i) at least some of the one or more applications of the first set and (ii) at least some of the one or more applications of the second set in a display area that occupies substantially the entire touch-sensitive display. In some cases, a user of a computing system must employ inefficient and time-consuming interactions to operate two or more applications. Allowing the user to use a single simple gesture (e.g., using a single finger to drag the affordance from the taskbar to the main display area) enables the user to easily and efficiently activate a mode in which two or more applications are displayed and operated simultaneously. Enabling the user to easily and efficiently activate the mode enhances the operability of the device and makes the human-machine interface more efficient (e.g., by enabling the user to activate the mode using a single gesture). Additionally, this enables continuous interaction with the touch-sensitive display that would otherwise not be possible due to having to employ multiple gestures to access two or more applications.

[0007] (A1.5) In some embodiments, before lift-off of contact is detected: after the contact moves to within a threshold distance from an edge of a touch-sensitive display adjacent to at least some of the one or more applications in the first set or at least some of the one or more applications in the second set, the one or more applications in the first set are resized to display a second portion of the display area adjacent to the resized one or more applications in the first set.

[0008] (A2) In some embodiments of the method of A1, the method includes: after the contact moves away from the taskbar by at least a threshold distance, adjusting the size of one or more applications of the first set to display a second portion of the display area adjacent to the resized one or more applications of the first set. In some cases, a user of the computing system may be unaware of changes that will result from certain interactions on the touch-sensitive display. Providing clear visual feedback (e.g., adjusting the size of one or more applications of the first set to display the second portion of the display area) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. Providing such clear visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible because the user would have to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0009] (A3) In some embodiments of the method of either A1.5 or A2, displaying (i) at least some of the one or more applications of the first set and (ii) at least some of the one or more applications of the second set in a display area that occupies substantially the entire touch-sensitive display includes: displaying at least some of the one or more applications of the second set in a second portion of the display area, and displaying at least some of the one or more applications of the first set in a remaining portion of the display area adjacent to the second portion of the display area.

[0010] (A4) In some embodiments of the method of any one of A1 to A3, the method further includes, before detecting the lift-off of the contact: displaying a representation of the enable representation near the contact until the contact moves a threshold distance away from the taskbar; and based on determining that the contact has moved at least the threshold distance away from the taskbar, ceasing to display the representation of the enable representation and displaying a thumbnail of content associated with the second application near the contact. In this way, additional visual feedback is provided to the user that helps them recognize and understand the changes that will occur after releasing (or lifting) the contact. Providing this additional visual feedback enhances the operability of the device and makes the human-computer interface more effective (for example, by enabling the user to better understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continued interaction with the touch-sensitive secondary display, which would otherwise not be possible because the user would have to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0011] (A5) In some embodiments of the method of A4, the method further includes: when the contact moves over the first part or the second part, respectively, adjusting the size of the thumbnail to match the display size of the first part or the second part. In this way, further visual feedback is provided to the user, which helps them recognize and understand the changes that will occur after releasing (or lifting off) the contact. Providing this further visual feedback enhances the operability of the device and makes the human-computer interface more effective (for example, by enabling the user to better understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continuous interaction with the touch-sensitive auxiliary display, which would otherwise not be possible because the user has to repeatedly undo or reverse unwanted changes that they cannot anticipate.

[0012] (A6) In some embodiments of the method according to any one of claims A1 to A5, displaying the taskbar includes: detecting an initial contact on the touch-sensitive display; and in response to detecting movement of the initial contact in a direction substantially perpendicular to a bottom portion of the touch-sensitive display, displaying the taskbar in a first portion of the touch-sensitive display. Allowing a user to activate the taskbar with a simple, easy-to-use, and repeatable gesture helps ensure that the user interface works more efficiently because the user does not need to waste time searching for an application-specific affordance, but is instead able to quickly access the taskbar and use the taskbar to open an application-specific affordance. This also helps ensure that the user continues to interact with the electronic device because the user is able to move smoothly from accessing the taskbar to activating a split view operating mode in which the user is able to interact with two different applications simultaneously (as discussed in more detail below).

[0013] (A6.1) In some embodiments of the method of any one of A1 to A4, the method includes, before detecting the lift-off of the contact: displaying a representation of the first enabled representation near the contact until the contact moves a threshold distance away from the taskbar; and based on determining that the contact has moved at least the threshold distance away from the taskbar, displaying a user interface object surrounding the representation of the first enabled representation near the contact.

[0014] (A6.2) In some embodiments of the method of A6.1, the user interface object has a first shape, and the method further includes, before detecting lift-off of the contact: when the contact moves over the first portion or the second portion, respectively, modifying the user interface object to have a second shape or a third shape, and the first shape is different from the second shape and the third shape.

[0015] (A7) In some embodiments of the method according to any one of claims A1 to A6, the first set includes a first application, the second set includes a second application, the affordance is associated with the one second application, and when lift-off of the contact is detected (e.g., when the contact is within a threshold distance of an edge of the touch-sensitive display adjacent to the one first application), the first application and the second application are displayed side by side in the display area. In the case where the first set includes a first application and the second set includes a second application, a user of the computing system is able to activate a split view mode in which the first application and the second application are displayed together by simply dragging the affordance corresponding to the second application on the second portion of the display. Providing clear visual feedback (e.g., displaying the second portion of the display area after the contact moves the threshold distance) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. Providing such clear visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to understand that releasing the contact will activate a split view mode including the first application and the second application). In addition, this enables continued interaction with the touch-sensitive display that would otherwise be impossible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0016] (A7.1) In some embodiments of the method of A7, when lift-off of the contact is detected while the contact is over the first application, the second application is displayed in the display area to cover the first application.

[0017] (A8) In some embodiments of the method of any of A1 to A7, the first set includes at least a first application and a second application, the second set includes a third application, and the enable representation is associated with the one third application. In the case where the first set includes the first application and the second application and the second set includes the one second application, a user of the computing system is subsequently able to add a new application to a split view mode in which the first application and the second application are displayed together by simply dragging the enable representation corresponding to the third application onto the display. Providing clear visual feedback (e.g., displaying a second portion of the display area after the contact moves a threshold distance) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. Additionally, this enables continued interaction with the touch-sensitive display that would otherwise be impossible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0018] (A9) In some embodiments of the method of A8, the method further includes, based on determining that the lift-off of the contact occurs on the second application, replacing the second application with a third application on the touch-sensitive display. In this way, the user is provided with the ability to replace the second application with the third application in split view mode, thereby enabling continuous interaction, wherein the user can quickly and easily switch to a new application that will be viewed simultaneously with the first application.

[0019] (A9.1) In some embodiments of the A9 method, the method further includes: when the contact is detected to be lifted while the contact is on the first application, the second application is displayed in the display area to cover the first application.

[0020] (A10) In some embodiments of the method of any of A1 to A9, the first set includes at least two fixed applications of the first group, the second set includes at least two fixed applications of the second group, and the enable representation is associated with the at least two fixed applications of the second group. In the case where the first set and the second set each include a fixed application group, a user of the computing system is subsequently able to add a new fixed application group to the split view mode by simply dragging the enable representation corresponding to the second group onto the display. Providing clear visual feedback (e.g., displaying a second portion of the display area after the contact moves a threshold distance) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. In addition, this enables continued interaction with the touch-sensitive display, which would otherwise be impossible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0021] (A11) In some embodiments of the method of A10, the method further includes: replacing at least two fixed applications of the first group with at least two fixed applications of the second group based on determining that the lift-off of the contact occurs on the first group of applications. In this way, the user is provided with the ability to quickly switch the first group of fixed applications to the second group of fixed applications, thereby achieving continuous interaction, wherein the user can quickly and easily change to a new group of fixed applications on the display.

[0022] (A12) In some embodiments of the method according to any one of claims A1 to A11, one or more applications of the first set include an instance of the first application, one or more applications of the second set include an additional instance of the first application, and the instance and the additional instance are different instances of the first application. In the case where the first set and the second set each include different instances of the same first application, a user of the computing system is subsequently able to pin two different instances of a single application adjacent to each other. Allowing two different instances to be pinned together provides an improved human-computer interface for the user (e.g., by ensuring that the user can interact with different features of a single application at the same time, such as editing a message in a messaging application while also copying content from a different message in the same messaging application). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0023] (A13) On the other hand, an electronic device is provided, and the electronic device includes: a touch-sensitive display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device will execute any method described in A1 to A12.

[0024] (A14) On the other hand, an electronic device is provided, comprising: a touch-sensitive display and a device for executing any one of the methods described in A1 to A12.

[0025] (A15) In another aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store executable instructions, which, when executed by an electronic device having a touch-sensitive display, will cause the electronic device to perform any of the methods described in A1 to A12.

[0026] (A16) In another aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided, wherein the graphical user interface includes a user interface displayed according to the method described in any one of A1 to A12.

[0027] (B1) According to some embodiments, in a device having a touch-sensitive display (e.g., Figure 1C An electronic device (e.g., a touch screen 112) Figure 1AA method is performed at a portable multifunction device 100 of a touch-sensitive display. The method includes displaying a first application and a second application in a display area that occupies substantially the entire touch-sensitive display. When the first application and the second application are displayed in the display area, the method also includes displaying an affordance near an edge of the second application and detecting a contact on the affordance that moves in a direction across the touch-sensitive display. Based on determining that the direction is a first direction, the method also includes: adjusting the size of the first application to substantially occupy the entirety of the display area; and displaying the second application that covers the resized first application. Based on determining that the direction is a second direction that is substantially perpendicular to the first direction, the method includes moving the second application within the display area. In some cases, users of a computing system must employ inefficient and time-consuming interactions to operate two or more applications. Allowing a user to interact with a handle to activate a sliding mode or move a specific application within the display area enables the user to easily and efficiently move applications and switch between side-by-side views or sliding views. Enabling the user to easily and efficiently move applications and switch between different modes of using two or more applications simultaneously enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to select how to interact with two or more applications simultaneously, e.g., using a side-by-side view or using a sliding / overlay mode). Additionally, this enables continued interaction with the touch-sensitive display that would otherwise not be possible due to having to employ multiple gestures to customize simultaneous interaction with two or more applications.

[0028] (B2) In some embodiments of the method of B1, the method further includes: detecting user input at the first application when the second application overlays the resized first application; and responding to the user input via the first application. Allowing a user to interact with the first application even if the second application overlays the first application also allows the user to customize how they wish to interact with two applications that are displayed simultaneously. Enabling users to customize their user experience in this manner enhances the operability of the device and makes the human-computer interface more efficient (for example, their customized choices will allow them to use various features more efficiently without having to work with the default layout). In addition, this enables continuous interaction with the touch-sensitive display, which would otherwise not be possible because the user is forced to interact with two or more applications that are displayed together in a default manner and without any user-specific customization.

[0029] (B3) In some embodiments of the method of B2, the method includes: after determining that the direction is the first direction, in response to a request to open a third application different from the first application and the second application, (i) displaying the third application in a portion of the display area previously occupied by the first application, and (ii) displaying the second application overlaying the third application. Allowing an overlay application to remain displayed after the user has opened a new application helps ensure continued interaction with features of the electronic device (for example, if the overlay application does not remain displayed, features accessed by the user through the overlay application may have to be reopened again). Because the user is also able to avoid having to reopen features of the overlay application, the operability of the device is improved and the human-computer interface works more efficiently.

[0030] (B4) In some embodiments of the method of any of B1 to B3, the second application is initially displayed adjacent to a first edge of the touch-sensitive display, and moving the second application within the display area includes: (i) moving the second application in a second direction until the second application abuts a second edge opposite to the first edge of the touch-sensitive display; and (ii) displaying the first application in a remaining portion of the display area other than the portion occupied by the second application such that the first application abuts the first edge of the touch-sensitive display. Allowing a user to move an application by dragging an affordance (e.g., a handle displayed on an overlay application) helps the user customize how they want to interact with two applications that are displayed simultaneously. Enabling users to customize their user experience in this manner enhances the operability of the device and makes the human-computer interface more efficient (e.g., their customized choices about where each application should be displayed will allow them to use various features more efficiently without having to work with the default arrangement). Additionally, this enables continued interaction with the touch-sensitive display that would otherwise not be possible because the user is forced to interact with two or more applications that are displayed together in a default manner and without any user-specific customization.

[0031] (B5) In some embodiments of the method of B4, a preview of the portion occupied by the second application is displayed next to the first application before the second application is moved until the second application abuts the second edge. In some cases, a user of the computing system may be unaware of changes that will result from certain interactions on the touch-sensitive display. Providing clear visual feedback (e.g., displaying a preview of the location to which the second application will be moved) gives the user an opportunity to preview the changes that will occur after they release (or lift off) contact. Providing such clear visual feedback enhances the operability of the device and makes the human-computer interface more efficient (e.g., by enabling the user to understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible because the user would have to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0032] (B6) In some embodiments of the method of any one of B1 to B5, the first direction is a downward direction relative to an edge of the second application, and the second direction is a lateral direction substantially perpendicular to the downward direction. Allowing the user to move the handle in different directions helps provide the user with an additional and simple way to flexibly customize how they wish to interact with two applications that are displayed simultaneously. Enabling users to customize their user experience in this manner enhances the operability of the device and makes the human-computer interface more efficient (for example, their customized choices about where each application should be displayed will allow them to use various features more efficiently without having to work with the default layout). In addition, this enables continued interaction with the touch-sensitive display that would otherwise not be possible because the user is forced to interact with two or more applications that are displayed together in a default manner and without any user-specific customization.

[0033] (B7) In some embodiments of the method of any one of B1 to B6, the method includes detecting user input at an edge of the second application before displaying the enable indication, and displaying the enable indication in response to detecting the user input. In this way, a simple gesture is provided to the user for activating the display of the handle, thereby enhancing the operability of the device (e.g., by ensuring that the handle is displayed only when the user needs it).

[0034] (B8) In some embodiments of the method of B7, the user input includes contact at an edge of the second application and movement of the contact along the first direction on the touch-sensitive display. As described above, this provides the user with a simple gesture for activating the display of the handle, thereby enhancing the operability of the device (e.g., by ensuring that the handle is displayed only when the user needs it).

[0035] (B8.1) In some embodiments of the method according to any one of claims B1 to B8, after resizing a first application and displaying a second application covering the resized first application: detecting additional contact on the enabling representation and movement of the additional contact along a third direction opposite to the first direction; in response to detecting that the additional contact has moved a threshold distance along the third direction, stopping displaying the second application covering the resized first application, and redisplaying the first application and the second application in a display area that occupies substantially the entire touch-sensitive display.

[0036] (B8.2) In some embodiments of the method according to any one of claims B1 to B8, the method further includes: after resizing the first application and displaying the second application covering the resized first application: detecting additional contacts on the enabled representation and movement of the additional contact along the first direction; and in response to detecting that the additional contact has moved a second threshold distance on the touch-sensitive display, shrinking the content associated with the second application based on the movement of the additional contact until a user interface object having a representation of the enabled representation corresponding to the second application is displayed.

[0037] (B8.3) In some embodiments of the method of B8.2, based on determining that the additional contact moves on the taskbar and is lifted from the touch-sensitive display, the second application is dismissed and the first application is displayed occupying substantially the entire touch-sensitive display; and based on determining that the additional contact moves to within a threshold distance of an edge of the touch-sensitive display and is lifted from the touch-sensitive display, the first application and the second application are redisplayed in a display area occupying substantially the entire touch-sensitive display.

[0038] (B9) In some embodiments of the method of any one of B1 to B8, displaying the first application and the second application in the display area includes simultaneously displaying the first application and the second application side by side in the display area.

[0039] (B10) On the other hand, an electronic device is provided, and the electronic device includes: a touch-sensitive display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device will execute any method described in B1 to B9.

[0040] (B11) On the other hand, an electronic device is provided, comprising: a touch-sensitive display and a device for executing any one of the methods described in B1 to B9.

[0041] (B12) In another aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store executable instructions, which, when executed by an electronic device having a touch-sensitive display, will cause the electronic device to perform any of the methods described in B1 to B9.

[0042] (B13) In another aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided, wherein the graphical user interface includes a user interface displayed according to the method described in any one of B1 to B9.

[0043] (C1) According to some embodiments, in a device having a touch-sensitive display (e.g., Figure 1C 112) and one or more processors (e.g., Figure 1A A method is performed at a portable multifunction device 100 of the present invention. The method includes displaying at least two applications in a display area that occupies substantially the entire touch-sensitive display. The method also includes receiving a user input corresponding to a request to display an application switcher user interface. In response to receiving the user input, the method also includes displaying an application switcher user interface on the touch-sensitive display, wherein the application switcher user interface includes corresponding representations of at least some applications that were recently used on the electronic device, and the corresponding representations include a representation of the display area having at least two applications.

[0044] (C2) In some embodiments of the method of C1, displaying an application switcher user interface includes displaying a taskbar that includes multiple enable representations corresponding to applications available on the electronic device. In some embodiments, the method also includes: (i) detecting contact on an enable representation of the multiple enable representations included in the taskbar; (ii) detecting movement of the contact toward the representation of the display area; and (iii) upon detecting lift-off of the contact on a first application of at least two applications included in the representation of the display area, the method also includes stopping displaying the first application in the representation of the display area, and instead displaying an additional application corresponding to the enable representation in the representation of the display area.

[0045] (C3) In some embodiments of the method of any one of C1 to C2, when an application switcher user interface is displayed, the method further includes: (i) detecting additional user input on a first application of at least two applications included in the representation of the display area; (ii) in response to detecting the additional user input, displaying a disabling indication on at least a portion of the first application; and (iii) when a selection of the disabling indication is detected, closing the first application and displaying updated representations of the remaining applications of the at least two applications other than the first application in the application switcher user interface.

[0046] (C4) In some embodiments of the method of any one of C1 to C3, the method further includes: (i) detecting a contact on a first application of at least two applications included in a representation of the display area; (ii) detecting movement of the contact away from the representation of the display area; and (iii) upon detecting that the contact has moved to a predefined area within an application switcher user interface, ceasing to display the first application in the representation and displaying updated representations of the remaining applications of the at least two applications other than the first application in the application switcher user interface.

[0047] (C5) In some embodiments of the method of C4, a taskbar is displayed to cover at least a predefined area of ​​an application switcher user interface, the taskbar including enable representations corresponding to multiple applications available on the electronic device.

[0048] (C6) In some embodiments of the method of any of C1 to C5, displaying the application switcher user interface includes gradually reducing the size of the display area until it reaches a predefined size associated with the representation.

[0049] (C7) In some embodiments of the method of any of the preceding claims, the corresponding representation includes an additional representation of an additional application associated with the overlay application, and the additional representation does not show the overlay application.

[0050] (C8) In some embodiments of the method described in any of the preceding claims, displaying the at least two applications in the display area includes displaying the at least two applications side by side in the display area, and displaying a representation of the display area with the at least two applications includes simultaneously displaying the at least two applications side by side in the representation of the display area.

[0051] (C9) On the other hand, an electronic device is provided, and the electronic device includes: a touch-sensitive display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device will execute any method described in C1 to C8.

[0052] (C10) On the other hand, an electronic device is provided, comprising: a touch-sensitive display and a device for executing any one of the methods described in C1 to C8.

[0053] (C11) In another aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store executable instructions, which, when executed by an electronic device having a touch-sensitive display, will cause the electronic device to perform any method described in C1 to C8.

[0054] (C12) In yet another aspect, an information processing device for use in an electronic device having a touch-sensitive display is provided, wherein the information processing device comprises: a device for executing the method described in any one of C1 to C8.

[0055] (C13) In another aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided, wherein the graphical user interface includes a user interface displayed according to the method described in any one of C1 to C8.

[0056] (D1) According to some embodiments, a method for activating two or more instances of the same application. For example, a first instance of a mail application (e.g., showing an inbox) is opened on one side of a screen, while a second instance of the same mail application (e.g., showing a single email reply) is opened on the other side of the screen. An instance is a specific single occurrence of an object that typically exists during the runtime of a computer program, e.g., each time a program or application runs, it is an instance of the program. In some embodiments, on a computer with a touch-sensitive display (e.g., Figure 1C 112) and one or more processors (e.g., Figure 1A The method is performed at a portable multifunction device 100 of the present invention. In this embodiment, first, a first instance of an application is displayed on substantially the entire touch-sensitive display. Then, a contact is detected on an affordance displayed within the first instance of the application. Movement of the contact on the touch-sensitive display is detected; upon detection of lift-off of the contact after the contact has moved on the touch-sensitive display and reaches a predefined portion of the touch-sensitive display, a split view operating mode of the electronic device is activated, the mode comprising simultaneously displaying: (i) a second instance of the application that is different from the first instance is displayed as having content associated with the affordance, and (ii) at least a portion of the first instance of the application, wherein the second instance and the portion of the first instance are displayed as occupying substantially the entire touch-sensitive display.

[0057] In some embodiments, the first instance and the second instance of the application are different executable processes executed on one or more processors of the electronic device (e.g., two different instantiations of the application are each executed separately on one or more processors of the electronic device). In other words, after lift-off is detected, the second instance of the application is instantiated and then displayed side-by-side with the first instance of the application, thereby enabling the user to easily / conveniently view and use content from the first instance while also interacting with the second instance.

[0058] In some instances, a user of a computing system loses access to certain features of an application after selecting to turn on other features within the application (e.g., a user may not be able to view a received email after selecting an option to reply to the received email). Opening multiple instances of a single application allows the user to continue to view and use these particular features. Enabling the user to continue to view and use these particular features enhances the operability of the device and makes the human-computer interface more efficient (e.g., by enabling the user to use these particular features without having to provide multiple inputs to turn them back on after turning them off). Additionally, this enables continued interaction with a touch-sensitive display that would otherwise not be possible due to having to employ multiple gestures or inputs to figure out how to turn these features back on or access them after they have been turned off or are no longer displayed.

[0059] (D2) In some embodiments of the method of D1, before activating the split view mode, the system detects at least one of the following: (i) the contact has moved at least a threshold distance (e.g., 1-2 cm) on the touch-sensitive display, and (ii) the contact has moved to a predefined portion of the touch-sensitive display. In other words, in some embodiments, the system determines whether the contact has moved at least a threshold distance (e.g., 1-2 cm) on the touch-sensitive display, while in other embodiments, the system determines whether the contact has moved to a predefined portion of the touch-sensitive display (e.g., the right third to half of the screen). In other embodiments, the system determines whether any one of these moves has occurred, while in other embodiments, the system determines whether two of these moves have occurred. In response to detecting that the contact has moved at least a threshold distance or moved to a predefined portion of the screen, (i) a first instance of the application is resized to occupy a first portion of the touch-sensitive display, and (ii) a predefined portion of the touch-sensitive display adjacent to the first portion is displayed. In this way, a user of the computing system is provided with a preview of the second portion of the display to provide them with a visual preview of how the split view mode will be displayed. This enhances the operability of the device because the user can decide whether to activate the split view mode after viewing the preview, thereby ensuring that the user can discover how to activate the split view mode and ensuring that the user can preview the mode before activating the mode.

[0060] (D3) In some embodiments of the method of D2, activating the split view operating mode includes displaying a second instance of the application in a predefined portion of the touch-sensitive display.

[0061] (D4) In some embodiments of the method of D1, a preview of the content associated with the affordance is displayed below the contact before movement of the contact on the touch-sensitive display is detected. In some embodiments, the preview is displayed only after the contact has been maintained on the affordance for a predetermined period of time (e.g., 1 or 2 seconds). In other embodiments, the preview is displayed only after the contact strength exceeds a strength threshold (e.g., a hard press or a deep press).

[0062] (D5) In some embodiments of the method of D4, the application is an email application, the affordance is a representation of a message displayed in a first instance of the email application, and a preview is displayed when a contact is detected having a contact intensity greater than a predefined intensity threshold.

[0063] (D6) In some embodiments of the method of D4, the application is an email application, the enabling representation is a representation of a message displayed in a first instance of the email application, and the preview is displayed when contact is detected on the enabling representation on the touch-sensitive display for more than a predefined amount of time (e.g., 1, 2, or 3 seconds).

[0064] (D7) In some embodiments of the methods of D1 to D4, the application is an email application, and the contact on the enabling representation is a contact on a link displayed in the first instance of the email application (e.g., a Reply, Reply All, or Compose New Message link).

[0065] (D8) On the other hand, an electronic device is provided, and the electronic device includes: a touch-sensitive display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device will execute any method described in D1 to D7.

[0066] (D9) On the other hand, an electronic device is provided, comprising: a touch-sensitive display and a device for executing any one of the methods described in D1 to D7.

[0067] (D10) In another aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store executable instructions, which, when executed by an electronic device having a touch-sensitive display, will cause the electronic device to perform any method described in D1 to D7.

[0068] (D11) In another aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided, wherein the graphical user interface includes a user interface displayed according to the method described in any one of D1 to D7.

[0069] (E1) According to some embodiments, in a device having a touch-sensitive display (e.g., Figure 1C An electronic device (e.g., a touch screen 112) Figure 1A A method is performed at a portable multifunction device 100 of the type described herein. The method includes: displaying content on a touch-sensitive display; and detecting movement of a contact in a direction substantially perpendicular to an edge of the touch-sensitive display. The method also includes, upon determining that movement of the contact satisfies a first motion-based threshold (e.g., the contact has moved more than a first threshold distance away from the edge), displaying a taskbar covering at least a portion of the content, the taskbar including a plurality of affordances that, when selected, cause the electronic device to open corresponding applications available on the electronic device. Upon displaying the taskbar and based on determining that the contact satisfies a second motion-based threshold (e.g., the contact has moved more than a second threshold distance, which is greater than the first threshold distance away from the edge), the method also includes activating an application switcher user interface that includes corresponding selectable representations of at least some applications that were recently used on the electronic device. By providing a single gesture for activating the taskbar and then activating the application switcher user interface, device operability is enhanced and the human-computer interface operates more efficiently (e.g., because the user is able to provide a single input to activate the taskbar and the application switching user interface, rather than having to employ multiple inefficient and time-consuming inputs to separately access the taskbar and then, at a later point in time, access the application switcher user interface).

[0070] (E1.5) In some embodiments of the method of E1, the first motion-based threshold and the second motion-based threshold are selected from the group consisting of a distance threshold, a speed threshold, or an acceleration threshold.

[0071] (E2) On the other hand, an electronic device is provided, and the electronic device includes: a touch-sensitive display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device will perform any method described in E1 to E1.5.

[0072] (E3) On the other hand, an electronic device is provided, comprising: a touch-sensitive display and a device for executing any one of the methods described in E1 to E1.5.

[0073] (E4) In another aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store executable instructions, which, when executed by an electronic device having a touch-sensitive display, will cause the electronic device to perform any method described in E1 to E1.5.

[0074] (E5) In another aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided, wherein the graphical user interface comprises a user interface displayed according to the method described in any one of E1 to E1.5.

[0075] It should be noted that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described in this specification are not comprehensive, and in particular, many additional features and advantages will be apparent to those of ordinary skill in the art based on the drawings, the specification, and the claims. In addition, it should be noted that the language used in this specification is selected in principle for readability and instructional purposes, and may not be selected to describe or define the subject matter of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] For a better understanding of the various described embodiments, reference should be made to the following detailed description section in conjunction with the following drawings, wherein like reference numerals refer to corresponding parts throughout the drawings.

[0077] Figure 1A is a high-level block diagram of a computing device with a touch-sensitive display according to some embodiments.

[0078] Figure 1B is a block diagram of exemplary components for event processing according to some embodiments.

[0079] Figure 1C is a schematic diagram of a portable multifunction device with a touch-sensitive display according to some embodiments.

[0080] Figure 1D is a schematic diagram used to illustrate a computing device according to some embodiments, the computing device having a touch-sensitive surface separate from the display.

[0081] Figure 2 is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to show a user interface of a menu of an application.

[0082] FIG. 3A to FIG. 3C Examples of dynamic intensity thresholds according to some embodiments are shown.

[0083] FIG. 4A to FIG. 4AR is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to illustrate a user interface for interacting with multiple applications simultaneously (and a gesture for activating a split view mode in which multiple applications can be interacted with simultaneously).

[0084] FIG. 5A to FIG. 5Ois a schematic diagram of a touch-sensitive display according to some embodiments, which is used to illustrate a user interface for switching between a split view mode (in which two applications are pinned together and displayed side by side on the display) and a slide mode (in which one application is displayed as covering another application on the display).

[0085] FIG. 6A to FIG. 6Q is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to illustrate a continuous gesture that allows first activating a taskbar and then activating an application switcher user interface on the display.

[0086] FIG. 7A to FIG. 7L is a schematic diagram of a touch-sensitive display illustrating an exemplary user interface and gestures for activating two or more instances of the same application.

[0087] FIG. 8A to FIG. 8C is a flowchart representation of a method for activating a split view mode (in which multiple applications are displayed and available for use simultaneously) by dragging an enable representation out of a taskbar according to some embodiments.

[0088] 9A to 9C is a flowchart representation of a method for interacting with a single enable representation to switch between a split view mode (in which two applications are pinned together and displayed side by side on a display) and a slide mode (in which one application is displayed as overlaying another application on the display) according to some embodiments.

[0089] FIG. 10A to FIG. 10C is a flowchart representation of a method for utilizing an application switcher user interface to display representations of applications pinned together according to some embodiments.

[0090] FIG. 11A to FIG. 11B is a flow chart depicting a method 1100 for activating two or more instances of the same application.

[0091] Fig.12 is a flowchart representation of a method utilizing a continuous gesture that allows first activating a taskbar and then activating an application switcher user interface on a display. DETAILED DESCRIPTION

[0092] Figures 1A to 1D and Figure 2 A description of an exemplary apparatus is provided. FIG. 3A to FIG. 3C An embodiment of a dynamic intensity threshold is shown. Figures 4A to 4AAare schematic diagrams of a touch-sensitive display for illustrating a user interface for interacting with multiple applications simultaneously (and a gesture for activating a split view mode in which multiple applications can be interacted with simultaneously), and these figures are used to illustrate FIG. 8A to FIG. 8C The method / process shown in . FIG. 5A to FIG. 5O is a schematic diagram of a touch-sensitive display for illustrating a user interface for switching between a split view mode (in which two applications are pinned together and displayed side by side on the display) and a slide over mode (in which one application is displayed overlaying another application on the display), and the figures are for illustrating FIG. 9A to FIG. 9B method / process. FIG. 6A to FIG. 6Q is a schematic diagram of a touch-sensitive display for illustrating a continuous gesture that allows first activating a taskbar (dock) and then activating an application switcher user interface on the display, and these figures are used to illustrate FIG. 10A to FIG. 10C and Fig.12 method / process. FIG. 7A to FIG. 7L are schematic diagrams of a touch-sensitive display for illustrating a user interface for simultaneously interacting with multiple instances of a single application (e.g., multiple different executable instances of a single email application), and the figures are for illustrating FIG. 11A to FIG. 11B method / process.

[0094] Exemplary Devices

[0095] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail, so as not to unnecessarily obscure various aspects of the embodiments.

[0096] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, without departing from the scope of the various described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact.

[0097] The terms used in the description of various described embodiments in this article are only for the purpose of describing specific embodiments, and are not intended to be limited. As used in the description of various described embodiments and in the appended claims, the singular forms "a", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more items in the items listed in association. It will also be understood that the terms "including" and / or "comprising" are used in this specification to specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their grouping.

[0098] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0099] The disclosure herein interchangeably refers to detecting touch input at, at, above, on top of, or substantially within a particular user interface element or portion of a touch-sensitive display. As used herein, a touch input detected “at” a particular user interface element can also be detected “on,” “over,” “on top of,” or “substantially within” the same user interface element. In some embodiments and as described in more detail below, the desired sensitivity level for detecting touch input is configured by a user of the electronic device (e.g., a user can decide (and configure the electronic device to be operated) that touch input should only be detected when the touch input is completely within a user interface element).

[0100] Embodiments of electronic devices, user interfaces for such devices, and processes associated with using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple iPod from Apple Inc (Cupertino, California). IPOD and Device. Other portable electronic devices, such as laptops or tablets with touch-sensitive surfaces (e.g., touch screen displays and / or trackpads) are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or trackpads).

[0101] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse and / or joystick.

[0102] The device typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0103] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and clear to the user.

[0104] Attention is now directed to embodiments of portable electronic devices having touch-sensitive displays. Figure 1A1 is a block diagram illustrating a portable multifunction device 100 (also interchangeably referred to herein as electronic device 100 or device 100) with a touch-sensitive display 112 according to some embodiments. Touch-sensitive display 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes referred to as or referred to as a touch-sensitive display system. Device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), controller 120, one or more processing units (CPUs) 122, peripheral device interface 118, RF circuit 108, audio circuit 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or a touch pad of device 100). These components optionally communicate via one or more communication buses or signal lines 103.

[0105] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface even when there is no change in the smoothness of the touch-sensitive surface. While such a user's interpretation of touch will be limited by the user's individualized sensory perceptions, many sensory perceptions of touch are common to most users. Thus, when tactile outputs are described as corresponding to a particular sensory perception of a user (e.g., "release click," "press click," "roughness"), unless otherwise stated, the generated tactile outputs correspond to physical displacements of the device or a component thereof that would generate the described sensory perceptions for a typical (or average) user.

[0106] It should be understood that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the EMBODIMENTS 2000 are implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0107] The memory 102 optionally includes high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, or other random access solid-state memory devices), and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 102 optionally includes one or more storage devices located remotely from the one or more processors 122. Access to the memory 102 by other components of the device 100 (such as the CPU 122 and the peripheral device interface 118) is optionally controlled by the controller 120.

[0108] The peripheral device interface 118 can be used to couple the input peripherals and output peripherals of the device to the CPU 122 and the memory 102. The one or more processors 122 run or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions of the device 100 and process data.

[0109] In some embodiments, peripherals interface 118, CPU 122, and controller 120 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0110] RF (radio frequency) circuit 108 receives and transmits RF signals, also referred to as electromagnetic signals. RF circuit 108 converts electrical signals into / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, user identity modules (SIM) cards, memory, and the like. RF circuit 108 optionally communicates with networks and other devices through wireless communications, such as the Internet (also referred to as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). Wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE802.11n).

[0111] The audio circuit 110, the speaker 111, and the microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into a sound wave audible to humans. The audio circuit 110 also receives an electrical signal converted from a sound wave by the microphone 113. The audio circuit 110 converts the electrical signal into audio data, and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack. The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single-ear headset or a dual-ear headset) and input (e.g., a microphone).

[0112] The I / O subsystem 106 connects input / output peripherals on the device 100, such as a touch screen 112 and other input control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from other input or control devices 116 or send electrical signals to the other input or control devices. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any one of the following (or not coupled to any one of the following): a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more buttons may optionally include an increase / decrease button for volume control of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button.

[0113] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch screen 112 and / or sends electrical signals to the touch screen. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to a user interface object.

[0114] The touch screen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on tactile and / or haptic contact. The touch screen 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the area under the user's finger.

[0115] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, or OLED (organic light emitting diode) technology, although other display technologies are used in other embodiments. The touch screen 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in Apple Inc. (Cupertino, California) IPOD and Technology in.

[0116] The touch screen 112 optionally has a video resolution of more than 400 dpi. In some embodiments, the touch screen 112 has a video resolution of at least 600 dpi. In other embodiments, the touch screen 112 has a video resolution of at least 1000 dpi. The user optionally uses any suitable object or finger such as a stylus or finger, etc. to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures. In some embodiments, the device converts finger-based input into precise pointer / cursor positions or commands for performing actions desired by the user.

[0117] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface that is separate from the touch screen 112 or an extension of the touch-sensitive surface formed by the touch screen.

[0118] Device 100 also includes a power system 162 for powering the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0119] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the optical sensor 164 optionally captures a static image or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to the touch screen 112 on the front of the device, so that the touch-sensitive display can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device so that the user can optionally obtain an image of the user for video conferencing while watching other video conference participants on the touch-sensitive display.

[0120] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor is shown coupled to force sensor controller 159 in I / O subsystem 106. Contact force sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). Contact force sensor 165 receives contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the rear of device 100 opposite touch screen 112 located on the front of device 100.

[0121] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1AA proximity sensor 166 is shown coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I / O subsystem 106. In some embodiments, when the multifunction device is placed near a user's ear (e.g., when the user is making a phone call), the proximity sensor turns off and disables touch screen 112.

[0122] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1A A tactile output generator is shown coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices, such as a speaker or other audio component; and / or an electromechanical device that converts energy into linear motion, such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts an electrical signal into a tactile output on the device). The contact force sensor 165 receives tactile feedback generation instructions from the tactile feedback module 133 and generates a tactile output on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is arranged in parallel or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on a rear portion of the device 100 opposite the touch-sensitive display 112 located on the front of the device 100 .

[0123] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch-sensitive display based on an analysis of data received from the one or more accelerometers. The device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to the one or more accelerometers 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0124] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, such as Figure 1A . The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display 112; a sensor state, including information obtained from various sensors and input control devices 116 of the device; and position information about the position and / or posture of the device (e.g., the orientation of the device). In some embodiments, the device / global internal state 157 communicates with the multitasking module 180 to track applications activated in multitasking mode (also known as shared screen view, shared screen mode, or multitasking mode). In this way, if the device 100 is rotated from a portrait display mode to a landscape display mode, the multitasking module 180 is able to retrieve multitasking state information (e.g., the display area of ​​each application in the multitasking mode) from the device / global internal state 157 to reactivate the multitasking mode after switching from portrait to landscape. Another embodiment of stateful application behavior in multitasking mode is discussed below with reference to Figures 43A to 45C.

[0125] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.

[0126] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., a universal serial bus (USB), FireWire, etc.) is suitable for direct coupling to other devices, or indirectly coupling through a network (e.g., the Internet, a wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, or similar to, and / or compatible with the 30-pin connector used in some embodiments of the IPOD device from APPLE Inc. In other embodiments, the external port is a multi-pin (e.g., 8-pin) connector that is the same as, or similar to, and / or compatible with the 8-pin connector used in the LIGHTNING connector from APPLE Inc.

[0127] The contact / motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the contact strength (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single contact (e.g., one finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and display controller 156 detect contact on a touch pad.

[0128] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touch-sensitive display hardware, the mouse "click" threshold of the touchpad or touch-sensitive display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0129] The contact / motion module 130 optionally detects gesture input from a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event and then detecting one or more finger drag events, and in some embodiments, also subsequently detecting a finger lift (lift-off) event.

[0130] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch screen 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, without limitation, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

[0131] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application or the like, along with coordinate data and other graphic attribute data if necessary, and then generates screen image data for output to the display controller 156. In some embodiments, the graphics module 132 retrieves the graphics stored with the multitasking data 176 for each application 136 ( Figure 1BIn some embodiments, multitasking data 176 stores multiple graphics of different sizes, allowing applications to quickly resize in shared screen mode (hereinafter combined with FIG. 6A to FIG. 6J , Figures 37A to 37G and Figures 40A to 40D discuss resizing an application in more detail).

[0132] Haptic feedback module 133 includes various software components for generating instructions used by tactile output generator 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

[0133] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts module 137, email client module 140, IM module 141, browser module 147, and any other application requiring text input).

[0134] The GPS module 135 determines the location of the device and provides such information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and maps / navigation widgets).

[0135] Application program ("app") 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0136] Contacts module 137 (sometimes called address book or contact list);

[0137] Telephone module 138;

[0138] Video conferencing module 139;

[0139] Email client module 140;

[0140] Instant messaging (IM) module 141;

[0141] Fitness module 142;

[0142] A camera module 143 for still images and / or video images;

[0143] Image management module 144;

[0144] Browser module 147;

[0145] Calendar module 148;

[0146] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, and a user-created widget 149-6;

[0147] Search module 151;

[0148] A video and music player module 152, optionally consisting of a video player module and a music player module;

[0149] Notepad module 153;

[0150] Map module 154; and / or

[0151] Online video module 155.

[0152] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, website creation applications, disk editing applications, spreadsheet applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, a widget creator module 149-6 for making user-created widgets, and voice replication.

[0153] In combination with the touch screen 112, display controller 156, contact module 130, graphics module 132 and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (for example, stored in the contact module 137 in the memory 102 or the memory 370), including: adding one or more names to the address book; deleting names from the address book; associating telephone numbers, email addresses, physical addresses or other information with names; associating images with names; categorizing and classifying names; providing telephone numbers or email addresses to initiate and / or facilitate communications via the telephone module 138, video conferencing module 139, email client module 140 or IM module 141; and the like.

[0154] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 is optionally used to enter a sequence of characters corresponding to a phone number, access one or more phone numbers in address book 137, modify an entered phone number, dial a corresponding phone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0155] In combination with RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137 and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0156] In conjunction with RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0157] In conjunction with the RF circuit 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for inputting a sequence of characters corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0158] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the video and music player module 146, the health module 142 includes executable instructions for the following operations: creating a fitness (e.g., with a time, distance and / or calorie burn goal); communicating with a fitness sensor (a sports device such as a watch or a pedometer); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting fitness music and playing it; and displaying, storing and transmitting fitness data.

[0159] In conjunction with touch screen 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0160] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still images and / or video images.

[0161] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) in accordance with user instructions.

[0162] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0163] In conjunction with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5) or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).

[0164] In combination with the RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134 and browser module 147, a desktop widget creator module (not shown) is optionally used by a user to create a desktop widget (e.g., to transfer a user-specified portion of a web page to a desktop widget).

[0165] In combination with the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0166] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions to display, present, or otherwise play back video (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes functionality of an MP3 player, such as the IPOD from APPLE Inc.

[0167] In conjunction with the touch screen 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do lists, etc. according to user instructions.

[0168] In combination with the RF circuit 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135 and the browser module 147, the map module 154 is optionally used to receive, display, modify and store executable instructions for maps and data associated with the maps (e.g., driving directions; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0169] In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on the touch screen or on an external display connected via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0170] like Figure 1A As shown, portable multifunction device 100 also includes a multitasking module 180 for managing multitasking operations on device 100 (e.g., communicating with graphics module 132 to determine appropriate display areas for simultaneously displayed applications). Multitasking module 180 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0171] Application selector 182;

[0172] Compatibility module 184;

[0173] Picture-in-picture / overlay module 186; and

[0174] Multitasking history 188, for storing information about the user's multitasking history (e.g., frequently used applications in multitasking mode, most recent display areas of applications in multitasking mode, applications pinned together for display in split view / multitasking mode, etc.).

[0175] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and contact intensity sensor 165, application selector 182 includes executable instructions to display affordances corresponding to applications (e.g., one or more of applications 136) and to allow a user of device 100 to select affordances to be used in multitasking / split view mode (e.g., a mode in which more than one application is simultaneously displayed and active on touch screen 112). In some embodiments, application selector 182 is a taskbar (e.g., taskbar 408 described below).

[0176] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and application selector 182, compatibility module 184 includes executable instructions to determine whether a particular application is compatible with multitasking mode (e.g., by checking a flag, such as a flag stored with multitasking data 176 for each application 136, such as Figure 1B shown).

[0177] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and contact force sensor 165, PIP / overlay module 186 includes executable instructions to determine a reduced size of an application that is to be displayed as an overlay on another application, and to determine an appropriate location on touch screen 112 for displaying the reduced size application (e.g., a location that avoids important content within an active application being covered by the reduced size application).

[0178] Each module and application identified above corresponds to a set of executable instructions for performing one or more functions and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) do not have to be implemented as independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores subsets of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0179] In some embodiments, the device 100 is a device where the operation of a predefined set of functions on the device is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

[0180] A predefined set of functions that are performed exclusively through the touch screen and / or trackpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 to a main menu, a home menu, or a root menu from any user interface displayed on the device 100. In such embodiments, a "menu button" is implemented using a trackpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0181] Figure 1B 1 is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) includes an event classifier 170 (e.g., in an operating system 126) and a selected from a portable multifunction device 100 ( Figure 1A )'s corresponding application 136-1 among the applications 136 (eg, any of the aforementioned applications stored in the memory 102 together with the application 136).

[0182] Event classifier 170 receives event information and determines the application 136-1 and the application view 175 of application 136-1 to which the event information is to be delivered. Event classifier 170 includes event monitor 171 and event distributor module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 175 to which the event information is to be delivered.

[0183] In some embodiments, application internal state 192 includes additional information, such as one or more of the following: resume information to be used when application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready for display by application 136-1, a state queue for enabling a user to return to a previous state or view of application 136-1, and a redo / undo queue of previous actions taken by the user. In some embodiments, multitasking module 180 uses application internal state 192 to help facilitate multitasking operations (e.g., multitasking module 180 retrieves resume information from application internal state 192 in order to redisplay a previously dismissed side application).

[0184] In some embodiments, each application 136-1 stores multitasking data 176. In some embodiments, multitasking data 176 includes a compatibility flag (e.g., a flag accessed by compatibility module 184 to determine whether a particular application is compatible with multitasking mode), a list of compatible sizes for displaying application 136-1 in multitasking mode (e.g., 1 / 4, 1 / 3, 1 / 2, or full screen), and graphics for various sizes (e.g., a different graphic for each size in the compatible size list).

[0185] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

[0186] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0187] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0188] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0189] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0190] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0191] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, higher views in the hierarchy will still remain as actively participating views.

[0192] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizer 178). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizer determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver 181.

[0193] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or is part of another module stored in memory 102, such as contact / motion module 130.

[0194] In some embodiments, application 136-1 includes multiple event handlers 177 and one or more application views 175, each of which includes instructions for handling touch events occurring within a corresponding view of the user interface of the application. Each application view 175 of application 136-1 includes one or more event identifiers 180. Typically, the corresponding application view 175 includes multiple event identifiers 180. In other embodiments, one or more of the event identifiers 180 are part of an independent module, which is a higher-level object such as a user interface toolkit (not shown) or application 136-1 from which methods and other properties are inherited. In some embodiments, the corresponding event handler 177 includes one or more of the following: data updater 177-1, object updater 177-2, GUI updater 177-3, and / or event data 179 received from event classifier 170. Event handler 177 optionally utilizes or calls data updater 177-1, object updater 177-2, or GUI updater 177-3 to update application internal state 192. Alternatively, one or more of application views in application views 175 include one or more corresponding event handlers 177. Additionally, in some embodiments, one or more of data updater 177-1, object updater 177-2, and GUI updater 177-3 are included in the corresponding application view 175.

[0195] The corresponding event identifier 178 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event from the event information. The event identifier 178 includes an event receiver 181 and an event comparator 183. In some embodiments, the event identifier 178 also includes metadata 189 and at least a subset of event delivery instructions 190 (which optionally includes sub-event delivery instructions).

[0196] Event receiver 181 receives event information from event classifier 170. Event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (for example, from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0197] Event comparator 183 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 183 includes event definition 185. Event definition 185 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 177.

[0198] In some embodiments, event definition 186 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 183 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 183 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 177, the event comparator uses the result of the hit test to determine which event handler 177 should be activated. For example, event comparator 183 selects an event handler associated with the sub-event and the object that triggered the hit test.

[0199] In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays the delivery of the event information until it has been determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0200] When a corresponding event recognizer 178 determines that the sub-event sequence does not match any event in event definition 185, the corresponding event recognizer 178 enters an event not possible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0201] In some embodiments, the corresponding event recognizers 178 include metadata 189 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, metadata 189 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 189 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0202] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 178 activates the event handler 177 associated with the event. In some embodiments, the corresponding event recognizer 178 delivers the event information associated with the event to the event handler 177. Activating the event handler 177 is different from sending (and deferred sending) the sub-event to the corresponding hit view. In some embodiments, the event recognizer 178 throws a flag associated with the identified event, and the event handler 177 associated with the flag obtains the flag and performs a predefined process.

[0203] In some embodiments, event delivery instructions 190 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event sequence or to an actively participating view. An event handler associated with a sub-event sequence or with an actively participating view receives the event information and executes a predetermined process.

[0204] In some embodiments, data updater 177-1 creates and updates data used in application 136-1. For example, data updater 177-1 updates phone numbers used in contact module 137, or stores video files used in video or music player module 145. In some embodiments, object updater 177-2 creates and updates objects used in application 136-1. For example, object updater 177-2 creates new user interface objects or updates the position of user interface objects. GUI updater 177-3 updates the GUI. For example, GUI updater 177-3 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display. In some embodiments, GUI updater 177-3 communicates with multitasking module 180 to facilitate adjustment of the size of various applications displayed in multitasking mode.

[0205] In some embodiments, event handler 177 includes or has access to data updater 177-1, object updater 177-2, and GUI updater 177-3. In some embodiments, data updater 177-1, object updater 177-2, and GUI updater 177-3 are included in a single module of the corresponding application 136-1 or application view 175. In other embodiments, they are included in two or more software modules.

[0206] It should be understood that the above discussion of event processing for user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the multifunction device 100, and not all user input is initiated on the touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events defining an event to be recognized.

[0207] Figure 1C201a. The touch-sensitive display optionally displays one or more graphics within the user interface (UI) 201a. In this embodiment, and in other embodiments described below, a user can select one or more of these graphics by, for example, making gestures on the screen with one or more fingers or with one or more styluses. In some embodiments, when the user (for example, by lifting a finger from the screen) interrupts contact with one or more graphics, selection of one or more graphics will occur. In some embodiments, the gesture optionally includes one or more tap gestures (for example, a series of touches to the screen after lifting), one or more swipe gestures (continuous contact during a gesture along the surface of the screen, for example, from left to right, from right to left, up and / or down) and / or rolling of a finger that has made contact with the device 100 (for example, from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with a graphic will not select the graphic. For example, when the gesture used to launch an application is a tap gesture, a swipe gesture that sweeps over an application enable representation (e.g., an icon) may not launch (e.g., open) the corresponding application.

[0208] The device 100 optionally also includes one or more physical buttons, such as a "home" button or a menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112. Figure 5J Additional details and alternative configurations of the primary button 204 are also provided.

[0209] In one embodiment, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for turning the device on and off and locking the device, one or more volume adjustment buttons 208, a subscriber identity module (SIM) card slot 210, a headphone jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and keeping the button pressed for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In an alternative embodiment, the device 100 also accepts voice input for activating or deactivating certain functions through the microphone 113. The device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touch screen 112, and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0210] Figure 1D is used to show that the device (for example, Figure 1A 100) having a touch-sensitive surface 195 (e.g., a tablet or trackpad) that is separate from display 194 (e.g., touch screen 112). In some embodiments, touch-sensitive surface 195 includes one or more contact force sensors (e.g., one or more of contact force sensors 359) for detecting the intensity of contacts on touch-sensitive surface 195, and / or one or more tactile output generators 357 for generating tactile output for a user of touch-sensitive surface 195.

[0211] Although some of the embodiments that follow will be given with reference to input on touch screen 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 1D In some embodiments, the touch-sensitive surface (e.g., Figure 1D 195) has a main axis (e.g., Figure 1D 198) corresponding to the main axis (for example, Figure 1D According to these embodiments, the device detects a position corresponding to a corresponding position on the display (e.g., Figure 1D , 197-1 corresponds to 196-1 and 197-2 corresponds to 196-2) with touch-sensitive surface 195 (e.g., Figure 1D Thus, on a touch-sensitive surface (e.g., Figure 1D 195) and a display of a multi-function device (e.g., Figure 1D194 in the figure) is separated, the user input detected by the device on the touch-sensitive surface (e.g., contact 197-1 and contact 197-2 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0212] In addition, although the following examples are primarily given with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detecting the contact, followed by ceasing to detect the contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and multiple finger contacts are optionally used simultaneously.

[0213] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 1D In the event that an input (e.g., a press input) is detected on touch-sensitive surface 195 (in some embodiments, touch-sensitive surface 195 is a touchpad) in the display, the particular user interface element is adjusted based on the detected input. In a device including a touch screen display (e.g., Figure 1AIn some implementations of a touch-sensitive display system 112 or touch screen 112 in a display device, a contact detected on the touch screen acts as a "focus selector" such that when an input (e.g., a press input by the contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted in accordance with the detected input. In some implementations, the focus moves from one area of ​​the user interface to another area of ​​the user interface without corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves in accordance with the movement of the focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user to convey the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a trackpad or touch-sensitive display), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).

[0214] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in pressure units). The intensity of the contact is used as an attribute of the user input, thereby allowing the user to access additional device functions that the user cannot easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, a touch-sensitive surface, or a physical control / mechanical control such as a knob or button).

[0215] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the portable computing system 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of the touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, the user of the device is provided with a software setting for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0216] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted off, before or after contact starts to move, before contact ends, before or after contact intensity is detected to increase, and / or before or after contact intensity is detected to decrease). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half-maximum value of the contact intensity, the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the intensity of the contact over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this example, a contact whose feature strength does not exceed the first threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the second threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more strength thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0217] In some embodiments, a portion of a gesture is identified for determining a characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an end position (e.g., a drag gesture), where the strength of the contact increases. In this embodiment, the characteristic strength of the contact at the end position may be based only on a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only a portion of the swipe contact at the end position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the strength of the swipe contact to achieve the purpose of determining the characteristic strength.

[0218] In some embodiments, one or more predefined intensity thresholds are used to determine whether a particular input meets the intensity-based criteria. For example, the one or more predefined intensity thresholds include (i) a contact detection intensity threshold IT0, (ii) a light press intensity threshold IT1, and (iii) a light press intensity threshold IT2. L (iii) Deep compression intensity threshold IT D (e.g., at least initially above I L ) and / or (iv) one or more other intensity thresholds (e.g., below I L The intensity threshold I H ). As used herein, IT L and I L Refers to the same light pressure intensity threshold, IT D and I D refers to the same deep compression intensity threshold, while IT H and I H Refers to the same intensity threshold. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, when a contact with a characteristic intensity lower than the light press intensity threshold (e.g., and higher than the nominal contact detection intensity threshold IT0, contacts lower than the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing operations associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface illustrations.

[0219] In some embodiments, the response of the device to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, the strength of the contact exceeding the first strength threshold during the input triggers a first response. In some embodiments, the response of the device to the input detected by the device depends on a criterion including both the contact strength during the input and a time-based criterion. For example, for some "deep press" inputs, as long as a delay time passes between satisfying the first strength threshold and satisfying the second strength threshold, the strength of the contact exceeding the second strength threshold greater than the first strength threshold of the light press during the input triggers a second response. The duration of the delay time is typically less than 200ms (e.g., 40ms, 100ms, or 120ms, depending on the magnitude of the second strength threshold, where the delay time increases as the second strength threshold increases). The delay time helps avoid accidental deep press inputs. As another example, for some "deep press" inputs, a period of reduced sensitivity will occur after reaching the first strength threshold. During this period of reduced sensitivity, the second strength threshold increases. This temporary increase in the second strength threshold also helps avoid accidental deep press inputs. For other deep press inputs, the response to detecting the deep press input is not dependent on time-based criteria.

[0220] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Example factors are described in U.S. patent applications 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety.

[0221] For example, Figure 3A A dynamic intensity threshold 380 is shown that changes over time based in part on the intensity of touch input 376 over time. Dynamic intensity threshold 380 is the sum of two components: a first component 374 that decays over time after a predefined delay time p1 from when touch input 376 is initially detected, and a second component 378 that tracks the intensity of touch input 376 over time. The initial high intensity threshold of first component 374 reduces accidental triggering of a “deep press” response while still allowing an immediate “deep press” response if touch input 376 provides sufficient intensity. Second component 378 reduces unintentional triggering of a “deep press” response through gradual intensity fluctuations of the touch input. In some embodiments, when touch input 376 satisfies dynamic intensity threshold 380 (e.g., within Figure 3A ), triggering a “deep press” response.

[0222] Figure 3B Another dynamic intensity threshold 386 (eg, intensity threshold I D ). Figure 3B Two other intensity thresholds are also shown: a first intensity threshold I H and the second intensity threshold I L .exist Figure 3B Although the touch input 384 satisfies the first intensity threshold I before time p2, H and the second intensity threshold I L , but no response is provided until a delay time p2 has elapsed at time 382. Similarly, Figure 3B , the dynamic intensity threshold 386 decays over time, wherein the decay is from time 382 (triggering the second intensity threshold I L This type of dynamic intensity threshold reduction begins immediately after the trigger is triggered with a lower threshold intensity (such as the first intensity threshold I H Or the second intensity threshold I L ) is triggered after or simultaneously with the response associated with the dynamic intensity threshold I D The associated response.

[0223] Figure 3C Another dynamic intensity threshold 392 (eg, intensity threshold I D ).exist Figure 3C In the example, after a delay time p2 has passed since the touch input 390 was initially detected, the trigger is triggered with the intensity threshold I L At the same time, the dynamic intensity threshold 392 decays after a predefined delay time p1 has passed since the touch input 390 was initially detected. L The associated response then reduces the intensity of the touch input 390, and then increases the intensity of the touch input 390 without releasing the touch input 390, which may trigger the touch input 390 to be in accordance with the intensity threshold I. D The associated response (e.g., at time 394) may be performed even when the intensity of touch input 390 is below another intensity threshold (e.g., intensity threshold I L ) is also the case.

[0224] The contact characteristic strength is below the light press strength threshold IT L The intensity increases to between the light pressure intensity threshold IT L and deep compression intensity threshold IT D The intensity between IT and the deep press intensity threshold is sometimes referred to as a "light press" input. D The intensity of the deep compression increases to above the deep compression intensity threshold IT DThe intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. L A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact lifted off the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some examples, shaded circles or ellipses are used to represent the intensity of a contact on the touch-sensitive surface. In some examples, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0225] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the strength of the contact (or multiple contacts) increases to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting that the strength of the corresponding contact increases to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and the strength of the contact subsequently decreases to below the press input strength threshold, and the corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases to below the press input threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0226] In some embodiments, the device employs intensity hysteresis to avoid unexpected input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the contact strength increases from an intensity equal to or below a hysteresis strength threshold to an intensity equal to or above a press input strength threshold and, optionally, the contact strength subsequently decreases to an intensity equal to or below the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (e.g., an increase in contact strength or a decrease in contact strength, depending on the circumstances).

[0227] For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input are optionally triggered in response to detecting that the intensity of the contact increases above the press input intensity threshold, the intensity of the contact increases from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreases below the press input intensity threshold, or the intensity of the contact decreases below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting that the intensity of the contact decreases below the press input intensity threshold, the operations are optionally performed in response to detecting that the intensity of the contact decreases below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, triggering of these operations is also dependent on satisfying a time-based criterion (e.g., a delay time has elapsed between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0228] Exemplary User Interfaces and Associated Processes

[0229] Attention is now turned to user interface (“UI”) embodiments and associated processes that may be implemented on an electronic device having a display and a touch-sensitive surface, such as device 100 .

[0230] Figure 2 is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to show a user interface of a menu of applications. A similar user interface is optionally provided on device 100 ( Figure 1AIn some embodiments, the user interface 201a includes the following elements or a subset or superset thereof:

[0231] One or more signal strength indicators 202 of one or more wireless communications such as cellular signals and Wi-Fi signals;

[0232] ·time;

[0233] Bluetooth indicator 205;

[0234] Battery status indicator 206;

[0235] A tray 203 with icons for commonly used applications, such as:

[0236] o an icon 216 labeled “Phone” of the phone module 138 , which optionally includes an indicator 214 of the number of missed calls or voice messages;

[0237] o an icon 218 of the email client module 140 labeled “Mail”, which optionally includes an indicator 210 of the number of unread emails;

[0238] o an icon 220 labeled "Browser" of the browser module 147; and

[0239] o an icon 222 labeled “iPod” of a video and music player module 152 (also referred to herein as a video or video browsing application, also referred to as an iPod (trademark of APPLE Inc.) module 152); and

[0240] Icons for other apps, such as:

[0241] o Icon 224 labeled “Messages” of IM module 141;

[0242] o Icon 226 labeled “Calendar” of calendar module 148;

[0243] o Icon 228 labeled “Photos” of the image management module 144;

[0244] o Icon 230 labeled “Camera” of camera module 143;

[0245] o Icon 232 labeled “Online Video” of the online video module 155;

[0246] o Icon 234 labeled “Stock Market” of the Stock Market widget 149-2;

[0247] o an icon 236 labeled “MAP” of the map module 154;

[0248] o Icon 238 labeled “Weather” of the Weather widget 149-1;

[0249] o Icon 240 labeled “Clock” of the alarm clock widget 149-4;

[0250] o An icon 242 labeled “Fitness” of the fitness module 142;

[0251] o Icon 244 labeled "Notepad" of the notepad module 153;

[0252] o an icon 246 of a settings application or module that provides access to settings for the device 100 and its various applications; and

[0253] o Other icons for additional applications such as App Store, iTunes, Voice Memos, and Utilities.

[0254] It should be noted that Figure 2 The icon labels shown in are merely exemplary. Other labels are optionally used for various application icons. For example, icon 242 of fitness module 142 is alternatively labeled "Fitness Support," "Workout," "Workout Support," "Sports," "Sports Support," or "Health." In some embodiments, the label of a corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0255] In some embodiments, the home screen includes two areas: a tray 203 and an icon area 201. Figure 2 As shown, icon area 201 is displayed above tray 203. However, icon area 201 and tray 203 are optionally displayed in locations other than those described herein.

[0256] Tray 203 optionally includes icons of favorite applications of the user on computing device 100. Initially, tray 203 may include a set of default icons. The user may customize tray 203 to include other icons in addition to the default icons. In some embodiments, the user customizes tray 203 by selecting an icon from icon area 201 and dragging and dropping the selected icon into tray 203 to add the icon to tray 203. In order to remove an icon from tray 203, the user selects an icon displayed in the favorites area for a threshold amount of time, which causes computing device 100 to display a control for removing the icon. The user's selection of the control causes computing device 100 to remove the icon from tray 203. In some embodiments, tray 203 is replaced by taskbar 408 (described in more detail below), and therefore, the details provided above with reference to tray 203 may also be applicable to taskbar 408, which may supplement the description of taskbar 408 provided in more detail below.

[0257] In the present disclosure, reference to “split view mode” refers to a mode in which at least two applications are displayed side by side on display 112 at the same time and in which the two applications can interact (e.g., application 1 and a video browsing application are displayed side by side on display 112). Fig. 4I The at least two applications may also be "pinned" together, which refers to an association between the at least two applications (stored in the memory of device 100) that causes the two applications to open together when either of the at least two applications is launched. Figure 5B The enabling indication 506) can be used to cancel the fixed application and instead display one of the at least two applications as covering the other application, and this covering display mode is called a sliding display mode (for example, in Figure 5C In the sliding mode shown, the video browsing application is displayed as an overlay application 1). The user is also able to use a border enable representation that is displayed within a border extending between the at least two applications while they are displayed in a split view mode to unpin or release one of the at least two applications (e.g., by dragging the border enable representation until it reaches the edge of the display 112 adjacent to the first application of the at least two applications, which is then released and the at least two applications are subsequently unpinned). The use of a border enable representation (or a gesture at the border between two applications) to release a pinned application is discussed in more detail in commonly owned U.S. patent application No. 14 / 732,618 (e.g., in Figures 37H to 37M and related descriptive paragraphs), which is hereby incorporated by reference in its entirety.

[0258] Figures 4A to 4AA is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to illustrate an example of a user interface for interacting with multiple applications simultaneously (and a gesture for activating a split view mode in which multiple applications can be interacted with simultaneously).

[0259] In some embodiments, the application is displayed on a touch-sensitive display, such as on a Figure 4A 4 and includes a map of the United States and a search box that allows the map application to be used to search for various locations. While the map application is displayed, gesture 450 is detected near an edge of touch-sensitive display 112 (e.g., the bottom-most edge of the touch-sensitive display relative to the viewer).

[0260] like Figure 4B As shown, in response to detecting that gesture 450 has traveled more than a threshold distance away from the edge (e.g., more than about 1 centimeter, but may depend on the size of touch-sensitive display 112), taskbar 408 is displayed on the touch-sensitive display as an overlay of the map application. As used herein, a taskbar is a user interface element that includes an affordance (e.g., a system-selected affordance (such as based on the time when the corresponding affordance was most recently used) or a user-selected affordance (selected by the user for inclusion in the taskbar)) that, when activated, launches a corresponding function or application. As gesture 450 moves toward an opposite edge of the touch-sensitive display (e.g., upward), more of taskbar 408 is displayed and is displayed as an overlay of the map application. Once gesture 450 has traveled at least a second threshold distance (e.g., 2-4 centimeters, but may depend on the size of touch-sensitive display 112) from the edge of the touch-sensitive display, taskbar 408 is presented in a predefined area of ​​the touch-sensitive display and no longer follows the movement of gesture 450. For example, as Figure 4C As shown, after the gesture moves away from the edge of the touch-sensitive display for more than at least a second threshold distance, the task bar 408 is displayed in the predefined area. In some embodiments, after the end of gesture 450 (e.g., contact is released or lifted from the touch-sensitive display and gesture 450 ends), the task bar 408 remains displayed and stationary in the predefined portion.

[0261] Also like Figure 4E As shown, an application-specific affordance displayed in task bar 408 may be selected to open a corresponding application (e.g., an application corresponding to which application-specific affordance is selected), and gestures may also be used (e.g., Figure 4E452) to drag an application-specific affordance out of taskbar 408. For example, in some embodiments, a simple tap on affordance 232 will launch the application associated with the affordance and display it under the taskbar. Additionally, in some embodiments, as shown by gesture 452, a dragging movement away from the taskbar after contact will launch the application associated with the affordance and display it under the taskbar.

[0262] Furthermore, in some embodiments, Figure 4F As shown, using gesture 452, the application-specific affordance 232 is dragged or pulled away from the taskbar and toward a predefined area 454 of the touch-sensitive display. In some embodiments, predefined area 454 is a portion of the touch-sensitive display that is reserved for displaying an application (e.g., Figure 4F In some embodiments, pinning an application involves creating an association between at least two different applications so that the applications can be subsequently launched together. For example, upon launch, one of the pinned applications is displayed in predefined area 454. As explained in more detail below, dragging a corresponding application-specific affordance toward predefined area 454 launches the associated application, initially displaying the application in a preview while dragging the application-specific affordance, and then displaying the associated application in predefined area 454 (when the contact is lifted) while displaying Application 1. In some embodiments, displaying the associated application in predefined area 454 causes Application 1 to resize or scale to accommodate the display of the associated application in predefined area 454.

[0263] Back to Figure 4F , gesture 452 continues to move across Figure 4G 408 (e.g., the contact associated with gesture 452 moves outside the boundaries of taskbar 408). Upon detecting that gesture 452 has moved beyond the boundaries of taskbar 408, in some embodiments, application-specific affordances 232 are no longer displayed as moving with gesture 452, and instead previews 456 of content associated with the application associated with application-specific affordances 232 (e.g., a video browsing application and content) are displayed to follow gesture 452. Additionally, in some embodiments, predefined area 454 is displayed as being associated with the currently displayed application (e.g., Figure 4G The applications 1) are adjacent.

[0264] In some embodiments, when the gesture moves beyond the border of taskbar 408, a user interface object including an application-specific affordance (e.g., affordance 232) is displayed instead of displaying preview 456. Examples of these user interface objects are shown in FIG. Figure 4AC to Figure 4AE 473 -B) and are shown in FIG. 4 (e.g., user interface objects 473 -A and 473 -B) and are described in more detail below.

[0265] like Figure 4H As shown, as gesture 452 continues to move, preview 456 also moves, and more of predefined area 454 is displayed adjacent to application 1. In conjunction with displaying predefined area 454, the size of application 1 is also adjusted (eg, Figures 4G to 4H ), to occupy the remainder of the touch-sensitive display 112 except the predefined area 454.

[0266] Fig. 4I It is shown that upon detecting the end of gesture 452 (e.g., the contact of the gesture is lifted off the touch-sensitive display), the video browsing application is displayed as occupying predefined area 454 and is now also pinned next to application 1 (e.g., the two applications are now associated and will open together when either application is launched). Boundary affordance 460 is also shown in FIG. Fig. 4I , and in some embodiments may be used to allow a user to resize a fixed application (e.g., in response to detecting contact on border indication 460 and movement of the contact on the touch-sensitive display in a direction substantially from right to left (e.g., Figures 4L to 4M 464), the size of application 1 is reduced or scaled, and the size of the video application is increased or scaled (such as Figures 4L to 4M Additional details regarding the use of border affordances (or dragging borders) to resize and / or unpin a pinned application are also provided in commonly owned U.S. patent application Ser. No. 14 / 732,618, specifically in FIGS. 37H through 37M and related descriptive paragraphs, which are incorporated by reference above in their entirety.

[0267] In some embodiments, the gesture (e.g., gesture 452) is not performed until it progresses beyond a threshold value (e.g., Figures 4AB to 4AE In this way, the user can decide whether to end the gesture on the currently displayed application (e.g., lift off gesture 481 on the currently displayed application 1, Figure 4AH ) to activate slide mode in which an overlay application is displayed over the currently displayed application, or to continue moving the gesture toward one of thresholds 464, 465 to activate split view mode in which a pinned application is displayed (e.g., as Figure 4AE to Figure 4AF As shown, gesture 472 moves toward left hand threshold 465, and then upon detecting liftoff of the contact associated with gesture 472, Figure 4AF The two applications shown in the figure are pinned next to each other). Figure 4AB to Figure 4AI Additional details are provided regarding these features of certain embodiments.

[0268] Fig. 4I Also shown is gesture 461, which includes an initial contact near the top-most edge of display 112 (relative to a user's viewpoint looking down at display 112), followed by movement of the contact in a substantially downward direction across the touch-sensitive display. In response to detecting that the gesture has traveled at least a threshold distance (e.g., 1-2 centimeters) in a substantially downward direction, affordance 462 ( Figure 4J ) is displayed as an overlay over the application on which gesture 461 is detected (e.g., Fig. 4I and Figure 4J ) as shown in the video application.

[0269] Figure 4J to Figure 4L Also shown is a gesture 471 (gesture 471 may include an initial contact proximate to taskbar 408 followed by movement of the contact in a generally downward direction) for moving taskbar 408 out of the touch-sensitive display while at least two applications are currently displayed side-by-side on display 112 (e.g., the at least two applications are pinned together / displayed in split view mode). In this way, taskbar 408 may be hidden so that a user may choose to interact only with applications displayed behind taskbar 408. Similar to the gesture used to activate the taskbar when a single application is displayed on display 112 (e.g., FIG. 4A to FIG. 4C 450), when more than one application is displayed on the display 112 (e.g., when application 1 is pinned next to a video application, such as Figure 4N to Figure 4P As shown), gesture 450 can also be used to activate the taskbar.

[0270] Now go to Figure 4Q , when multiple applications are simultaneously displayed on touch-sensitive display 112, gesture 466 is detected on an application-specific affordance displayed within task bar 408. For example, gesture 466 includes initial contact on affordance 220 corresponding to a web browsing application and then moving the initial contact on touch-sensitive display 112. Gesture 466 is detected on a touch-sensitive display 112. Figure 4R466 continues to move, and the application-specific affordance moves in accordance with the movement of gesture 466 until gesture 466 crosses the boundary of taskbar 408. Upon detecting that gesture 466 has moved beyond the boundary of taskbar 408, in some embodiments, the application-specific affordance is no longer displayed below gesture 466, and a preview 498 of content associated with the web browsing application is displayed in its place ( Figure 4S ).

[0271] Also like Figure 4S As shown, in some embodiments, once gesture 466 moves beyond the boundaries of taskbar 408 (e.g., affordance 492 corresponding to the fixed state of the map application and the video application is displayed in the position previously occupied by the affordance of the web browsing application within taskbar 408), the new affordance is added to the taskbar. In some other embodiments, the new affordance is not added to taskbar 408 until the end of gesture 466 is detected (in other words, before the end of gesture 466 is detected). Figure 4T to Figure 4U Before the lift-off of gesture 466 is detected, affordance 492 is not added to taskbar 408). In some embodiments, the user may choose to return the affordance to the taskbar rather than continue dragging the gesture (or they may end the gesture before reaching predefined area 454 or move the gesture out of the predefined area). In this way, the user is provided with an improved human-computer interface and the ability to continue interacting with the device, which would otherwise be impossible if they needed to reverse an unintended action.

[0272] Returning to the description of exemplary gesture 466, in some embodiments, the size of preview 498 is adjusted depending on which application the gesture is currently moving over. Figure 4S As shown, when gesture 466 is over application 1, preview 498 has a first display size; and, as shown Figure 4T As shown, when gesture 466 is over the video application, preview 498 is resized to have a second display size that is different from the first display size. FIG. 4S to FIG. 4T As shown, the first display size is wider (compared to the second display size) to match the dimensions of Application 1, and the second display size is narrower (compared to the first display size) to match the dimensions of the video application.

[0273] Upon detecting the end of gesture 466 (e.g., lifting the gesture from the touch-sensitive display while gesture 466 is over the video application), the web browsing application is then displayed in the area of ​​touch-sensitive display 112 previously occupied by the video application ( Figure 4U ).

[0274] In some embodiments, a gesture (e.g., Figure 4VThe gesture 468 in FIG. 4 can also be used to display multiple applications (e.g., Figure 4V When gesture 468 crosses the border of taskbar 408, a preview of the content corresponding to the affordance will be displayed ( Figure 4W ), and then displaying the content when the gesture 468 is detected to be lifted off the touch-sensitive display 112 ( Figure 4X ). In some embodiments, the content is displayed to occupy substantially the entire touch-sensitive display 112, and the previously displayed multiple applications are no longer displayed. In other embodiments, both the content and the multiple applications are displayed (e.g., the content is displayed above the multiple applications, allowing the user to interact with more than one set of fixed applications at the same time). In some embodiments, this feature is only available on devices with larger displays (e.g., displays with a diagonal dimension of at least 9.7 inches).

[0275] In some embodiments, multiple instances of a single application may also be pinned together, and affordances corresponding to the multiple instances of the single application (e.g., affordances 404) may also be pinned together using gesture 470 ( Figure 4Y to Figure 4AA ) drag out the taskbar 408.

[0276] although Figures 4A to 4AA Although not shown in the figure, some embodiments also allow the user to drag the affordance out of the taskbar and onto an overlay application that is not currently displayed (e.g., to drag the affordance toward the edge of the touch-sensitive display to temporarily display a hidden overlay application (an overlay application that the user has swiped away from the display)) and then drop the affordance onto the temporarily displayed overlay application. For example, the user can drop application-specific affordance 244 corresponding to the Notepad application from Figure 4AN The task bar 408 is shown being dragged toward the rightmost edge of the touch-sensitive display 112, and when a threshold distance from the rightmost edge is reached, the hidden overlay video application (in Figure 4AL to Figure 4AM In the example, the video application program (hidden using gesture 485 in the video player) will be temporarily displayed, and the user can then place the affordance of dragging on top of the hidden overlay application program, thereby replacing the video application program with the notepad application program. In this way, the user can use a simple drag gesture to replace an overlay application program that may not be currently visible.

[0277] Now go to Figure 4AB, exemplary thresholds 464 and 465 are shown near the rightmost edge and leftmost edge of the touch-sensitive display 112, respectively. In some embodiments, the thresholds are located approximately 1-2 centimeters from the respective edges of the display 112, and these thresholds are used to help determine when a predefined area of ​​the display 454 is displayed that provides an indication to the user that an application can subsequently be pinned in the predefined area. For example, Figures 4AB to 4AF As shown, gesture 472 is detected on application-specific affordance 232 corresponding to a video browsing application. Gesture 472 then moves beyond Figure 4A 408 in the taskbar, and in response to detecting that gesture 472 has moved beyond the boundaries of the taskbar, the application-specific enable representation 232 is no longer displayed beneath the contact corresponding to gesture 472, and instead a user interface object 473-A is displayed beneath the contact.

[0278] In some embodiments, user interface object 473-A has a first shape (e.g., a generally oval shape having a representation of an application-specific affordance positioned within it), and the contact corresponding to gesture 472 has not yet crossed thresholds 464, 465 ( Figure 4AC In some embodiments, in response to detecting that gesture 472 has crossed one of thresholds 464, 465 (and while the contact remains above the threshold), predefined area 454 is displayed and user interface object 473-B is displayed below the contact instead of user interface object 473-A ( Figure 4AD and Figure 4AE ). In some embodiments, user interface object 473-B has a second shape that is different from the first shape of user interface object 473-A (e.g., a substantially rectangular shape that includes a representation of an application-specific affordance located within). By displaying predefined area 454 and changing the shape of user interface object 473, a clear visual indication is provided to the user that they can now pin the application / activate split view mode on display 112. Providing these clear visual indications improves the human-computer interface by ensuring that new features are discoverable and enables continued interaction with display 112 that would otherwise be unavailable due to the user having to continually undo actions they could not anticipate.

[0279] like Figure 4AE to Figure 4AF As shown, the user may subsequently decide to end the gesture (e.g., by lifting the contact corresponding to gesture 472 from display 112), and in response, the application corresponding to the application-specific affordance that was dragged during gesture 472 (e.g., the video browsing application) is subsequently pinned next to application 1 on display 112 ( Figure 4AF ).

[0280] Figure 4AF Also shown is an example of a gesture (e.g., gesture 477) that can be used to unpin an application after it has been pinned. As shown, example gesture 477 involves contact that is then moved toward an edge (e.g., the leftmost edge of display 112 relative to a user looking down at display 112). In response to detecting that gesture 477 has moved to within a predetermined distance of the edge (e.g., within 1-2 centimeters of the edge), the user may then unpin the application. Figure 4AG The application is shown unpinned (and in some embodiments, the memory of device 100 is updated to reflect that the video application and application 1 are no longer pinned together). Additional details about using gestures to unpin applications are provided in commonly owned U.S. Application No. 14 / 732,618 (e.g., in FIGS. 37K through 37M, and in related paragraphs describing these figures), which is incorporated by reference above in its entirety. In other embodiments, detecting that gesture 477 has moved toward an edge at a speed or acceleration greater than a predetermined speed unpins an application.

[0281] Figure 4AG Also shown is gesture 478 that can be used to activate a slide mode in which an application is displayed overlaying another application. For example, a contact associated with gesture 478 is initially detected on an application-specific affordance 220 corresponding to a web browsing application, and in response to detecting that the contact moves beyond the boundaries of taskbar 408, UI object 473-A is displayed along with application-specific affordance 220. Figure 4AH In the example, lift-off of the contact from touch-sensitive display 112 is also detected on application 1 (and without the contact crossing either threshold 464 or 465), and in response, the web browsing application is then displayed as an overlay on application 1 (e.g., Figure 4AI shown).

[0282] Figure 4AI Also shown is an embodiment in which a gesture 482 is shown moving in a generally downward direction toward the bottom-most edge of the display 112, and in response to detecting that the gesture has moved a predetermined distance (e.g., 1-2 centimeters) in the downward direction, the web browsing application is then fixed next to the application 1 (e.g., Figure 4AJ In some embodiments, in addition to the following reference FIG. 5D to FIG. 5F In addition to the gesture 518 described above, this downward swipe gesture can also be used to fix the overlay application (while in other embodiments, the downward swipe gesture is FIG. 5D to FIG. 5FIn some embodiments, the gesture must be initiated at or near the top of the overlay application (e.g., above displayed affordance 462 near the top of the overlay application).

[0283] like Figure 4AJ to Figure 4AL As shown, in some embodiments, the user can also overlay an application on top of two fixed applications. For example, in response to detecting the end of gesture 484 on the boundary between the two fixed applications ( Figure 4AK ), and then displays the app as an overlay on the two pinned apps (for example, the video browsing app is displayed as an overlay on the web browsing app pinned next to app 1, Figure 4AL ). In addition, in some embodiments where picture-in-picture video is available, the user can also display the picture-in-picture video content on top of an overlay application that is on top of two fixed applications. In other words, the picture-in-picture video is always the topmost layer. The picture-in-picture video feature is described in more detail in commonly owned U.S. Application No. 14 / 732,618 (e.g., in Figures 48A to 52D, and in the relevant paragraphs describing these figures), which is incorporated by reference above in its entirety.

[0284] In order to use space efficiently, in some embodiments, the user can also temporarily hide the overlay application. Figure 4AL to Figure 4AN As shown, a gesture (e.g., gesture 485) may be used to swipe across the touch-sensitive display 112 in a direction substantially from left to right over the overlay application to temporarily hide the overlay application. The user may then use another gesture (e.g., gesture 486) that moves in the opposite direction to the gesture used to hide the overlay application to redisplay the overlay application (e.g., gesture 486 moves across the touch-sensitive display 112 in a direction substantially from right to left). Figure 4AO An example of an overlay application that has been redisplayed in response to gesture 486 is shown.

[0285] In order to allow the user to easily pin the currently overlaid application, thereby effectively and easily adjusting their workspace on display 112 (which results in an improved human-machine interface and allows the user to continuously interact with the application displayed on display 112), a gesture may be provided that allows the user to easily switch the application from an overlaid state to a pinned state. For example, a contact may be detected on affordance 506 displayed with the overlaid application, and the contact may then be moved on the touch-sensitive display in a substantially downward direction toward the bottom-most edge of touch-sensitive display 112 (e.g., Figure 4AOIn response to detecting that gesture 488 has moved at least a predetermined distance (e.g., 4-5 centimeters) on the touch-sensitive display, the overlay application is no longer displayed, and instead, UI object 473-B is subsequently displayed below the contact associated with gesture 488. In some embodiments, a gradual reduction in the content corresponding to the overlay application is shown until the content is no longer displayed and only UI object 473-A remains on display 112.

[0286] In some embodiments, the predetermined distance used to determine when to display UI object 473-B is greater than the predetermined distance used to determine when to switch from the covered state to the fixed state in response to a downward swipe gesture. In some embodiments, and as also described below with reference to method 1200, a threshold based on speed or acceleration may be utilized rather than a threshold based on distance. In these embodiments, the speed or acceleration threshold used to determine when to display UI object 473-B corresponds to a faster speed (e.g., 8 cm / sec) or acceleration than the speed or acceleration threshold used to determine when to switch from the covered state to the fixed state in response to a downward swipe gesture (e.g., 4 cm / sec).

[0287] In some embodiments, gesture 488 then continues with the user dragging UI object 473-B on touch-sensitive display 112. Figure 4AP to Figure 4AQ As shown, gesture 488 may be moved over application 1, and in response, the shape of UI object 473-B may subsequently change (such that UI object 473-B becomes UI object 473-C having a shape that is different from the shapes of UI object 473-B and UI object 473-A discussed previously) to provide the user with the following indication: if contact lift-off occurs, the video browsing application will replace application 1 on display 112.

[0288] like Figure 4AP and Figure 4AQ As shown, when gesture 488 is moved over the web browsing application and then over application 1, the content corresponding to each of these applications is blurred (eg, Figure 4AP and Figure 4AQ 473) to provide another visual indication to the user (in addition to changing the shape of UI object 473) that dropping the UI object over either application will cause a change.

[0289] In some embodiments, when the end of gesture 488 is detected (e.g., a lift-off of the contact associated with gesture 488 occurs on application 1), the video browsing application replaces application 1 on display 112 and is now pinned next to the web browsing application (the memory of device 100 may also be updated to reflect that the web browsing application and application 1 are no longer pinned together, and that the web browsing application and the video browsing application are now pinned together). Such an example is shown in FIG. Figure 4AR , where the video browsing application is now pinned next to the web browsing application on display 112.

[0290] Allowing a user to easily pin a currently overlaid application, thereby effectively and easily adjusting their workspace on display 112, ensuring an improved human-computer interface (e.g., by ensuring a consistent user experience between dragging an application out of a taskbar and dragging an overlaid application to switch it to a pinned state), and also allowing a user to continue to interact with the application displayed on display 112 (e.g., by ensuring that the content used to pin the application is not completely obscured during a drag gesture to switch between an overlaid state and a pinned state).

[0291] In some embodiments, to exit the video browsing application during gesture 488 described above, the user may end gesture 488 on taskbar 408. In these embodiments, in response to detecting the end of gesture 488 on taskbar 408 (e.g., lift-off of the contact associated with gesture 488 on the taskbar), the overlaid video browsing application is then dismissed, and the split view mode including Application 1 and the web browsing application remains displayed on display 112.

[0292] In some embodiments, each application installed on device 100 (in Figure 1AIn some embodiments, application-specific affordances corresponding to incompatible applications (such as those shown in the memory of the device in the example) may be associated with an indication as to whether they are compatible with the split view display mode. Based on determining that the particular application is not compatible with the split view display mode (e.g., by examining the indication associated with the particular application and determining that it indicates that the particular application is not compatible), no display change is made when the application-specific affordance corresponding to the particular application is dragged out of the taskbar 408. In contrast, dragging the application-specific affordance corresponding to the particular application and then dropping it over an application displayed in split view mode (or over a single application) will cause the particular application to then occupy the entire touch-sensitive display (rather than causing the particular application to be displayed in split view mode or to be displayed as overlaying any application). In some embodiments, application-specific affordances corresponding to incompatible applications (such as the particular applications used as examples here) may not be dragged out of the taskbar 408 at all (rather, these applications may simply be tapped to let the user know that these applications cannot be used in conjunction with the split view mode or the sliding display mode).

[0293] The following reference method 800 provides information about FIG. 4A to FIG. 4AR Additional description of .

[0294] FIG. 5A to FIG. 5O is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to show a user interface for switching between a split view mode (in which two applications are pinned together and displayed side by side on the display) and a slide mode (in which one application is displayed as covering another application on the display).

[0295] exist Figure 5A , two applications are displayed in split view mode on touch-sensitive display 112, and gesture 508 is detected near an edge of display 112 that is adjacent to a first application of the two applications displayed in split view mode. The gesture may initially exit from a bezel surrounding display 112 and pass through the edge of display 112, and then continue in a substantially downward direction (relative to a user looking down at display 112). In response to detecting that gesture 508 has traveled at least a threshold distance (e.g., 1-2 centimeters) in a substantially downward direction, affordance 506 is displayed to overlay the first application (e.g., affordance 506 overlays the first application). Figure 5B The video application is also interchangeably referred to herein as a video browsing application.

[0296] like Figure 5BAs shown, gesture 509 may be performed on affordance 506, which gesture allows the user to switch between split view mode and slide mode (and the switch allows the user to pin and unpin applications). Gesture 509 may include an initial contact on affordance 506, followed by an initial contact on a first application (e.g., Figure 5B The video browsing application shown in FIG. 10 is moved in a substantially downward direction. Figure 5C As shown, after gesture 509 has moved at least a threshold distance (e.g., 1-2 centimeters) in a substantially downward direction, split view mode is then exited and instead a slide over mode is entered in which a first application is displayed overlaying another application (e.g., a video browsing application is displayed overlaying the second application). Figure 5C In conjunction with the existing split view mode, the first application may also be unpinned from application 1 (eg, the memory of device 100 may be updated to remove the association between the first application and application 1).

[0297] When the applications are displayed in slide mode, application 1 may still be available and responsive to user input (even when the video browsing application covers application 1). For example, in response to detecting contact 510 on link 504 within application 1, contact 510 is processed and application 1 responds by accessing and displaying content associated with link 504 ( Figure 5D ).

[0298] Figure 5D Also shown is gesture 511 that can be used to switch back to split view mode and exit slide mode. Gesture 511 may include an initial contact on affordance 506 followed by movement of the contact in a substantially upward direction toward the topmost edge of touch-sensitive display 112. In response to detecting that gesture 511 has moved at least a threshold distance (e.g., 1-2 cm) in a substantially upward direction, the video application is again displayed in split view mode adjacent to application 1 on touch-sensitive display 112 (e.g., 1-2 cm). Fig. 5F ). The memory of device 100 may also be updated to again include an association between the video browsing application and application 1, indicating that the two applications are pinned together. In some embodiments, when gesture 511 is detected and moves in a substantially upward direction, a predefined area (e.g., Figure 5E The predefined area 591 is a predefined area where the video browsing application is subsequently displayed after the end of the gesture 511 is detected.

[0299] Fig. 5FAdditionally shown is gesture 512 that can be used to move the fixed video browsing application to a different location on touch-sensitive display 112. Gesture 512 may include an initial contact on affordance 506 followed by movement of the contact in a substantially lateral direction substantially parallel (e.g., within + / - 5 degrees of parallel) to the top-most edge of touch-sensitive display 112. Figure 5G 5 shows that as gesture 512 continues to travel in a substantially lateral direction, the video browsing application moves on the touch-sensitive display in accordance with gesture 512 until a position at the opposite side of application 1 is reached (e.g., as shown in FIG. Figure 5H As shown, when the lift-off of gesture 512 is detected, the video browsing application is then displayed on the left side of Application 1). The association between the two applications may also be updated to reflect that the video browsing application is now pinned on the left side of Application 1.

[0300] Figure 5H 1 shows that when the video browsing application is pinned to the left side of application 1, gesture 509 can be used to switch from the split view mode to the slide mode. For example, after detecting that gesture 509 has traveled at least a threshold distance in a substantially downward direction (e.g., 1-2 centimeters in a substantially downward direction), the split view mode is exited and the video browsing application is displayed as an overlay application 1 ( Fig.5I ). Fig.5I As shown, gesture 517 (similar to gesture 512 described above) can be used to drag the overlaid video browsing application to a different location on the touch-sensitive display 112 (e.g., Figure 5J shown).

[0301] Figure 5J Also shown is contact 527 on home button 204 (which may be a physical hardware button that moves when pressed, a solid-state button that does not move when pressed but simulates movement by providing tactile feedback, or a button displayed on touch-sensitive display 112 that can simulate movement by simultaneously providing tactile feedback). In response to detecting contact 527 on home button 204, the home screen ( Figure 5K ). While the home screen is displayed, contact 516 is detected on an affordance corresponding to a camera application ( Figure 5K ), and in response, the camera app opens with the video browsing app overlaid on top ( Figure 5L ). In this way, the user can have an overlay application displayed along with an application launched (from the home screen or application switcher UI) so that they have ongoing interaction with the overlay application while also changing the application displayed behind it.

[0302] Figure 5LIt is also shown that gesture 518 for dragging the affordance out of taskbar 408 can also be used to switch which application is overlaid (e.g., when gesture 518 is over an overlaid video browsing application, switching from the video browsing application to the web browsing application after ending the gesture). Figure 5M to Figure 5O )).

[0303] The following reference method 900 provides information about FIG. 5A to FIG. 5O Additional description of .

[0304] Now turn your attention to FIG. 6A to FIG. 6Q , which is a schematic diagram of a touch-sensitive display according to some embodiments, which is used to illustrate a continuous gesture that allows first activating a taskbar and then activating an application switcher user interface on the display.

[0305] like Fig. 6A 1, a first application (e.g., Application 1) is displayed adjacent to (or in other words, side-by-side / in split view mode) a second application (e.g., Application 2 or a video application). Application 1 and the video application are displayed in a display area 601a that occupies substantially all of the available display area of ​​touch screen 112 of portable multifunction device 100. While the side-by-side view is displayed, device 100 receives user input, such as a substantially upward drag, swipe, or slide gesture 602, on touch screen 112 of portable multifunction device 100. In response to receiving user input 602, the device gradually displays taskbar 408 from the bottom of touch screen 112, as shown in FIG. Figure 6B As shown. In some embodiments, taskbar 408 covering a portion of display area 601a includes enable representations corresponding to applications available on device 100, for example, a phone enable representation corresponding to a phone application, a mail enable representation corresponding to a mail application, a browser enable representation corresponding to a web browsing application, and a notepad enable representation corresponding to a notepad application available on device 100. In some embodiments, after gesture 602 moves a first threshold distance (e.g., 1-2 centimeters) on display 112, then as gesture 602 continues to move on display 112, taskbar 408 begins to follow the gesture. Then, in some embodiments, after gesture 602 has moved a second threshold distance (e.g., a total of 2-4 centimeters or an additional 1-2 centimeters beyond the first threshold distance), taskbar 408 remains in a static position and no longer follows gesture 602 ( Figure 6C ). When gesture 602 continues to move beyond a second threshold distance, the currently displayed application (e.g., a split view mode with application 1 and a video application) begins to shrink in size, and an application switcher user interface ( Fig.6DIn some embodiments, once the gesture moves at least a third threshold distance (e.g., 3-6 centimeters total or an additional 1-2 centimeters beyond the second threshold distance), an application switcher user interface is displayed with representations 604 displayed in their corresponding chronological order ( Fig. 6E ) (as discussed in more detail below). In some embodiments, movement between these threshold distances occurs without breaking contact with the screen surface, e.g., without any lift-off or release of contact.

[0306] Fig.6D An application switcher user interface is shown displayed on touch screen 112 in response to receiving user input 602. In some embodiments, the application switcher user interface displays representations of at least some of the applications that were recently used on portable multifunction device 100, such as application 3 (606), application 4 (608), application 5 (610), and application 6 (612). In some embodiments, the application switcher user interface also includes Fig. 6A A representation 604 of display area 601a is shown.

[0307] In some embodiments, such as Fig.6D As shown, previously Fig. 6A The display area 601a shown in is animated to gradually reduce in size or scale to fit the displayed representation 604. Representation 604 includes a miniaturized side-by-side view of application 1 and the video application. In some embodiments, the representations of recently used applications 606-612 are arranged in chronological order. For example, application 3 (606) was used more recently than application 4 (608) and is therefore displayed to the right of application 4 (608). Similarly, application 5 (610) was used more recently than application 6 (612) and is therefore displayed above application 6. In some embodiments, the application switcher user interface also includes a control center 614 that includes commonly used system controls, such as volume control, brightness control, etc.

[0308] Fig. 6E The reduction or zooming of representation 604 is shown to continue until representation 604 reaches a predetermined size, such as the same size as representations 606 - 612 . Fig. 6E It is further shown that the representations of the recently used applications 606-612 can be rearranged to make room for representation 604, for example, by moving representations 606-612 to the left and / or to another row. Since application 1 and application 2 (video application) are the most recently used applications, representation 604 representing application 1 and application 2 is placed before representations 606-612.

[0309] FIG. 6F to FIG. 6G Switching applications displayed in an application switcher interface according to some embodiments is shown. While displaying an application switcher interface including a taskbar 408 and representations 604 corresponding to application 1 and application 2, device 100 detects contact 603 above a web browsing enable representation in taskbar 408, followed by gesture 603 (e.g., continuous movement of a contact on touch screen 112 toward representation 604 displayed in the display area). In some embodiments, the gesture occurs without a release or lift-off detected between detection of the contact and movement. Gesture 603 ends when the user lifts off contact on application 2 in representation 604. When lift-off is detected, as shown in FIG. Figure 6G As shown, device 100 replaces application 2 in representation 604 with a representation of a web browsing application. Replacement representation 616 includes a representation of application 1 and a representation of a web browsing application in a side-by-side view. In some embodiments, the two applications are now pinned together until either is released from the pinned relationship, as described below.

[0310] FIG. 6H to FIG. 6M 604 , the device 100 detects user input, such as contact 618a above application 2 in representation 604, while displaying the application switcher user interface. Figure 6H shown.

[0311] In some embodiments, such as Fig.6I As shown, the system determines whether the input is of a predetermined type, such as a non-tap selection 618b, such as a press and hold contact that lasts for more than a predefined amount of time, or a contact with an intensity greater than a threshold intensity (e.g., a deep / hard press as described in more detail above). In response to detecting the predetermined type of input 618b, device 100 displays a disarming indication 620 above application 2 in indication 604. Subsequently, as shown Figure 6J As shown, upon detecting selection 622 of the disabling indication 620 (e.g., tapping the disabling indication 620), the device 100 closes the application 2 and updates the indication 604 to no longer include the application 2, as shown in FIG. Figure 6K In some embodiments, representations of more than two applications that are pinned together may also interact to dismiss and / or replace more than two applications.

[0312] In other embodiments, Figure 6LAs shown, the input is a drag gesture 618c. Drag gesture 626 includes contact on application 2 in representation 604, followed by movement of the contact away from representation 604. In some embodiments, drag gesture 626 includes an initial contact (e.g., a press and hold) that lasts for more than a predefined amount of time. In some embodiments, the initial contact is substantially stationary. In response to detecting that contact 626 moves to a predefined area (e.g., taskbar 408) or moves more than a predetermined distance from representation 604, device 100 closes application 2 and updates representation 604 to no longer include application 2, as shown. Figure 6M shown.

[0313] Figures 6N to 6Q A gesture is shown for activating an application switcher user interface received via a sliding display mode according to some embodiments, wherein a representation of the sliding display mode is not included in the application switcher user interface.

[0314] exist Figure 6N , application 1 is displayed in display area 601 b, and application 2 is displayed as an overlay in display area 601 b. In some embodiments, the overlaid display of application 2 includes a handle near the top edge of application 2, such as affordance 628. Display area 601 b occupies substantially all of the available display area of ​​touch screen 112. Fig.6O 1. Device 100 is shown receiving user input, such as a substantial drag, swipe, or slide gesture 602 away from an edge of the screen (e.g., dragging upward) on touch screen 112 of portable multifunction device 100 while displaying the overlay view. In response to receiving user input 602, the device gradually displays taskbar 408 from the bottom of touch screen 112, and taskbar 408 covers display area 601b (e.g., at the bottom of touch screen 112). Once in position, taskbar 408 will no longer move even if the gesture continues, and thus the taskbar remains in a stationary position. However, when continued gesture 602 continues to move further away from the edge (e.g., upward) after taskbar 408 stops moving, an application switcher interface is displayed, such as the one shown in FIG. Figure 6P shown.

[0315] As the gesture continues, Figure 6N Display area 601b is shown in 601b as gradually decreasing in size or scaling to fit representation 634. Representation 634 includes the representation of application 1, but does not include the overlaid representation of application 2. Figure 6Q The shrinking of representation 634 is shown to continue until representation 634 is the same size as the representations of recently used applications 636-642, which can be rearranged to make room for representation 634. In some embodiments, three different threshold distances are used (e.g., FIG. 6A to FIG. 6E The first, second, and third threshold distances discussed above are used to determine when to begin displaying the taskbar 408, when the taskbar should be placed in its resting position, and when the application switcher user interface should be displayed.

[0316] The following reference methods 1000 and 1200 provide information about FIG. 6A to FIG. 6Q Additional description of .

[0317] FIG. 7A to FIG. 7L is a schematic diagram of a touch-sensitive display for illustrating exemplary user interfaces and gestures for activating two or more instances of the same application. An instance is a specific single occurrence of an object that typically exists during the runtime of a computer program, e.g., each time a program or application runs, it is an instance of that program.

[0318] In some embodiments, a method for activating two or more instances of the same application is performed on a computer having a touch-sensitive display (e.g., touch screen 112) and one or more processors ( Figure 1A 122) of electronic devices (e.g., Figure 1A The portable multifunction device 100 is executed. For example, Fig. 7A A system having a touch-sensitive display 112 and one or more processors ( Figure 1A 122) of the electronic device 100 running an application 702a. As shown, the first instance of the application is displayed on substantially the entire touch-sensitive display 112.

[0319] In this example, the application 702a is an electronic mail, e-mail or mail application, but in other embodiments, the application can be any application capable of running more than one instance on the electronic device 100 and capable of displaying more than one instance of the application on the touch screen 112 (e.g., as described above with reference to FIG. Figure 1A and Figure 2 any application being discussed).

[0320] like Fig. 7A As shown, the inbox of the first instance of the mail application is displayed on the touch screen 112. In this example, the displayed inbox includes a left panel and a right panel. The left panel lists the various emails received by the mail application, and the right panel displays one of the received emails in more detail.

[0321] Figure 7B1 shows a user input (e.g., contact such as a tap or press) 704 on the touch screen 112. Here, the user input or contact 704 is located on or above an arrow indication that is used to reply, forward, or print an open email shown in the right panel. When contact is made with the touch screen 112 (also referred to herein as a touch-sensitive display or touch-sensitive screen or touch-sensitive display screen) at the arrow indication, the touch screen 112 and the processor ( Figure 1A 122) determines that the user has selected the arrow indication and displays an indication for replying to an email, forwarding an email, or printing an email. Figure 7B As shown, these affordances appear in a pop-up window that overlays an open email.

[0322] Figure 7C Another user input (e.g., contact or gesture) 706 is shown on touch screen 112. Here, user input (e.g., contact or gesture) 706 is on or over a reply affordance for replying to an open email. In some embodiments, user input (e.g., contact or gesture) 706 is not a simple tap or touch, but rather a deep or hard press (e.g., a press with a contact exceeding a predetermined contact force or intensity—see the description of the embodiment of the present invention). FIG. 3A to FIG. 3C ) or a press and hold for longer than a predetermined amount of time (e.g., 1 second).

[0323] including, for example, a touch screen 112, a processor ( Figure 1A 1 ), a system of contact intensity sensors ( 165 of FIG. 1 ), and / or intensity sensor controllers ( 159 of FIG. 1 ) determine whether user input (e.g., contact or gesture) 706 is of a predetermined type (e.g., a deep press or a press and hold) on the reply-enabled representation. When it is determined that user input 706 is of a predetermined type on the reply-enabled representation, touch screen 112 displays a preview 763 of a new email that is a reply to a previous email, such as Fig.7D In this example, a preview 763 of a new email replying to a previous email is displayed in a pop-up window covering the previous email.

[0324] Fig. 7E continue Fig.7D Here, after detecting that user input 706 is of a predetermined type (e.g., a deep press or a press and hold), without detecting any lift-off of the input or contact from the reply affordance, touch screen 112 and processor ( Figure 1A122) detects a gesture 706 of a predetermined type. In some embodiments, the gesture of the predetermined type is the contact moving a certain distance (or greater than) a certain distance (e.g., 1-2 cm). In other embodiments, the gesture of the predetermined type is the contact moving to a predefined second portion of the touch screen (e.g., the right third to half of the screen). This gesture or movement of contact 706 is performed by moving from Fig.7D The arrows extending from contact 706 are depicted.

[0325] When the gesture 706 is detected as a predetermined type, the first instance of the application is resized to the first portion of the touch screen display, such as Fig. 7E As shown. Fig. 7E In the example shown, the first portion is the left half of the touch screen. At the same time, the remainder of the touch screen (e.g., the entire touch screen area minus the first portion) is cleared of content other than preview 763 to provide an indication to the user of that area that a new email replying to a previous email (as depicted by preview 763) will eventually occupy that area if the user releases or lifts off contact with the screen. In some embodiments, the remainder of the touch screen is the second portion 762 mentioned above.

[0326] If the user does not want to open the new email in reply to the previous email in split view mode (e.g., next to the first instance of the mail application), the user can move the contact back over the reply affordance and lift or release the contact from the surface of the touch screen. However, if the user lifts or releases contact with the touch screen after a predetermined type of gesture or contact (see above), the user can move the contact back over the reply affordance and lift or release the contact from the surface of the touch screen. Fig.7D ), then a new email replying to the previous email is displayed in the second part 762 of the touch screen, such as Figure 7F or Figure 7G In some embodiments, when lift-off or release of contact is detected, preview 763 ( Fig. 7E ) is expanded to occupy the entire second portion 762 ( Fig. 7E ). In some embodiments, the expansion is animated.

[0327] In some embodiments, once preview 763 ( Fig. 7E ) is expanded to occupy the entire second portion 762, then the first instance of the mail application is displayed in the first portion, such as Figure 7F In other embodiments, once preview 763 ( Fig. 7E ) is expanded to occupy the entire second portion 762, then the previous email (to which the new email shown in the second portion is a reply) is displayed in the first portion, such as Figure 7G As shown. Figure 7G In some related implementations, when preview 763 ( Fig. 7E) is expanded to occupy the entire second portion 762, the first instance of the mail application slides off the screen (e.g., to the left) and the previous email slides into the first portion in an animated sequence (e.g., from the right).

[0328] Figure 7H Another example of a user interface and gesture for activating two or more instances of the same application (here, a mail application) is shown. Fig. 7A Similarly, the first instance of the application is displayed, which occupies substantially all displayable areas of the touch screen. As shown, user input or contact 712 is detected at an optional affordance (here, a single email in the list of received emails shown in the inbox).

[0329] including, for example, a touch screen 112, a processor ( Figure 1A 1 ), a system of contact force sensors ( 165 of FIG. 1 ), and / or force sensor controllers ( 159 of FIG. 1 ) determine whether user input (e.g., contact or gesture) 712 is of a predetermined type (e.g., a deep press or a press and hold; see the above description introduced herein) on the reply affordance. Upon determining that user input 712 is of a predetermined type on the affordance, touch screen 112 displays a preview 765 of a new instance of the application (here, an email replying to the selected email in the inbox), such as Fig.7I In this example, preview 765 is displayed in a pop-up window covering a first instance of the application (eg, a mail application).

[0330] like Fig.7I As shown, after detecting that the user input 712 is of a predetermined type, without detecting any lift-off of the input or contact 712, the touch screen 112 and the processor ( Figure 1A 122) system detects a predetermined type of gesture 712. In some embodiments, the predetermined type of gesture is the contact moving (or greater than) a certain distance (e.g., 1-2 cm). In other embodiments, the predetermined type of gesture is moving to a predefined second portion of the touch screen (e.g., the right 1 / 3 of the screen). This gesture or movement of contact 706 is determined by Fig.7I The arrows in the figure depict the

[0331] When the gesture 712 is detected as a predetermined type, a first instance of an application (e.g., a mail application displaying an inbox) is resized to a first portion of the touch screen display, such as Figure 7J As shown. Figure 7JIn the example shown, the first portion is the left half of the touch screen. At the same time, the remainder of the touch screen (e.g., the entire touch screen area minus the first portion) is cleared of content other than the preview 765 to provide a visual indication to the user of that area that the new reply email (as depicted by the preview 765) will eventually occupy that area if the user lifts off contact with the screen. In some embodiments, the remainder of the touch screen is the second portion 764 mentioned above.

[0332] If the user does not want to open the new reply email next to the first instance of the mail application in split view mode, the user can move the contact back to the location from which the contact began and lift or release the contact from the surface of the touch screen. However, if the user lifts or releases contact with the touch screen after a predetermined type of gesture or contact (see above Fig.7I ), then a new email replying to the previous email is displayed in the second part 762 of the touch screen, such as Figure 7K In some embodiments, when lift-off or release of contact is detected, preview 765 ( Figure 7J ) is expanded to occupy the entire second portion 764 ( Figure 7J ). In some embodiments, the expansion is animated.

[0333] In some embodiments, once preview 765 ( Figure 7J ) is expanded to occupy the entire second portion 764, then the first instance of the mail application is displayed in the first portion, such as Figure 7K In other embodiments, once preview 765 ( Figure 7J ) is expanded to occupy the entire second portion 764, then the previous email (to which the new email shown in the first portion is a reply) is displayed in the first portion, such as Figure 7L As shown. Figure 7L In some related embodiments, when preview 765 ( Figure 7J ) is expanded to occupy the entire second portion 764, the first instance of the mail application slides off the screen (e.g., to the left) and the previous email slides into the first portion in an animated sequence (e.g., from the right).

[0334] As above combined FIG. 7A to FIG. 7G and Figures 7H to 7K As described, a second instance of an application (e.g., an email reply of a mail application) can be opened side by side in split-screen mode on a touch screen adjacent to a first instance of the same application (e.g., displaying the inbox of the mail application). Other embodiments are used to display more than two instances of an application side by side in split view mode, while other embodiments display multiple applications side by side in split view mode along with multiple instances of the same application.

[0335] In some embodiments, a combination of all displayed applications, instances of applications, or a combination of applications and multiple instances of applications are displayed so that they occupy substantially the entire displayable area of ​​the touch screen.

[0336] In some embodiments, such as Figure 7F , Figure 7G , Figure 7K and Figure 7L As shown, a border affordance 710 is displayed between two displayed instances of an application, e.g., between a first portion and a second portion. The border affordance 710 can be used to drag the border between the two instances, e.g., to the left or to the right. Dragging the border affordance 710 adjusts the size of the respective instances of the application, and when a lift occurs, the lift maintains the respective instances at their respective sizes. Further details can be found in commonly owned U.S. Application No. 14 / 732,618 (e.g., in FIGS. 37H to 37M , and in the relevant paragraphs describing these figures), which is incorporated by reference above in its entirety.

[0337] In some embodiments, the first instance and the second instance of the application are executed on one or more processors ( Figure 1A 122) are different executable processes executed on the electronic device (for example, two different instantiations of the application are executed separately on one or more processors of the electronic device).

[0338] In some instances, a user of a computing system loses access to certain features of an application after selecting to turn on other features within the application (e.g., a user may not be able to view a received email after selecting an option to reply to the received email). Opening multiple instances of a single application allows the user to continue to view and use these particular features. Enabling the user to continue to view and use these particular features enhances the operability of the device and makes the human-computer interface more efficient (e.g., by enabling the user to use these particular features without having to provide multiple inputs to turn them back on after turning them off). Additionally, this enables continued interaction with a touch-sensitive display that would otherwise not be possible due to having to employ multiple gestures or inputs to figure out how to turn these features back on or access them after they have been turned off or are no longer displayed.

[0339] The following reference method 1100 provides information about FIG. 7A to FIG. 7L Additional description of .

[0340] FIG. 8A to FIG. 8C is a flowchart representation of a method for activating a split view mode (in which multiple applications are displayed and available for use simultaneously) by dragging an enable representation out of a taskbar according to some embodiments. Figures 4A to 4AA Used to show FIG. 8A to FIG. 8C Although some of the examples that follow will be given with reference to input on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface 195 that is separate from the display 194, such as Figure 1D as shown in .

[0341] In some embodiments, method 800 is performed by an electronic device (e.g., Figure 1A The method 800 is performed by one or more components of the electronic device (e.g., I / O subsystem 106, operating system 126, etc.) and / or the portable multifunction device 100 of the device. In some embodiments, the method 800 is performed by a computer program stored in a non-transitory computer-readable storage medium and executed by one or more processors of the device, such as one or more processors 122 ( Figure 1A ) is managed by instructions executed by the device 100. For ease of explanation, the following describes a method 800 executed by the device 100. In some embodiments, referring to Figure 1A The operations of method 800 are performed at least in part by or using a multitasking module (e.g., multitasking module 180) and its components, a contact / motion module (e.g., contact / motion module 130), a graphics module (e.g., graphics module 132), and a touch-sensitive display (e.g., touch-sensitive display system 112). Some operations in method 2200 are optionally combined, and / or the order of some operations is optionally changed.

[0342] As described below, method 800 provides an intuitive way to activate split view mode using a gesture at the taskbar, and then interact with an application displayed on a touch-sensitive display in split view mode. The method reduces the number of inputs required by the user to activate the split view mode (e.g., the user only needs to provide a simple drag gesture instead of performing multiple inputs), thereby ensuring that the battery life of the electronic device implementing method 800 is extended because less power is required to process a smaller number of inputs (and this savings will be repeatedly achieved as the user becomes more familiar with more intuitive and simple gestures). As also explained in detail below, the operation of method 800 helps ensure that the user is able to interact continuously (e.g., they do not need to frequently perform undo actions, which interrupts their interaction with the device), and the operation of method 800 helps to produce a more effective human-computer interface.

[0343] In some embodiments, a device configured to perform method 800 first displays (802) a first set of one or more applications in a display area that occupies substantially the entire touch-sensitive display. For example, the first set of one or more applications may include a single application (e.g., Figure 4A) or as shown in the Maps application Fig. 4I As shown, the one or more applications of the first set may include more than one application (e.g., application 1 on the left portion of display 112 and a video browsing application on the right portion of display 112 so that the two applications are displayed together in a split view mode). Fig. 4I , and the combination of the two applications takes up essentially the entire touch-sensitive display).

[0344] While displaying the first set of one or more applications (804), the device displays (806) a task bar (e.g., Figure 4A Displaying the taskbar may include (808) detecting an initial contact on the touch-sensitive display and, in response to detecting movement of the initial contact in a direction substantially perpendicular to a bottom edge of the touch-sensitive display (e.g., within + / - 5 degrees of vertical), displaying the taskbar in a first portion of the touch-sensitive display. In other words, the taskbar is activated by dragging in a substantially upward direction, such as FIG. 4A to FIG. 4C 4 is shown as gesture 450 in and is described in more detail below with reference to method 1000.

[0345] In some embodiments, a task bar (e.g., Figure 4A The taskbar 408 of FIG. 408 is a user interface object that includes multiple affordances, each affordance corresponding to a respective set of one or more applications. The respective set of one or more applications may include a single application (e.g., Figure 4C The affordance 220 may include multiple applications (e.g., multiple applications displayed / pinned together in a split view mode, such as Figure 4S 4, which corresponds to a map application pinned next to a video browsing application). In some embodiments, the taskbar includes two different sets of affordances, a first set of default affordances (e.g., affordances corresponding to the phone and mail applications on the left side of a divider displayed in the taskbar) and a second set of recently used affordances (e.g., affordances corresponding to the web browsing and video browsing applications on the right side of the divider). In some embodiments, the taskbar includes a predetermined number (e.g., 2, 3, 4, or 5) of recently used applications in the second set of recently used affordances.

[0346] The device then detects (810) on the touch-sensitive display a contact on a first affordance displayed within the taskbar (different from the initial contact used to activate the taskbar), and the first affordance is associated with one or more applications of the second set. For example, an exemplary contact associated with gesture 452 is received on affordance 232 corresponding to a video browsing application, such as Figure 4E As shown. Figure 4V As shown, another exemplary contact associated with gesture 468 is received on enable representation 468, and enable representation 468 is associated with a split view mode in which a map application is displayed adjacent to a messaging application and a web browsing application is displayed in a side-by-side view that occupies substantially the entire touch-sensitive display.

[0347] Then detect the movement of the contact away from the taskbar (812).In some embodiments, the movement is a continuous movement detected without detecting any lift-off of the contact from the touch-sensitive display.

[0348] In some embodiments, before any lift-off of the contact is detected, a representation of the affordance is displayed (814) near the contact until the contact moves a threshold distance (e.g., 1-2 cm) away from the taskbar. For example, the representation of the affordance is displayed within very close proximity (e.g., 1px or 2px) of the contact or directly below the contact so that the user is able to view the representation of the affordance moving on the touch-sensitive display while moving the contact (e.g., FIG. 4E to FIG. 4F As shown, affordance 232 is displayed as following the contact associated with gesture 452 as it moves away from taskbar 408).

[0349] Based on determining that the contact has moved at least a threshold distance away from the taskbar, the device stops displaying the representation of the affordance and displays a thumbnail of the content associated with the second application near the contact (e.g., once the contact has moved beyond the threshold distance from the border around the taskbar, the preview / thumbnail 456 is displayed below the contact). The thumbnail of the content corresponds to a miniaturized preview of the content associated with the second application, the miniaturized preview including the content that will be displayed within the second application when the second application is opened, and the content is displayed at about 1 / 5 to 1 / 10 of its size when the second application is opened. In this way, additional visual feedback is provided to the user, which helps them recognize and understand the changes that will occur after releasing (or lifting off) the contact. Providing this additional visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to better understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continuous interaction with the touch-sensitive auxiliary display, which otherwise cannot be achieved because the user has to repeatedly undo or reverse unwanted changes that they cannot anticipate.

[0350] In some embodiments, the device also adjusts the size of the thumbnail to match the display size of the portion of the touch-sensitive display where the contact was made. Figure 4S ) may have a first display size, while the contact associated with gesture 466 is on a first portion of display 112 displaying application 1. Also, preview 498 may have a second display size different from the first display size, while the contact associated with gesture 466 is on a second portion of display 112 displaying a video browsing application. In this way, further visual feedback is provided to the user that helps them recognize and understand the changes that will occur after releasing (or lifting off) the contact. Providing this further visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to better understand the changes that will occur and avoiding having to reverse unwanted changes). Additionally, this enables continued interaction with a touch-sensitive secondary display that would otherwise not be possible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0351] In some embodiments, rather than providing thumbnails of content, a user interface object (e.g., UI objects 473-A, 473-B, and / or 473-C) is displayed that includes a representation of the application-specific affordance being dragged. Features described above with respect to the use of these thumbnails also apply to embodiments in which user interface objects are provided rather than thumbnails. Examples of the use of user interface objects are shown in FIG. Figures 4AB to 4AE , Figures 4AG to 4AH , Figures 4AJ to 4AK , Figure 4AO to Figure 4AQ , and descriptions of these examples are also provided above.

[0352] Thus, in some embodiments, method operation 814 may be replaced with (or a new method operation may be added as an alternative to method operation 814) the following operation, which includes: displaying a representation of the first enable representation near the contact until the contact moves away from the taskbar a threshold distance; and based on determining that the contact has moved away from the taskbar by at least the threshold distance, displaying a user interface object surrounding the representation of the first enable representation near the contact. Figures 4AB to 4AC In the example, video browsing affordance 232 is surrounded by user interface object 473-A (and the shape may change as the contact moves over the first portion and the second portion, so that the UI object then has the same shape as in the first portion and the second portion, respectively). Figure 4AP and Figure 4AQ UI objects 473-B and 473-C are shown in FIG. 4 and are substantially similar in shape).

[0353] Liftoff of the contact from the touch-sensitive display is detected while the contact is on a second portion of the display area that is different from the first portion (e.g., FIG. 4H to FIG. 4I The contact lift-off or contact associated with gesture 452 between Figure 4T to Figure 4U 466), the device then (simultaneously) displays (818) (i) at least some of the one or more applications in the first set and (ii) at least some of the one or more applications in the second set occupying substantially the entire display area of ​​the touch-sensitive display. In some embodiments, the first portion of the display area is used to display the one or more applications in the first set (e.g., Figure 4E 1), and the second portion of the display area is a portion of the display area that can be used to display one or more affordances of the second set (e.g., Figure 4F A predefined area 454 may be used to display one or more applications of a second set when lift-off of contact is detected).

[0354] In some cases, users of a computing system must employ inefficient and time-consuming interactions to operate two or more applications. Allowing a user to use a single simple gesture (e.g., dragging an affordance from a taskbar to a primary display area using a single finger as described above with respect to at least operations 804-812 and 818 of method 800) enables a user to easily and efficiently activate a mode in which two or more applications are displayed and operated simultaneously. Enabling a user to easily and efficiently activate the mode enhances the operability of the device and makes the human-computer interface more efficient (e.g., by enabling a user to activate the mode using a single gesture). In addition, this enables continuous interaction with a touch-sensitive display that would otherwise not be possible due to having to employ multiple gestures to access two or more applications.

[0355] Now go to Figure 8B In some embodiments, before any lift-off of the contact from the touch-sensitive display is detected and after the contact moves at least a threshold distance away from the taskbar, the device resizes (820) the size of the one or more applications of the first set to display a second portion of the display area adjacent to the resized one or more applications of the first set. In some embodiments, the one or more applications of the first set are resized to occupy a predetermined percentage of the display area (e.g., approximately 75% or 66% of the display area), and then the second portion (e.g., the predetermined area 454 or a portion of the display area for displaying fixed applications, such as for displaying Figure 4T The portion of the video browsing application in the display) occupies a predetermined remaining percentage of the display area (e.g., approximately 25% or 34% of the display area).

[0356] As a specific example, FIG. 4E to FIG. 4H As shown, when one or more applications of the first set correspond to a single application (e.g., Figure 4E 1), the size of the single application is reduced to make room for displaying one or more enabling representations of a second set adjacent to the resized single application (for example, presenting a preview of a predefined area, which in some embodiments includes temporarily displaying a wallpaper associated with the home screen).

[0357] When one or more applications of the first set correspond to multiple applications (e.g., Figure 4N When the first fixed application 1 and the second fixed video browsing application are displayed, one of the multiple applications may no longer be displayed ( Figure 4T) to make room for displaying a preview of the predefined area 454 (which may include displaying a wallpaper associated with the home screen of the device). In some other embodiments, when one or more applications of the first set correspond to multiple applications, two of the multiple applications may remain displayed (e.g., Figure 4W ), and one of the multiple applications is only available when the gesture (e.g. Figure 4W The gesture is removed from the display only after the contact associated with gesture 468 is lifted.

[0358] In some cases, a user of the electronic device may be unaware of changes that will result from certain interactions on the touch-sensitive display. Providing clear visual feedback (e.g., resizing one or more applications of the first set to display a second portion of the display area) gives the user an opportunity to preview the changes that will occur after they release (or lift off) contact. Providing such clear visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continued interaction with the touch-sensitive display that would otherwise not be possible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0359] In some embodiments, a threshold distance (e.g., 1-2 cm, such as 1-2 cm) is detected before lift-off of contact is detected and before the contact moves to an edge of the touch-sensitive display adjacent to at least some of the one or more applications in the first set or at least some of the one or more applications in the second set. Figure 4AB After the thresholds 464, 465 are within the range, the method includes (820A) adjusting the size of the one or more applications of the first set to display a second portion of the display area adjacent to the resized one or more applications of the first set (e.g., Figure 4AD and Figure 4AE shown).

[0360] In a particular exemplary embodiment, when the one or more applications of the first set correspond to a single application (e.g., Figure 4AB 1), a single application is only reduced in size so as to determine a gesture to drag the application-specific affordance out of taskbar 408 (e.g., Figure 4AB The gesture 472) has crossed a threshold such as Figure 4AB When the threshold is 464 or 465, space is made for displaying a preview of the predefined area. Figures 4AB to 4AF and described in more detail above.

[0361] In some embodiments (such as the one particular exemplary embodiment described above), when the one or more applications of the first set correspond to multiple applications, the multiple applications may not be resized to display a preview of predefined area 454. Instead, the user can drag the application-specific affordance over a boundary between the multiple applications to overlay another application on top of the multiple applications (e.g., Figure 4AJ to Figure 4AL ), or dragging an application-specific affordance onto a specific application of the plurality of applications to replace the specific application with a new application corresponding to the application-specific affordance (e.g., as shown in Figure 4AP to Figure 4AR See above. Figure 4AJ to Figure 4AL and Figure 4AP to Figure 4AR Examples of such implementations are provided in further detail.

[0362] Also like Figure 8B As shown, in some embodiments, displaying (i) at least some of the one or more applications of the first set and (ii) at least some of the one or more applications of the second set in a display area that occupies substantially the entire touch-sensitive display includes (822 and 822A): displaying at least some of the one or more applications of the second set in a second portion of the display area (for example, displaying a web browsing application in a predefined portion, such as Figure 4U ), and displaying at least some of the one or more applications of the first set in the remainder of the display area adjacent to the second portion of the display area (e.g., displaying Application 1 in the remainder of the display area adjacent to the second portion / predefined area). In some embodiments, the second portion of the display area is separated from the remainder of the display area by a draggable border that extends along the length of the second portion and the remainder. In some embodiments, a contact on the draggable border (e.g., on border indication 460, Figure 4R ) allows the user to resize the portion of the display area allocated to one or more applications of the first set and the second set (also as described above with reference to Figures 4L to 4M discussed).

[0363] See now Figure 8C In various embodiments or situations, the first set and the second set may include one or more applications, and the lifting of the contact may result in different replacement behaviors. Different examples are provided below with reference to 824-830.

[0364] For example, at 824, the first set includes a first application, the second set includes a second application, the affordance is associated with the second application, and upon detecting lift-off of the contact, the first application and the second application are displayed side by side in the display area. FIG. 4E to FIG. 4IAn example of this is shown in FIG. 10. In some embodiments, when the contact is within a threshold distance of an edge of the touch-sensitive display adjacent to the first application (e.g., when the contact is within a threshold distance of an edge of the touch-sensitive display adjacent to the first application Figures 4AB to 4AE Lift-off of the contact is detected beyond one of the thresholds 464, 465 shown, and a description of which is provided above. In a case where the first set includes a first application and the second set includes a second application, a user of the computing system is able to activate a split view mode in which the first application and the second application are displayed together by simply dragging an affordance corresponding to the second application on a second portion of the display. Providing clear visual feedback (e.g., after the contact moves a threshold distance, such as displaying a second portion of the display area) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. Providing such clear visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to understand that releasing the contact will activate a split view mode that includes the first application and the second application). Additionally, this enables continued interaction with the touch-sensitive display that would otherwise be impossible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0365] In some embodiments, if lift-off of the contact is detected while the contact is over the one first application (rather than when the contact is within a threshold distance of an edge of display 112, such as one of thresholds 464, 465), the second application is displayed in overlay mode rather than side by side with the first application. For example, when lift-off of the contact is detected while the contact is over the one first application (825) (e.g., as Figure 4AH As shown, the lift-off occurs on the first application), and the second application is displayed as covering the first application in the display area (as shown in FIG. Figure 4AI shown).

[0366] As another example at 826, the first set includes at least a first application and a second application, the second set includes a third application, and the affordance is associated with the third application (e.g., Figure 4Q to Figure 4UIn a case where the first set includes a first application and a second application and the second set includes one second application, a user of the computing system is then able to add the new application to a split view mode in which the first application and the second application are displayed together by simply dragging an affordance corresponding to a third application onto the display. Providing clear visual feedback (e.g., showing a second portion of the display area after the contact moves a threshold distance) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. Additionally, this enables continued interaction with the touch-sensitive display that would otherwise be impossible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0367] In the example at 826, based on determining that the lift-off of the contact occurred on the second application, the device replaces the second application with a third application on the touch-sensitive display. Figure 4Q to Figure 4U An example of this is shown in In this way, the user is provided with the ability to replace a second application with a third application in split view mode, thereby enabling continuous interaction, wherein the user can quickly and easily swap in a new application that will be viewed simultaneously with the first application.

[0368] In some embodiments, if the lift-off of the contact occurs at the boundary between the first application and the second application, then instead of replacing one of the fixed applications, the third application instead covers one of the first application or the second application. Figures 4AJ to 4AK As shown, the lift-off of the contact associated with gesture 484 occurs at the boundary between application 1 and the video browsing application, and based on determining that the lift-off of the contact occurs at the boundary, the device displays (827) on the touch-sensitive display a third application (e.g., Figure 4AL video browsing application) (for example, the video browsing application covers Figure 4AL In some embodiments, after determining that a lift-off has occurred on the boundary between the first application and the second application, the third application always covers the rightmost application.

[0369] Another example is described at 828, where the first set includes at least two fixed applications of the first group, the second set includes at least two fixed applications of the second group, and the affordance is associated with the at least two fixed applications of the second group (e.g., Figure 4V to Figure 4X). Where the first set and the second set each include a fixed application group, a user of the computing system can subsequently add the new fixed application group to the split view mode by simply dragging the affordance corresponding to the second group onto the display. Providing clear visual feedback (e.g., showing a second portion of the display area after the contact moves a threshold distance) gives the user an opportunity to preview the changes that will occur after they release (or lift off) the contact. Additionally, this enables continued interaction with the touch-sensitive display that would otherwise be impossible due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0370] In the example at 828, based on determining that the lift-off of the contact occurred on the first group of applications, the device replaces at least two fixed applications of the first group with at least two fixed applications of the second group (e.g., Figure 4X In this way, the user is provided with the ability to quickly switch from a first set of pinned applications to a second set of pinned applications, thereby enabling continuous interaction, wherein the user can quickly and easily change to a new set of pinned applications on the display.

[0371] A final example is at 830, where one or more applications of a first set include an instance of a first application, one or more applications of a second set include an additional instance of the first application, and the instance and the additional instance are different instances of the first application. Where the first set and the second set each include different instances of the same first application, a user of the computing system is subsequently able to pin two different instances of a single application adjacent to each other. Allowing two different instances to be pinned together provides an improved human-computer interface for the user (e.g., by ensuring that the user is able to interact with different features of a single application simultaneously, such as editing a message in a messaging application while also copying content from a different message in the same messaging application). Additionally, this enables continued interaction with a touch-sensitive display that would otherwise be unavailable due to the user having to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0372] In some embodiments, and as explained in more detail below with reference to method 900, the second application is displayed with a handle (e.g., Figure 4J The handle may be used to activate the sliding mode or drag the second application to be displayed in a different part of the touch-sensitive display. Therefore, aspects / operations of methods 800 and 900 may be interchanged, replaced, and / or added between these methods. For the sake of brevity, these details are not repeated here.

[0373] Additionally, in some embodiments, and as explained in more detail below with reference to method 1000, the representation shown in the application switcher user interface includes a representation of the split view mode. Therefore, aspects / operations of methods 800 and 1000 may also be interchanged, replaced, and / or added between these methods. For the sake of brevity, these details are not repeated here.

[0374] Furthermore, in some embodiments, and as explained in more detail below with reference to method 1100, the one or more applications of the first set may include two different instances of the same application. Thus, aspects / operations of methods 800 and 1100 may be interchanged, replaced, and / or added between these methods. For the sake of brevity, these details are not repeated here.

[0375] FIG. 9A to FIG. 9B is a flowchart representation of a method for interacting with a single enable representation to switch between a split view mode (in which two applications are pinned together and displayed side by side on a display) and a slide mode (in which one application is displayed as overlaying another application on the display) according to some embodiments. FIG. 5A to FIG. 5O Used to show FIG. 9A to FIG. 9B Although some of the examples that follow will be given with reference to input on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface 195 that is separate from the display 194, such as Figure 1D as shown in .

[0376] In some embodiments, method 900 is performed by an electronic device (e.g., Figure 1A The method 800 is performed by one or more components of the electronic device (e.g., I / O subsystem 106, operating system 126, etc.) and / or the portable multifunction device 100 of the device. In some embodiments, the method 800 is performed by a computer program stored in a non-transitory computer-readable storage medium and executed by one or more processors of the device, such as one or more processors 122 ( Figure 1A ) is managed by instructions executed by the device 100. For ease of explanation, the following describes a method 900 executed by the device 100. In some embodiments, referring to Figure 1A The operations of method 800 are performed at least in part by or using a multitasking module (e.g., multitasking module 180) and its components, a contact / motion module (e.g., contact / motion module 130), a graphics module (e.g., graphics module 132), and a touch-sensitive display (e.g., touch-sensitive display system 112). Some operations in method 900 are optionally combined, and / or the order of some operations is optionally changed.

[0377] As described below, method 900 provides an intuitive way to switch between split view mode and slide mode, and also provides an intuitive way to manipulate the location where multiple applications are displayed simultaneously. The method reduces the number of inputs required by the user to switch between these modes (e.g., the user only needs to provide a simple gesture instead of performing multiple inputs), thereby ensuring that the battery life of the electronic device implementing method 900 is extended because less power is required to process a smaller number of inputs (and this savings will be repeatedly achieved as the user becomes more familiar with more intuitive and simple gestures). As also explained in detail below, the operation of method 900 helps ensure that users are able to interact continuously (e.g., they do not need to frequently perform undo actions, which interrupts their interaction with the device), and the operation of method 900 helps to produce a more effective human-computer interface.

[0378] In some embodiments, the device executing method 900 first displays (902) a first application and a second application in a display area that occupies substantially the entire touch-sensitive display. In some embodiments, the first application is a primary application that is displayed as occupying approximately 2 / 3 (approximately 66%) of the available display area of ​​display 112 (e.g., application 1 is displayed as occupying the right side of display 112, Figure 5A ), and the second application is a side application that is pinned next to the primary application and is displayed as occupying the remaining 1 / 3 (approximately 33%) of the available display area of ​​display 112 (e.g., a video browsing application that is displayed as occupying the left side of display 112 and pinned next to application 1 in split view mode).

[0379] In some embodiments, displaying the first application and the second application in the display area includes (903) simultaneously displaying the first application and the second application side by side in the display area.

[0380] When displaying the first application and the second application in the display area (904), the device can perform operations 906, 908, and 910. At operation 906, the device displays an affordance near an edge of the second application. For example, the affordance is a user interface object (e.g., Figure 5B At 908, in some embodiments, before displaying the affordance, the device detects user input at an edge of the second application (e.g., Figure 5A ) and displaying an affordance ( Figure 5B). In this way, the user is provided with a simple gesture for activating the display of the handles, thereby enhancing the operability of the device (e.g., by ensuring that the handles are only displayed when the user needs them).

[0381] In some embodiments, also at operation 908, the user input may include an initial contact at an edge of the second application (e.g., associated with gesture 508) and movement of the initial contact along a first direction on the touch-sensitive display. In some embodiments, before displaying the enabling indication, the device detects that the contact has moved at least a threshold distance away from the edge, such as 1-2 centimeters. As described above, this provides the user with a simple gesture for activating the display of the handle, thereby enhancing the operability of the device (e.g., by ensuring that the handle is displayed only when the user needs it).

[0382] At 910 (which is also performed when the first application and the second application are displayed in the display area), the device detects (910) a contact on the affordance that moves in a direction on the touch-sensitive display (different from the initial contact discussed above, such as the contact associated with gesture 509, Figure 5B ). Based on determining that the direction is the first direction (912) (e.g., the contact associated with gesture 509 is shown as moving in a substantially downward direction relative to the topmost edge of the second application, Figure 5B ), the device performs operations 914 and 916. Operation 914 includes adjusting the size of the first application to occupy substantially all of the display area (e.g., Figure 5C As shown, application 1 now occupies substantially all of the display area of ​​touch-sensitive display 112); and operation 916 includes displaying a second application overlaying the resized first application (eg, also as shown in FIG. 1 ). Figure 5C As shown, the video browsing application is displayed overlaying the now resized application 1).

[0383] Method 900 also checks whether the direction is a second direction, rather than the first direction discussed above, and if so, the method proceeds to move the application on which affordance 506 is displayed (rather than switching between split view mode and slide mode). Specifically, based on determining that the direction is a second direction that is substantially perpendicular to the first direction (e.g., within + / - 5 degrees perpendicular to the first direction), the device moves (918) the second application within the display area. For example, the contact associated with gesture 512 is at Figure 5F to Figure 5G , the second application is shown as moving in a second direction (e.g., substantially from left to right on the touch-sensitive display), and then the second application moves laterally within the display area (e.g., from Fig. 5F Move to the rightmost position of Figure 5H). In some cases, users of the computing system must employ inefficient and time-consuming interactions to operate two or more applications. Allowing the user to interact with an affordance (such as a handle, at least in conjunction with operations 902, 904, 906, 910, 912, 914, 916, and 918) to activate a sliding mode or move a particular application within the display area enables the user to easily and efficiently move applications and switch between a side-by-side view or a sliding view. Enabling the user to easily and efficiently move applications and switch between different modes of using two or more applications simultaneously enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to select how to interact with two or more applications simultaneously, e.g., using a side-by-side view or using a sliding / overlay mode). In addition, this enables continuous interaction with a touch-sensitive display that would otherwise not be possible due to the need to employ multiple gestures to customize simultaneous interaction with two or more applications.

[0384] As briefly described above, in some embodiments, the first direction is a downward direction relative to an edge of the second application (e.g., the direction in which gesture 508 is traveling, Figure 5A ), and the second direction is a lateral direction substantially perpendicular to the downward direction (eg, the direction in which gesture 512 is traveling, Figure 5F to Figure 5G ). Allowing the user to move the handle in different directions helps provide the user with an additional and simple way to flexibly customize how they wish to interact with two applications displayed simultaneously. Enabling the user to customize their user experience in this manner enhances the operability of the device and makes the human-computer interface more efficient (e.g., their customized choices about where each application should be displayed will allow them to use various features more efficiently without having to work with the default arrangement). Additionally, this enables continued interaction with the touch-sensitive display that would otherwise not be possible due to the user being forced to interact with two or more applications displayed together in a default manner and without any user-specific customization.

[0385] Now go to Fig. 9B Prior to performing operation 918, when the second application overlays the resized first application, the device detects (922) user input at the resized first application (e.g., receiving contact 510 on link 504, Figure 5C The device then responds to the user input via the first application (924) (e.g., opens the content associated with link 504, Figure 5D). In other words, the first application remains available while the second application covers it (e.g., the first application remains available because features of the first application remain available even during overlay / slide mode). In some embodiments, the first application and the second application can be used simultaneously, even if the second application covers the first application, allowing the user to interact with various features of each of the two applications at the same time. Allowing the user to interact with the first application even if the second application is overlaid on top of the first application allows the user to customize how they want to interact with both applications displayed simultaneously. Enabling users to customize their user experience in this manner enhances the operability of the device and makes the human-computer interface more efficient (e.g., their customized choices will allow them to use various features more efficiently without having to work with the default arrangement). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible because the user is forced to interact with two or more applications displayed together in a default manner and without any user-specific customization.

[0386] In some embodiments, after performing operations 922 and 924 (or alternatively, before performing these operations), in response to opening a third application (e.g., contact 516 on a camera application) that is different from the first application and the second application, Figure 5K ), the device displays (926): (i) a third application in a portion of the display area previously occupied by the first application, and (ii) the second application overlaying the third application. Figure 5K to Figure 5L As shown, the user activates their home screen and requests to open a third application by clicking on the enable representation corresponding to the third application. In response to the request, the third application is then displayed in a portion of the display area previously occupied by the first application, and the second application is displayed as overlaying the third application. In other words, the overlay application (e.g., the second application) remains even after the user switches to open the new application. Allowing the overlay application to remain displayed after the user has opened the new application helps ensure that continued interaction with the features of the electronic device is maintained (e.g., if the overlay application does not remain displayed, the features accessed by the user through the overlay application may have to be reopened again). Because the user is also able to avoid having to reopen the features of the overlay application, the operability of the device is improved and the human-computer interface works more efficiently.

[0387] Referring back to the description of operation 918, where it is determined that the direction (in which the contact on the affordance is traveling) is a second direction, in some embodiments, the additional operation shown at 928 is also performed. First, the second application is initially displayed adjacent to the first edge of the touch-sensitive display (e.g., Fig. 5F The video browsing application is shown as being adjacent to the rightmost edge of the touch-sensitive display 112). Additionally, moving the second application within the display area may include moving the second application in a second direction until the second application is adjacent to a second edge of the touch-sensitive display opposite the first edge (e.g., the video browsing application follows the gesture as the gesture 512 moves across the touch-sensitive display 112, such as Figure 5G In these embodiments, moving the second application within the display area may also include: displaying the first application in a remaining portion of the display area different from the portion occupied by the second application, such that the first application is adjacent to the first edge of the touch-sensitive display (e.g., Figure 5H ). Allowing the user to move applications by dragging the affordance 506 helps the user customize how they wish to interact with two applications displayed simultaneously. Enabling the user to customize their user experience in this manner enhances the operability of the device and makes the human-machine interface more efficient (e.g., their customized choices about where each application should be displayed will allow them to use various features more efficiently without having to work with the default arrangement). Additionally, this enables continued interaction with the touch-sensitive display that would otherwise not be possible due to the user being forced to interact with two or more applications displayed together in a default manner and without any user-specific customization.

[0388] In the embodiment described above with reference to 928, a preview of the portion occupied by the second application is displayed next to the first application before the second application is moved until the second application abuts the second edge (930). In some cases, a user of the computing system may be unaware of the changes that will result from certain interactions on the touch-sensitive display. Providing clear visual feedback (e.g., displaying a preview of the location to which the second application will be moved) gives the user an opportunity to preview the changes that will occur after they release (or lift off) contact. Providing such clear visual feedback enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to understand the changes that will occur and avoiding having to reverse unwanted changes). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible because the user would have to repeatedly undo or reverse unwanted changes that they could not anticipate.

[0389] Now go to Fig. 9C , additional interactions available after displaying the overlay application are shown in operations 932, 934, and 936. For example, at operation 932, after resizing the first application and displaying the second application overlaying the resized first application (e.g., Figure 4AIAs shown, the web browsing application overlays application 1): the device detects additional contacts on the affordance (e.g., contacts associated with gesture 482, Figure 4AI ) and movement of the additional contact along a third direction opposite to the first direction (e.g., along Figure 4AI In response to detecting that the additional contact has moved a threshold distance (e.g., 1-2 centimeters) in the third direction, the device stops displaying the second application overlying the resized first application and redisplays the first application and the second application in a display area that occupies substantially the entire touch-sensitive display (e.g., as shown). Figure 4AJ As shown, the web browsing application is now pinned next to Application 1 such that both applications essentially take up the entire touch-sensitive display and are displayed in a side-by-side / split view format).

[0390] Continue to refer Fig. 9C In some embodiments, after resizing a first application and displaying a second application overlaying the resized first application (e.g., Figure 4AI As shown, the web browsing application overlays the application 1): the device detects (934) additional contact on the enabling representation and movement of the additional contact along a first direction (e.g., Figure 4AO The contact associated with gesture 487 is on enabling representation 467 and moves in a first direction). Although Figure 4AO The example in 4 shows two applications displayed side by side and another application covering both applications, but gesture 487 is still available when the overlaying application is also displayed on a single application (e.g., gesture 487 can also be used in Figure 4AI In response to detecting that the additional contact has moved a second threshold distance (e.g., 1-2 centimeters) on the touch-sensitive display, the device scales down content associated with the second application based on the movement of the additional contact until a user interface object having a representation corresponding to the enable representation of the second application is displayed (e.g., the content is scaled down until the content is no longer displayed and UI object 473-B is displayed instead, as shown in FIG. 4 ). Figure 4AP shown).

[0391] In some embodiments, the user can then move UI object 473-B to completely dismiss the second application or to pin the second application next to the first application. For example, based on determining that an additional contact is moved on the taskbar and lifted off the touch-sensitive display (e.g., a contact associated with gesture 488, Figure 4AO to Figure 4AQ , dragged over taskbar 408 and subsequently lifted off (not shown), the device dismisses the second application and displays the first application occupying substantially the entire touch-sensitive display (e.g., displaying Figure 4AG, where application 1 occupies substantially the entire touch-sensitive display); and, based on determining that the additional contact moves to within a threshold distance of an edge of the touch-sensitive display (e.g., to Figures 4AB to 4AE The device redisplays the first application and the second application in a display area that substantially occupies the entire touch-sensitive display (e.g., displays the first application and the second application). Figure 4AJ The user interface shown, where application 1 is displayed side by side with a web browsing application in a split view mode).

[0392] Although the examples provided above generally involve overlaying a second application on top of a currently displayed first application, in some embodiments, an application may be overlaid on top of two applications fixed in a side-by-side display format. Figure 4AJ to Figure 4AR Examples of this situation are provided in and described above. Aspects of these examples may supplement and / or replace certain operations described above with respect to method 900.

[0393] It should be understood that FIG. 9A to FIG. 9B The specific order of operations described in the description is merely an example and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, method 900 may be supplemented (or various operations may be reordered) to incorporate aspects of method 800, for example, regarding using a taskbar to activate split view mode. For the sake of brevity, these details are not repeated here.

[0394] FIG. 10A to FIG. 10C is a flow chart representation of a method 1000 according to some embodiments. FIG. 6A to FIG. 6Q Used to show FIG. 10A to FIG. 10C Although some of the examples that follow will be given with reference to input on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface 195 that is separate from the display 194, such as Figure 1D as shown in .

[0395] In some embodiments, method 1000 is performed by an electronic device (e.g., Figure 1A The method 1000 is performed by one or more components of the electronic device (e.g., I / O subsystem 106, operating system 126, etc.) and / or the portable multifunction device 100 of the device. In some embodiments, the method 1000 is performed by a computer program stored in a non-transitory computer-readable storage medium and executed by one or more processors of the device, such as one or more processors 122 ( Figure 1A) is managed by instructions executed by the device 100. For ease of explanation, the following describes a method 1000 executed by the device 100. In some embodiments, referring to Figure 1A , the operations of method 1000 are performed at least in part by or using the following items: a multitasking module (e.g., multitasking module 180) and its components, a contact / motion module (e.g., contact / motion module 130), a graphics module (e.g., graphics module 132), and a touch-sensitive display (e.g., touch-sensitive display system 112). Some operations in method 1000 are optionally combined, and / or the order of some operations is optionally changed.

[0396] As described below, method 1000 provides intuitive and seamless interaction for multitasking on a handheld electronic device. Method 1000 reduces the number of inputs from a user in order to display multiple applications on a device and switch to different applications, thereby creating a more efficient human-computer interface. For battery-powered electronic devices, providing interaction for multitasking both saves power and increases the time between battery charges (e.g., saving time and eliminating energy-consuming operations when a user must close one application and open another).

[0397] See now Fig. 10A In executing method 1000, the device simultaneously displays (1002) at least two applications in a display area that occupies substantially the entire touch-sensitive display. Fig. 6A , a first application (Application 1) is displayed adjacent to / side by side with a second application (Application 2 or a video application), and both applications are displayed in a display area 601a that substantially occupies the entire touch screen 112.

[0398] The device receives (1004) user input corresponding to a request to display an application switcher user interface. In some embodiments, the application switcher user interface may refer to a user interface in which representations of recently used applications are displayed (e.g., applications that are open and in use in the foreground of the electronic device and have not yet been closed by a user of the device). In some embodiments, the user input may correspond to a double-click input at a physical or solid-state home button of the electronic device, or may alternatively correspond to a continuous upward gesture on a touch-sensitive display (e.g., gesture 602, FIG. 6A to FIG. 6D and Figure 6O to Figure 6P). For example, in some embodiments, a gesture that reaches a first threshold distance (e.g., 1-2 centimeters) from the edge of the screen displays the taskbar, and the taskbar begins to follow the gesture as the gesture moves across the display; after the gesture has moved a second threshold distance (e.g., 2-4 centimeters in total or an additional 1-2 centimeters beyond the first threshold distance), the taskbar remains in a static position and no longer follows the gesture; when gesture 602 continues to move beyond the second threshold distance, the currently displayed application begins to shrink in size and an application switcher user interface is displayed; and once the gesture moves at least a third threshold distance (e.g., 3-6 centimeters in total or an additional 1-2 centimeters beyond the second threshold distance), the application switcher user interface is displayed.

[0399] In response to receiving the user input, the device displays (1006) an application switcher user interface (e.g., FIG. 6D to FIG. 6M and Figure 6P to Figure 6Q In some embodiments, displaying the application switcher user interface includes (1008) gradually reducing the size of the display area or scaling the display area until it reaches a predefined size associated with the representation. FIG. 6D to FIG. 6E middle, Fig. 6A The size of the illustrated display area 601a gradually decreases until the miniaturized view of the display 601a reaches a predefined size associated with the representation. In this manner, a clear visual indication is provided to the user as to where the pinned application they are viewing will be located within the application switcher user interface, which helps enhance the operability of the device (e.g., by ensuring that the user can easily locate the pinned applications within the application switcher UI without having to waste time trying to locate them within this UI).

[0400] In some embodiments, the application switcher user interface includes (1010) corresponding representations of at least some applications that were recently used on the electronic device, and the corresponding representations include representations of a display area having at least two applications. Figure 6P and Figure 6Q , representation 638 includes a representation of a display area with two applications (Application 4 and Application 5). In some embodiments, the corresponding representation shows the activity state for each of at least some applications recently used on the electronic device (e.g., the last displayed screen of a particular application). In some embodiments, the representation of the display area with at least two applications is a miniaturized view of the display area that previously occupied substantially the entire touch-sensitive display. For example, in Fig. 6E In the middle, it means that 604 includes Fig. 6AA miniaturized view of display area 601a is shown, and the miniaturized view shows the displays of Application 1 and the Video application side by side.

[0401] In some embodiments, the corresponding representation includes (1012) an additional representation of an additional application associated with the overlay application, and the additional representation does not show the overlay application. Figure 6N As shown, the additional applications are Application 1 and Application 2, with the video application overlaid on Application 1. Application 1 is displayed in display area 601b that occupies substantially the entire touch-sensitive display 112. Figure 6P to Figure 6Q , additional representation 634 does not show the overlaid video application. In this way, the human-computer interface is made more efficient by presenting only representations of the pinned application (which can be interacted with directly within the application switcher user interface) without presenting representations of overlaid applications. To avoid cluttering the application switcher user interface, in some embodiments, the additional representation representing application 1 does not show the overlaid second application.

[0402] In some embodiments, displaying at least two applications in the display area includes (1014) simultaneously displaying at least two applications side by side in the display area (e.g., Fig. 6A 1 and the video application in the display area), and displaying the representation of the display area with at least two applications includes simultaneously displaying the at least two applications side by side in the representation of the display area (e.g., representation 604 is a miniaturized view showing a split view mode of application 1 and the video application, FIG. 6D to FIG. 6F ).

[0403] In some embodiments, displaying the application switcher user interface includes (1016) displaying a taskbar including a plurality of enable representations corresponding to applications available on the electronic device, e.g., FIG. 6B to FIG. 6M and FIG. 6O to FIG. 6Q In such an embodiment, reference is made to the task bar 408 including multiple affordances. Fig. 10B The device further detects (1018) a contact on an affordance of the plurality of affordances included in the taskbar (e.g., Fig. 6F 603) (e.g., contact 603 on the browser affordance in task bar 408, Fig. 6F ), and detecting (1020) movement of the contact toward the representation of the display area (e.g., movement of contact 603 away from taskbar 408 and toward representation 604, Fig. 6F), without detecting a lift-off between detecting the contact and the movement. In some embodiments, upon detecting a lift-off of the contact on a first application of at least two applications included in the representation in the display area, the device stops (1022) displaying the first application in the representation in the display area and instead displays an additional application corresponding to the affordance within the representation in the display area. For example, FIG. 6F to FIG. 6G , after contact lift-off is detected, representation 604 is updated to show that the web browsing application is now displayed side by side with application 1 (and the memory of device 100 may also be updated to reflect that these applications have been pinned together). In updated representation 616, the video application is released from representation 604 and replaced by the web browsing application.

[0404] As described above, allowing a user to view a representation within the application switcher user interface that shows the applications displayed in split view mode enables the user to easily and efficiently identify the pinned applications. By also enabling the user to interact with the pinned applications directly from the application switcher user interface, the operability of the device is enhanced and the human-computer interface works more efficiently (e.g., directly interacting with the pinned applications from within the application switcher user interface ensures that the user avoids having to exit the application switcher user interface in order to perform their interaction). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible due to having to employ multiple gestures to leave the application switcher user interface and then begin interacting with (or dismissing / replacing) the pinned application.

[0405] In some embodiments, while the application switcher user interface is displayed, the device detects (1024) additional user input (e.g., a non-tap selection 618b, such as a contact that lasts for more than a predefined amount of time (a long press) or a contact that increases in intensity to above a threshold intensity (e.g., a deep / hard press), on a first application of at least two applications included in the representation of the display area. Fig.6I In response to detecting additional user input, in some embodiments, the device displays (1026) a disabling indication on at least a portion of the first application (e.g., disabling indication 620 corresponds to an “x” displayed on the portion of the first application, as shown). Fig.6I and Figure 6J Once a selection to release the affordance is detected (e.g., user input 622 is detected, Figure 6J ), the device closes (1028) the first application and displays updated representations of the remaining at least two applications other than the first application in an application switcher user interface, e.g., Figure 6K, updated representation 624 displays application 1 as the remaining application of the two applications previously displayed in representation 604.

[0406] As described above, allowing a user to view a representation within the application switcher user interface that shows an application displayed in split view mode enables the user to easily and efficiently identify pinned applications. By also enabling the user to interact with the pinned applications directly from the application switcher user interface, the operability of the device is enhanced and the human-computer interface works more efficiently (e.g., directly interacting with the pinned application from within the application switcher user interface ensures that the user avoids having to exit the application switcher user interface in order to perform their interaction). In addition, this enables continued interaction with the touch-sensitive display, which would otherwise not be possible due to having to employ multiple gestures to leave the application switcher user interface and then begin interacting with the pinned application.

[0407] Reference now Fig. 10C In some embodiments, the device detects (1030) a contact on a first application of at least two applications included in the representations in the display area, and detects (1032) movement of the contact away from the representations in the display area (e.g., drag gesture 626, where the first portion is the contact on the video application in representation 604, and where the second portion is movement of the contact away from representation 604, Figure 6L ). Upon detecting that the contact has moved to a predefined area within the application switcher user interface, the device stops (1034) displaying the first application in the representation and displays updated representations of the remaining at least two applications other than the first application in the application switcher user interface. In such an embodiment, a taskbar is displayed (1036) covering at least a predefined area of ​​the application switcher user interface, the taskbar including enable representations corresponding to a plurality of applications available on the electronic device (in other words, movement of the contact to the predefined area includes movement of the contact within the taskbar). For example, in Figure 6L In the example, task bar 408 includes a phone affordance, an email affordance, a browser affordance, and a note affordance, which correspond to the phone application, email application, web browsing application, and notepad application available on device 100. Figure 6L As shown, upon detecting the drag 626 of the video application on the taskbar 408, the device releases the video application from the representation 604. Figure 6MAs shown, updated representation 624 includes application 1 as the remaining application of the two applications previously included in representation 604. By displaying a taskbar covering at least the predefined area (additional details and examples regarding the taskbar are provided above with reference to method 800), a target is provided to the user to drag the dismissed application to, thereby enhancing the operability of the device (e.g., by ensuring that the user can dismiss the application directly from the application switcher UI without having to waste input by leaving the application switcher UI and using a large, complex, and difficult to remember input combination to dismiss the application).

[0408] In this way, users are provided with a simple gesture for undocking pinned applications directly from the application switcher user interface, thereby enhancing device operability (e.g., by ensuring that users can undock applications directly from the application switcher UI without wasting input by leaving the application switcher UI and using large, complex, and difficult-to-remember input combinations to undock applications).

[0409] It should be understood that FIG. 10A to FIG. 10C The specific order of operations described in the description is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art will think of many ways to reorder the operations described herein. For the sake of brevity, these details will not be repeated here.

[0410] FIG. 11A to FIG. 11B is a flow chart depicting a method 1100 for activating two or more instances of the same application. FIG. 7A to FIG. 7L Used to show the combination FIG. 11A to FIG. 11B The methods and / or processes described.

[0411] In some embodiments, method 1100 is performed by an electronic device (e.g., Figure 1A The method 1100 is performed by one or more components of the electronic device (e.g., I / O subsystem 106, operating system 126, etc.) and / or the portable multifunction device 100 of the device. In some embodiments, the method 1100 is performed by a computer program stored in a non-transitory computer-readable storage medium and executed by one or more processors of the device, such as one or more processors 122 ( Figure 1A ) is managed by instructions executed by the device 100. For ease of explanation, the following describes a method 1100 executed by the device 100. In some embodiments, referring to Figure 1A, the operations of method 1100 are performed at least in part by or using the following items: a multitasking module (e.g., multitasking module 180) and its components, a contact / motion module (e.g., contact / motion module 130), a graphics module (e.g., graphics module 132), and a touch-sensitive display (e.g., touch-sensitive display system 112). Some operations in method 1100 are optionally combined, and / or the order of some operations is optionally changed.

[0412] As described below, method 1100 provides an intuitive way to activate two or more instances of the same application. Sometimes, a user of a computing system loses access to certain features of an application after choosing to open other features within the application (e.g., a user may not be able to view a received email after selecting the option to reply to the received email). Opening multiple instances of a single application allows the user to continue to view and use these specific features. Enabling the user to continue to view and use these specific features enhances the operability of the device and makes the human-computer interface more effective (e.g., by enabling the user to use these specific features without having to provide multiple inputs to reopen them after closing these specific features). In addition, this enables continuous interaction with a touch-sensitive display, which would otherwise not be possible due to having to use multiple gestures or inputs to figure out how to reopen or access these features after they are closed or no longer displayed.

[0413] According to some embodiments, a method 1100 for activating two or more instances of the same application is provided. For example, a first instance of a mail application (e.g., showing an inbox) is opened on one side of a screen, while a second instance of the same mail application (e.g., showing a single email reply) is opened on the other side of the screen. An instance is a specific single occurrence of an object that typically exists during the runtime of a computer program, e.g., each time a program or application runs, it is an instance of the program. In some embodiments, on a computer having a touch-sensitive display (e.g., Figure 1C 112) and one or more processors (e.g., Figure 1A The method is performed at a portable multifunction device 100).

[0414] In this embodiment, first, a first instance of an application is displayed on substantially the entire touch-sensitive display 1102. Then, a contact is detected 1104 on an affordance displayed within the first instance of the application. For example, at 1106, the application is an email application, and the contact on the affordance is a contact on a link or affordance (e.g., a reply arrow or affordance, a reply all affordance, a compose new message link or affordance, or a hyperlink) displayed in the first instance of the email application.

[0415] In some embodiments, the user input or contact is not a simple tap or touch, but a deep or heavier press (e.g., a press with a force or intensity greater than a predetermined contact force or intensity—see FIG. 3A to FIG. 3C ) or a press and hold for a predetermined amount of time (e.g., 1-3 seconds). Touch screen ( Figure 1A 112), processor ( Figure 1A 122), a contact force sensor (165 of FIG. 1) and / or a force sensor controller (159 of FIG. 1) determine whether the user input (e.g., contact or gesture) is of a predetermined type (e.g., a deep press or a press and hold) on the reply indication.

[0416] Upon detecting a contact on the reply affordance (and, in some embodiments, determining that the user input or contact is of a predetermined type), the touch screen displays a preview of the new instance of the application 1108. For example, at 1110, the application is an email application, the affordance is a representation of a message displayed in the first instance of the email application, and the preview is displayed upon detecting a contact having a contact intensity greater than a predefined intensity threshold, or upon maintaining the contact on the affordance for more than a predefined amount of time on the touch-sensitive display (referred to as a press and hold contact or gesture).

[0417] Movement of the contact on the touch-sensitive display is then detected 1112. In some embodiments, the movement needs to occur without the contact being lifted off between the initial contact and the movement (e.g., the contact is continuous and the contact is not lifted off the touch-sensitive display).

[0418] Then, at 1114, in response to detecting that the contact has moved more than a threshold distance (e.g., 1-2 centimeters) on the touch-sensitive display, (i) the first instance of the application is resized to occupy a first portion (e.g., the left half) of the touch-sensitive display, and (ii) a predefined portion (e.g., the second portion) of the touch-sensitive display adjacent to the first portion is displayed (e.g., the right third to half of the display). In other embodiments, in response to detecting that the contact has moved to a predefined portion (e.g., the second portion below) of the touch screen (e.g., moved to the right third to half of the screen), (i) the first instance of the application is resized to occupy the first portion (e.g., the left half) of the touch-sensitive display, and (ii) a predefined portion (e.g., the second portion) of the touch-sensitive display adjacent to the first portion is displayed (e.g., the right third to half of the display). In other embodiments, either or both of these movements activate split view mode.

[0419] In some embodiments, the predefined portion is a blank area that occupies approximately one-third to one-half of the total display area of ​​the touch-sensitive display (e.g., Fig. 7E portion 762). In other embodiments, the blank area displays the home screen of the device within the predefined area. In some embodiments, displaying the blank area provides a visual indication to the user that, when the contact is lifted, the content (currently displayed in the preview) will be displayed in a predefined portion adjacent to the now resized first instance of the application displayed in the first portion. In this way, a preview of the second portion of the display is provided to the user of the computing system to provide them with a visual preview of how the split view mode will appear. This enhances the operability of the device because the user is able to decide whether to activate the split view mode after viewing the preview, thereby ensuring that the user can discover how to activate the split view mode and that the user can preview the mode before activating such mode.

[0420] Then, at 1116, upon detection of lift-off of the contact after the contact has moved a threshold distance on the touch-sensitive display or to a predefined second portion of the display, two instances of the same application are simultaneously displayed side by side. Specifically, (i) a second instance of the application that is different from the first instance is displayed as having content associated with the selected affordance, and (ii) at least a portion of the first instance of the application is displayed. The second instance and the first instance (or portions thereof) are displayed as occupying substantially all of the displayable area of ​​the touch-sensitive display.

[0421] In some embodiments, as depicted at 1118, the split view mode of operation includes displaying a second instance of the application in a predefined portion (e.g., the second portion) of the touch-sensitive display.

[0422] In some embodiments, the first instance and the second instance of the application are different executable processes executed on one or more processors of the electronic device (e.g., two different instantiations of the application are each executed separately on one or more processors of the electronic device). In other words, after lift-off is detected, the second instance of the application is instantiated and then displayed side-by-side with the first instance of the application, thereby enabling the user to easily / conveniently view and use content from the first instance while also interacting with the second instance.

[0423] It should be understood that FIG. 11A to FIG. 11B The specific order of operations described in the description is only an example and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art will think of many ways to reorder the operations described herein. For the sake of brevity, these details will not be repeated here.

[0424] Fig.12 is a flow diagram depicting a method 1200 for activating a taskbar and then an application switcher user interface using a continuous gesture. FIG. 6A to FIG. 6Q Used to show the combination Fig.12 The methods and / or processes described.

[0425] In some embodiments, method 1200 is performed by an electronic device (e.g., Figure 1A The method 1200 is performed by one or more components of the electronic device (e.g., I / O subsystem 106, operating system 126, etc.) and / or the portable multifunction device 100 of the device. In some embodiments, the method 1200 is performed by a non-transitory computer-readable storage medium and by one or more processors of the device, such as one or more processors 122 ( Figure 1A ) is managed by instructions executed by the device 100. For ease of explanation, the following describes a method 1200 performed by the device 100. In some embodiments, reference Figure 1A , the operations of method 1100 are performed at least in part by or using the following items: a multitasking module (e.g., multitasking module 180) and its components, a contact / motion module (e.g., contact / motion module 130), a graphics module (e.g., graphics module 132), and a touch-sensitive display (e.g., touch-sensitive display system 112). Some operations in method 1200 are optionally combined, and / or the order of some operations is optionally changed.

[0426] As described below, method 1200 provides a single continuous gesture for a user that allows activation of a taskbar and then an application switcher user interface. The method reduces the number of inputs required by a user to activate the taskbar and the application switcher user interface (e.g., the user only needs to provide a simple gesture instead of performing multiple inputs), thereby ensuring that the battery life of an electronic device implementing method 1200 is extended because less power is required to process a smaller number of inputs (and this savings will be repeatedly achieved as the user becomes more familiar with more intuitive and simple gestures). As also explained in detail below, the operation of method 1200 helps ensure that users are able to interact continuously (e.g., they do not need to frequently perform undo actions, which interrupts their interaction with the device), and the operation of method 1200 helps produce a more effective human-computer interface.

[0427] In some embodiments, a device configured to perform method 1200 displays (1202) content on a touch-sensitive display (e.g., device 100 displays application 1 and also displays a video browsing application in split view mode on the touch-sensitive display, Fig. 6A ). The device then detects (1204) a contact (e.g., a contact associated with gesture 602, Fig. 6A) in a direction substantially perpendicular to an edge of the touch-sensitive display (e.g., within + / - 5 degrees) (e.g., the contact moves in a substantially perpendicular direction relative to a bottom-most edge of the touch-sensitive display (relative to a user looking down at the touch-sensitive display)). Upon determining that the movement of the contact satisfies a first motion-based threshold (e.g., the contact has moved more than a threshold distance away from the edge or the contact has traveled at a speed or acceleration equal to or greater than a first threshold), the device displays (1206) a taskbar (e.g., such as a taskbar that overlays at least a portion of the content) that covers the content. FIG. 6A to FIG. 6C As shown), the taskbar includes multiple enable indications that, when selected, cause the electronic device to open corresponding applications available on the electronic device. In some embodiments, when the contact moves in a direction substantially perpendicular to the edge, the taskbar is displayed based on the movement or speed of the contact (for example, the taskbar is first displayed after the contact moves an initial threshold distance of, for example, 1-2 cm, and then the taskbar subsequently reaches a stationary position when the contact moves a first threshold distance of, for example, 1-2 cm or a total of 2-4 cm from the initial threshold distance). The above threshold distance can be an initial threshold distance or a first threshold distance, or it can also be a threshold based on speed or acceleration.

[0428] When the taskbar is displayed, and based on determining that movement of the contact satisfies a second motion-based threshold (e.g., the contact has moved beyond a second threshold distance, such as 3-6 centimeters away from an edge or an additional 1-2 centimeters from the first threshold distance, or if the first threshold speed corresponds to a speed of 4 centimeters per second, the contact has traveled at a speed of 8 centimeters per second) greater than the first motion-based threshold, the device activates (1208) an application switcher user interface that includes corresponding selectable representations of at least some of the applications that were most recently used on the electronic device (e.g., FIG. 6C to FIG. 6D In some embodiments, when the contact moves away from the edge of the touch-sensitive display beyond a second threshold distance or travels at a speed equal to or greater than a second velocity or acceleration threshold, the content (e.g., content associated with the foreground application) shrinks to its temporal position within the application switching UI (e.g., as shown in FIG. 1 ). FIG. 6D to FIG. 6E shown).

[0429] In some embodiments, the first motion-based threshold and the second motion-based threshold are selected (1210) from the group consisting of a distance threshold, a speed threshold, or an acceleration threshold.

[0430] In some embodiments, any distance-based thresholds described herein (e.g., determining whether a contact associated with gesture 602 travels more than a threshold distance before displaying a taskbar and subsequently displaying an application switcher user interface) may be replaced with movement-based thresholds (e.g., distance-based thresholds or thresholds based on movement and speed or changes in speed, such as speed- or acceleration-based thresholds) that instead determine whether a contact has moved at a speed or acceleration greater than a speed threshold (e.g., having a speed of at least 8 cm / second) or an acceleration threshold before performing a particular action (e.g., displaying a taskbar and then displaying an application switcher user interface).

[0431] In some embodiments, when a contact (e.g., a contact associated with gesture 602, FIG. 6A to FIG. 6D ) moves beyond a first threshold distance and then away from the edge of the touch-sensitive display beyond a second threshold distance without detecting a lift-off of the contact (for example, the movement of the contact is a continuous, uninterrupted gesture that first causes the electronic device to display a taskbar and then causes the electronic device to display an application switcher user interface).

[0432] By providing a single gesture for activating the taskbar and then activating the application switcher user interface, device operability is enhanced and the human-computer interface operates more efficiently (e.g., because the user is able to provide a single input to activate the taskbar and the application switching user interface, rather than having to employ multiple inefficient and time-consuming inputs to separately access the taskbar and then, at a later point in time, access the application switcher user interface).

[0433] It should be understood that Fig.12 The specific order of the operations described herein is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art...

Claims

1. A method comprising: At an electronic device having a touch-sensitive display and one or more processors: displaying content on the touch-sensitive display; detecting movement of a contact in a direction substantially perpendicular to an edge of the touch-sensitive display; upon determining that the movement of the contact satisfies a first movement-based threshold, displaying a taskbar covering at least a portion of the content, the taskbar comprising a plurality of affordances that, when selected, cause the electronic device to open corresponding applications available on the electronic device, and While the taskbar is displayed and based on determining that movement of the contact satisfies a second motion-based threshold that is greater than the first motion-based threshold, activating an application switcher user interface that includes corresponding selectable representations of at least some applications most recently used on the electronic device. 2 . The method of claim 1 , wherein the first motion-based threshold and the second motion-based threshold are selected from the group consisting of a distance threshold, a speed threshold, or an acceleration threshold.

3. A computer-readable storage medium storing executable instructions, which, when executed by one or more processors of an electronic device having a touch-sensitive display, causes the electronic device to perform operations including: displaying content on the touch-sensitive display; detecting movement of a contact in a direction substantially perpendicular to an edge of the touch-sensitive display; upon determining that the movement of the contact satisfies a first movement-based threshold, displaying a taskbar covering at least a portion of the content, the taskbar comprising a plurality of affordances that, when selected, cause the electronic device to open corresponding applications available on the electronic device, and While the taskbar is displayed and based on determining that movement of the contact satisfies a second motion-based threshold that is greater than the first motion-based threshold, activating an application switcher user interface that includes corresponding selectable representations of at least some applications most recently used on the electronic device. 4 . The computer-readable storage medium of claim 3 , wherein the first motion-based threshold and the second motion-based threshold are selected from the group consisting of a distance threshold, a speed threshold, or an acceleration threshold.

5. An electronic device comprising: one or more processors; Touch-sensitive display; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: displaying content on the touch-sensitive display; detecting movement of a contact in a direction substantially perpendicular to an edge of the touch-sensitive display; upon determining that the movement of the contact satisfies a first movement-based threshold, displaying a taskbar covering at least a portion of the content, the taskbar comprising a plurality of affordances that, when selected, cause the electronic device to open corresponding applications available on the electronic device, and While the taskbar is displayed and based on determining that movement of the contact satisfies a second motion-based threshold that is greater than the first motion-based threshold, activating an application switcher user interface that includes corresponding selectable representations of at least some applications most recently used on the electronic device. 6 . The electronic device of claim 5 , wherein the first motion-based threshold and the second motion-based threshold are selected from the group consisting of a distance threshold, a speed threshold, or an acceleration threshold.

7. A computer program product comprising one or more programs, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 2.

8. A method comprising: At an electronic device having a touch-sensitive display and one or more processors: displaying the first application and the second application in a display area occupying substantially the entire touch-sensitive display; When the first application and the second application are displayed in the display area: displaying an affordance near an edge of the second application; detecting a contact on the affordance moving in a direction across the touch-sensitive display; Determining that the direction is the first direction according to: resizing the first application to occupy substantially all of the display area; as well as displaying the second application overlaying the resized first application; and Based on determining that the direction is a second direction substantially perpendicular to the first direction, the second application is moved within the display area.

9. A method comprising: At an electronic device having a touch-sensitive display and one or more processors: displaying a first instance of an application on substantially all of the touch-sensitive display; detecting a contact on an affordance displayed within the first instance of the application; detecting movement of the contact across the touch-sensitive display; When lift-off of the contact is detected after the contact has moved across the touch-sensitive display and reached a predefined portion of the touch-sensitive display, a split-view operating mode of the electronic device is activated, the mode comprising simultaneously displaying: (i) a second instance of the application that is different from the first instance and is displayed as having content associated with the enable representation, and (ii) at least a portion of the first instance of the application, wherein the displayed second instance and the portion of the first instance occupy substantially the entire touch-sensitive display.

Images