Systems and methods for proactively identifying and surfacing relevant content on touch-sensitive devices

Abstract

Disclosed herein are systems and methods for proactively identifying and surfacing relevant content on an electronic device having a touch-sensitive display. In one aspect, the method includes executing an application on the electronic device in response to a command from a user of the electronic device. The method further includes collecting usage data while the application is running. The usage data includes at least one action performed by the user within the application. The method also includes automatically and without human intervention obtaining at least one trigger condition based on the collected usage data and associating the at least one trigger condition with a particular action from the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that a particular action associated with the trigger condition is available.

Description

[Technical Field]

[0001] TECHNICAL FIELD Embodiments disclosed herein relate generally to electronic devices having touch-sensitive displays, and more particularly to systems and methods for proactively identifying and surfacing relevant content on electronic devices in communication with a display and a touch-sensitive surface. [Background technology]

[0002] Handheld electronic devices with touch-sensitive displays are widespread. Users of these widespread handheld electronic devices currently install numerous applications on their devices and use these applications to perform their daily activities more efficiently. However, to access these applications, users must typically unlock their device, locate the desired application (e.g., by browsing around a home screen to locate an icon associated with the desired application or by searching for the desired application in a search interface), and then also locate the desired function within the desired application. Thus, users often spend a significant amount of time locating the desired application and the desired function within these applications, instead of simply being able to instantly run the desired application (e.g., with a single-touch input) and / or perform the desired function.

[0003] Furthermore, with many installed applications, users are inundated with a continuous stream of information that cannot all be viewed instantly. As a result, users often want to return to view or use specific information that they previously noticed. However, users often cannot locate specific information or cannot remember how to locate specific information.

[0004] It is therefore desirable to provide an intuitive and easy-to-use system and method for proactively identifying and surfacing relevant content (e.g., specific information) on an electronic device in communication with a display and a touch-sensitive surface. Summary of the Invention

[0005] Thus, there is a need for electronic devices with faster, more efficient methods and interfaces for quickly accessing applications and desired functions within those applications. Furthermore, there is a need for electronic devices that assist users in managing the continuous flow of information they receive daily by proactively identifying and providing relevant information (e.g., contacts, nearby places, applications, news articles, addresses, and other information available on the device) before this information is explicitly requested by the user. Such methods and interfaces optionally complement or replace traditional methods for accessing applications. Such methods and interfaces result in more efficient human-machine interfaces by requiring less input from the user to locate desired information. For battery-powered devices, such methods and interfaces conserve power and extend the time between battery charges (e.g., by requiring fewer touch inputs to perform various functions). Furthermore, such methods and interfaces increase the lifespan of touch-sensitive displays by requiring fewer touch inputs (e.g., instead of having to continuously and aimlessly tap a touch-sensitive display to locate desired information, the methods and interfaces disclosed herein proactively provide this information without requiring user input).

[0006] The above deficiencies and other problems associated with user interfaces for electronic devices with touch-sensitive surfaces are addressed by the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touchscreen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in the memory for performing a number of functions. In some embodiments, a user interacts with the GUI primarily through stylus and / or finger contacts and gestures on the touch-sensitive surface. In some embodiments, the functions optionally include image editing, drawing, presenting, word processing, website creation, disc authoring, spreadsheet creation, game playing, making phone calls, video conferencing, emailing, instant messaging, fitness support, digital photography, digital videography, web browsing, digital music playback, and / or digital video playback. Executable instructions to perform these functions are optionally contained on a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0007] (A1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive display (touchscreen 112, FIG. 1C). The method includes executing an application on the electronic device in response to instructions from a user of the electronic device. The method further includes collecting usage data while the application is running. The usage data includes at least one action (or types of actions) performed by the user within the application. The method also includes (i) automatically, and without human intervention, obtaining at least one trigger condition based on the collected usage data, and (ii) associating the at least one trigger condition with a particular action of the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that a particular action associated with the trigger condition is available.

[0008] (A2) In some embodiments of the method described in A1, obtaining the at least one trigger condition includes transmitting usage data from the electronic device to one or more servers remotely located, and receiving the at least one trigger condition from the one or more servers.

[0009] (A3) In some embodiments of the methods described in A1 or A2, providing the indication includes displaying a user interface object on a lock screen of the touch-sensitive display that corresponds to a particular action associated with the trigger condition.

[0010] (A4) In some embodiments of the method described in A3, the user interface object includes a description of the particular action associated with the trigger condition.

[0011] (A5) In some embodiments of the method according to A4, the user interface objects further include an icon associated with the application.

[0012] (A6) In some embodiments of the method of any one of A3 to A5, the method further includes detecting a first gesture at a user interface object. In response to detecting the first gesture, (i) displaying an application on the touch-sensitive display, and (ii) performing a particular action associated with the trigger condition while displaying the application.

[0013] (A7) In some embodiments of the method according to A6, the first gesture is a swipe gesture on a user interface object.

[0014] (A8) In some embodiments of the method of any one of A3 to A5, the method further includes detecting a second gesture at the user interface object, and in response to detecting the second gesture and while continuing to display the lock screen on the touch-sensitive display, performing a particular action associated with the trigger condition.

[0015] (A9) In some embodiments of the method described in A8, the second gesture is a single tap on a predefined area of ​​the user interface object.

[0016] (A10) In some embodiments of the method according to any one of A3 to A9, the user interface object is displayed in a default center portion of the lock screen.

[0017] (A11) In some embodiments of the method described in A1, providing an indication to the user that a particular action associated with the trigger condition is available includes performing the particular action.

[0018] (A12) In some embodiments of the method according to A3, the user interface object is an icon associated with the application, and the user interface object is displayed substantially in a corner of the lock screen of the touch-sensitive display.

[0019] (A13) In some embodiments of the method of any one of A1 to A12, the method further includes receiving an instruction from a user to unlock the electronic device. The method includes, in response to receiving the instruction, displaying a home screen of the electronic device on the touch-sensitive display. The method also includes providing, on the home screen, an indication to the user that a particular action associated with the trigger condition is available.

[0020] (A14) In some embodiments of the method described in A13, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion displayed adjacent to the first portion for launching a second set of applications available on the electronic device, the second portion being displayed on all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion.

[0021] (A15) In some embodiments of the method described in A14, the second set of applications is different from and smaller than the first set of applications.

[0022] (A16) In some embodiments of the methods described in any one of A1 to A15, determining that at least one trigger condition is met includes determining that the electronic device is coupled to a second device different from the electronic device.

[0023] (A17) In some embodiments of the method according to any one of A1 to A16, determining that at least one trigger condition is met includes determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user.

[0024] (A18) In some embodiments of the method described in A17, determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user includes monitoring motion data from an accelerometer of the electronic device and determining, based on the monitored motion data, that the electronic device has not moved for more than a threshold amount of time.

[0025] (A19) In some embodiments of the method according to any one of A1 to A18, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application continues to run. At least one trigger condition is further based on the verbal command provided to the virtual assistant application.

[0026] (A20) In some embodiments of the method described in A19, the verbal command includes a request to create a reminder that corresponds to a current state of the application, the current state corresponding to the state of the application at the time the verbal command was provided.

[0027] (A21) In some embodiments of the method described in A20, the state of the application when the verbal command is provided is selected from the group consisting of a page displayed within the application when the verbal command is provided, content being played within the application when the verbal command is provided, a notification displayed within the application when the verbal command is provided, and an active portion of a page displayed within the application when the verbal command is provided.

[0028] (A22) In some embodiments of the method described in A20, the verbal command includes the word "this" in reference to the current state of the application.

[0029] (A23) In another aspect, a method is performed on one or more electronic devices (e.g., portable multifunction device 100 (FIG. 5) and one or more servers 502 (FIG. 5)). The method includes executing an application on a first electronic device of the one or more electronic devices in response to instructions from a user of the first electronic device. The method includes automatically and without human intervention collecting usage data while the application is running, the usage data including at least one or more actions (or types of actions) performed by the user within the application. The method further includes automatically and without human intervention establishing at least one trigger condition based on the collected usage data. The method further includes associating the at least one trigger condition with a particular action among the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that the particular action associated with the trigger condition is available.

[0030] (A24) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of A1 to A22.

[0031] (A25) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for performing the method of any one of A1 to A22.

[0032] (A26) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform a method described in any one of A1 to A22.

[0033] (A27) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of A1 to A22.

[0034] (A28) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4201, FIG. 42), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4203, FIG. 42), and a processing unit (e.g., processing unit 4205, FIG. 42). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. IE (i.e., computing devices A through D). For ease of explanation, FIG. 42 shows display unit 4201 and touch-sensitive surface unit 4203 as being integrated into electronic device 4200, although in some embodiments, one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit is coupled to the touch-sensitive surface unit and the display unit. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into a single touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes an execution unit (e.g., execution unit 4207, FIG. 42), a collection unit (e.g., collection unit 4209, FIG. 42), an acquisition unit (e.g., acquisition unit 4211, FIG. 42), an association unit (e.g., association unit 4213, FIG. 42), a providing unit (e.g., providing unit 4215, FIG. 42), a sending unit (e.g., sending unit 4217, FIG. 42), a receiving unit (e.g., receiving unit 4219, FIG. 42), a display unit (e.g., display unit 4221, FIG. 42), a detection unit (e.g., detection unit 4223, FIG. 42), an implementation unit (e.g., implementation unit 4225, FIG. 42), a judgment unit (e.g., judgment unit 4227, FIG. 42), and a monitoring unit (e.g., monitoring unit 4229, FIG. 42).The processing unit (or one or more components thereof, such as units 4207 to 4229) is configured to: execute (e.g., by the execution unit 4207) an application on the electronic device in response to instructions from a user of the electronic device; collect (e.g., by the collection unit 4209) usage data while executing the application, the usage data including at least one action or actions performed by the user within the application; automatically and without human intervention, obtain (e.g., by the obtaining unit 4211) at least one trigger condition based on the collected usage data; associate (e.g., by the associating unit 4213) the at least one trigger condition with a particular action or actions performed by the user within the application; and, upon determining that the at least one trigger condition is satisfied, provide (e.g., by the providing unit 4215) an indication to the user that the particular action associated with the trigger condition is available.

[0035] (A29) In some embodiments of the electronic device described in A28, obtaining the at least one trigger condition includes transmitting usage data from the electronic device to one or more servers located remotely (e.g., by a transmitting unit 4217) and receiving the at least one trigger condition from the one or more servers (e.g., by a receiving unit 4219).

[0036] (A30) In some embodiments of the electronic device described in A28 or A29, providing the indication includes displaying (e.g., by display unit 4217 and / or display unit 4201) a user interface object corresponding to a particular action associated with the trigger condition on a lock screen of the touch-sensitive display unit.

[0037] (A31) In some embodiments of the electronic device described in A30, the user interface object includes a description of a particular action associated with the trigger condition.

[0038] (A32) In some embodiments of the electronic device described in A31, the user interface objects further include an icon associated with the application.

[0039] (A33) In some embodiments of the electronic device described in any one of A30 to A32, the processing unit is further configured to detect a first gesture at the user interface object (e.g., by the detection unit 4223 and / or the touch-sensitive surface unit 4203). In response to detecting the first gesture, (i) display an application on the touch-sensitive display unit (e.g., by the display unit 4217 and / or the display unit 4201), and (ii) perform a particular action associated with the trigger condition (e.g., by the implementation unit 4225) while displaying the application.

[0040] (A34) In some embodiments of the electronic device described in A33, the first gesture is a swipe gesture on a user interface object.

[0041] (A35) In some embodiments of the electronic device described in any one of A30 to A33, the processing unit is further configured to detect a second gesture at the user interface object (e.g., by detection unit 4223 and / or touch-sensitive surface unit 4203). In response to detecting the second gesture, and while continuing to display the lock screen on the touch-sensitive display unit, the processing unit is configured to perform (e.g., by implementation unit 4225) a particular action associated with the trigger condition.

[0042] (A36) In some embodiments of the electronic device described in A35, the second gesture is a single tap on a predefined area of ​​the user interface object.

[0043] (A37) In some embodiments of the electronic device according to any one of A30 to A36, the user interface object is displayed in a default center portion of the lock screen.

[0044] (A38) In some embodiments of the electronic device described in A28, providing an indication to the user that a particular action associated with the trigger condition is available includes performing the particular action (e.g., by the implementation unit 4225).

[0045] (A39) In some embodiments of the electronic device described in A30, the user interface object is an icon associated with the application, and the user interface object is displayed substantially in a corner of the lock screen of the touch-sensitive display unit.

[0046] (A40) In some embodiments of the electronic device described in any one of A28 to A39, the processing unit is further configured to receive (e.g., by the receiving unit 4219) an instruction from a user to unlock the electronic device. In response to receiving the instruction, the processing unit is configured to display (e.g., by the display unit 4217 and / or the display unit 4201) a home screen of the electronic device on the touch-sensitive display unit. The processing unit is also configured to provide (e.g., by the providing unit 4215) an indication to the user on the home screen that a particular action associated with the trigger condition is available.

[0047] (A41) In some embodiments of the electronic device described in A40, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion, displayed adjacent to the first portion, for launching a second set of applications available on the electronic device. The second portion is displayed over all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion (e.g., by display unit 4217 and / or display unit 4201).

[0048] (A42) In some embodiments of the electronic device according to A41, the second set of applications is different from and smaller than the first set of applications.

[0049] (A43) In some embodiments of the electronic device described in any one of A28 to A42, determining that at least one trigger condition is met includes determining (e.g., by the determining unit 4227) that the electronic device has been coupled to a second device different from the electronic device.

[0050] (A44) In some embodiments of the electronic device described in any one of A28 to A43, determining that at least one trigger condition is met includes determining (e.g., by the determining unit 4227) that the electronic device has arrived at an address corresponding to a home or work address associated with the user.

[0051] (A45) In some embodiments of the electronic device described in A44, determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user includes monitoring motion data from an accelerometer of the electronic device (e.g., by monitoring unit 4229) and determining (e.g., by determining unit 4227) based on the monitored motion data that the electronic device has not moved for more than a threshold amount of time.

[0052] (A46) In some embodiments of the electronic device described in any one of A28 to A45, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application continues to run. At least one trigger condition is further based on the verbal command provided to the virtual assistant application.

[0053] (A47) In some embodiments of the electronic device described in A46, the verbal command includes a request to create a reminder that corresponds to a current state of the application, the current state corresponding to the state of the application at the time the verbal command was provided.

[0054] (A48) In some embodiments of the electronic device described in A47, the state of the application when the verbal command is provided is selected from the group consisting of: a page displayed in the application when the verbal command is provided; content being played in the application when the verbal command is provided; a notification displayed in the application when the verbal command is provided; and an active portion of a page displayed in the application when the verbal command is provided.

[0055] (A49) In some embodiments of the electronic device described in A46, the verbal command includes the word "this" in reference to the current state of the application.

[0056] (B1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive display (touchscreen 112, FIG. 1C). The method includes obtaining at least one trigger condition based on usage data associated with a user of the electronic device, the usage data including one or more actions (or types of actions) performed by the user within an application while the application is running on the electronic device. The method also includes associating the at least one trigger condition with a particular action from the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that the particular action associated with the trigger condition is available.

[0057] (B2) In some embodiments of the method described in B1, the method further comprises the method described in any one of A2 to A22.

[0058] (B3) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in B1 or B2.

[0059] (B4) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for performing the method of B1 or B2.

[0060] (B5) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform a method as described in B1 or B2.

[0061] (B6) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to the method of B1 or B2.

[0062] (B7) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4201, FIG. 42), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4203, FIG. 42), and a processing unit (e.g., processing unit 4205, FIG. 42). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 42 shows display unit 4201 and touch-sensitive surface unit 4203 as being integrated into electronic device 4200, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an execution unit (e.g., execution unit 4207, FIG. 42), a collection unit (e.g., collection unit 4209, FIG. 42), an acquisition unit (e.g., acquisition unit 4211, FIG. 42), an association unit (e.g., association unit 4213, FIG. 42), a providing unit (e.g., providing unit 4215, FIG. 42), a sending unit (e.g., sending unit 4217, FIG. 42), a receiving unit (e.g., receiving unit 4219, FIG. 42), a display unit (e.g., display unit 4221, FIG. 42), a detection unit (e.g., detection unit 4223, FIG. 42), an implementation unit (e.g., implementation unit 4225, FIG. 42), a judgment unit (e.g., judgment unit 4227, FIG. 42), and a monitoring unit (e.g., monitoring unit 4229, FIG. 42).The processing unit (or one or more components thereof, such as units 4207 to 4229) is configured to: obtain (e.g., by the obtaining unit 4211) at least one trigger condition based on usage data associated with a user of the electronic device, the usage data including one or more actions performed by the user within the application while the application is running on the electronic device; associate (e.g., by the associating unit 4213) the at least one trigger condition with a particular action from the one or more actions performed by the user within the application; and, upon determining that the at least one trigger condition is satisfied, provide (e.g., by the providing unit 4215) an indication to the user that the particular action associated with the trigger condition is available.

[0063] (B8) In some embodiments of the electronic device described in B7, obtaining the at least one trigger condition includes transmitting usage data from the electronic device to one or more servers located remotely (e.g., by a transmitting unit 4217) and receiving the at least one trigger condition from the one or more servers (e.g., by a receiving unit 4219).

[0064] (B9) In some embodiments of the electronic device described in B7 or B8, providing the indication includes displaying (e.g., by display unit 4217 and / or display unit 4201) a user interface object corresponding to a particular action associated with the trigger condition on a lock screen of the touch-sensitive display.

[0065] (B10) In some embodiments of the electronic device described in B9, the user interface object includes a description of a particular action associated with the trigger condition.

[0066] (B11) In some embodiments of the electronic device described in B10, the user interface objects further include an icon associated with the application.

[0067] (B12) In some embodiments of the electronic device described in any one of B9 to B11, the processing unit is further configured to detect a first gesture at the user interface object (e.g., by the detection unit 4223 and / or the touch-sensitive surface unit 4203). In response to detecting the first gesture, (i) display an application on the touch-sensitive display (e.g., by the display unit 4217 and / or the display unit 4201), and (ii) perform a particular action associated with the trigger condition (e.g., by the implementation unit 4225) while displaying the application.

[0068] (B13) In some embodiments of the electronic device described in B12, the first gesture is a swipe gesture on a user interface object.

[0069] (B14) In some embodiments of the electronic device described in any one of B9 to B12, the processing unit is further configured to detect a second gesture at the user interface object (e.g., by detection unit 4223 and / or touch-sensitive surface unit 4203). In response to detecting the second gesture, and while continuing to display the lock screen on the touch-sensitive display, the processing unit is configured to perform (e.g., by implementation unit 4225) a particular action associated with the trigger condition.

[0070] (B15) In some embodiments of the electronic device described in B14, the second gesture is a single tap on a predefined area of ​​the user interface object.

[0071] (B16) In some embodiments of the electronic device of any one of B9 to B15, the user interface object is displayed in a default center portion of the lock screen.

[0072] (B17) In some embodiments of the electronic device described in B7, providing an indication to the user that a particular action associated with the trigger condition is available includes performing the particular action (e.g., by the implementation unit 4225).

[0073] (B18) In some embodiments of the electronic device described in B9, the user interface object is an icon associated with the application, and the user interface object is displayed substantially in a corner of the lock screen of the touch-sensitive display.

[0074] (B19) In some embodiments of the electronic device described in any one of B7 to B18, the processing unit is further configured to receive (e.g., by receiving unit 4219) an instruction from a user to unlock the electronic device. In response to receiving the instruction, the processing unit is configured to display (e.g., by display unit 4217 and / or display unit 4201) a home screen of the electronic device on the touch-sensitive display. The processing unit is also configured to provide (e.g., by providing unit 4215) an indication to the user on the home screen that a particular action associated with the trigger condition is available.

[0075] (B20) In some embodiments of the electronic device described in B19, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion, displayed adjacent to the first portion, for launching a second set of applications available on the electronic device, the second portion being displayed over all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion (e.g., by display unit 4217 and / or display unit 4201).

[0076] (B21) In some embodiments of the electronic device according to B20, the second set of applications is different from and smaller than the first set of applications.

[0077] (B22) In some embodiments of the electronic device described in any one of B7 to B21, determining that at least one trigger condition is met includes determining (e.g., by the determining unit 4227) that the electronic device has been coupled to a second device different from the electronic device.

[0078] (B23) In some embodiments of the electronic device described in any one of B7 to B22, determining that at least one trigger condition is met includes determining (e.g., by the determining unit 4227) that the electronic device has arrived at an address corresponding to a home or work address associated with the user.

[0079] (B24) In some embodiments of the electronic device described in B23, determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user includes monitoring motion data from an accelerometer of the electronic device (e.g., by the monitoring unit 4229) and determining (e.g., by the determining unit 4227) based on the monitored motion data that the electronic device has not moved for more than a threshold amount of time.

[0080] (B25) In some embodiments of the electronic device described in any one of B7 to B24, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application continues to run. At least one trigger condition is further based on the verbal command provided to the virtual assistant application.

[0081] (B26) In some embodiments of the electronic device described in B25, the verbal command includes a request to create a reminder that corresponds to a current state of the application, the current state corresponding to the state of the application at the time the verbal command was provided.

[0082] (B27) In some embodiments of the electronic device described in B26, the state of the application when the verbal command is provided is selected from the group consisting of a page displayed within the application when the verbal command is provided, content being played within the application when the verbal command is provided, a notification displayed within the application when the verbal command is provided, and an active portion of a page displayed within the application when the verbal command is provided.

[0083] (B28) In some embodiments of the electronic device described in B26, the spoken command includes the word "this" in reference to the current state of the application.

[0084] (C1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive display (touch screen 112, FIG. 1C). The method includes detecting a search activation gesture by a user of the electronic device on the touch-sensitive display. The method includes, in response to detecting the search activation gesture, displaying a search interface on the touch-sensitive display, the search interface including (i) a search entry portion and (ii) a prediction portion that is displayed before receiving any user input in the search entry portion. The prediction portion is populated with one or more of: (a) at least one affordance for contacting a person from a plurality of previously contacted people, where the person is automatically selected from the plurality of previously contacted people based, at least in part, on a current time; and (b) at least one affordance for performing a predicted action within an application from a plurality of applications available on the electronic device, where the predicted action is automatically selected based, at least in part, on an application usage history associated with a user of the electronic device.

[0085] (C2) In some embodiments of the method of C1, the person is further selected based at least in part on location data corresponding to the electronic device.

[0086] (C3) In some embodiments of the method described in C1 or C2, application usage history and contact information for the person are retrieved from the memory of the electronic device.

[0087] (C4) In some embodiments of the method described in C1 or C2, the application usage history and contact information for the person are retrieved from a server located remotely from the electronic device.

[0088] (C5) In some embodiments of the methods described in any one of C1 to C4, the prediction portion is further populated with at least one affordance for executing a predicted application, and the predicted application is automatically selected based, at least in part, on application usage history.

[0089] (C6) In some embodiments of the method of any one of C1 to C5, the prediction portion is further populated with at least one affordance of a predicted place (or nearby place) category, the predicted place category being automatically selected based at least in part on one or more of the current time and location data corresponding to the electronic device.

[0090] (C7) In some embodiments of the method of any one of C1 to C6, the method further includes detecting a user input to scroll the predicted portion. The method includes, in response to detecting the user input to scroll the predicted portion, scrolling the predicted portion according to the user input. The method includes, in response to scrolling, revealing at least one affordance of a predicted news article in the predicted portion (e.g., the predicted news article is one that is expected to be of interest to the user).

[0091] (C8) In some embodiments of the method described in C7, the predicted news stories are automatically selected based at least in part on location data corresponding to the electronic device.

[0092] (C9) In some embodiments of the method of any one of C1 to C8, the method further includes detecting a selection of at least one affordance for performing a predicted action within the application, the method including, in response to detecting the selection, displaying the application on the touch-sensitive display and performing the predicted action within the application.

[0093] (C10) In some embodiments of the method described in C3 or C4, the method further includes detecting a selection of at least one affordance for contacting the person, the method including, in response to detecting the selection, contacting the person using contact information for the person.

[0094] (C11) In some embodiments of the method described in C5, the method further includes detecting a selection of at least one affordance for executing the predicted application, the method including, in response to detecting the selection, displaying the predicted application on the touch-sensitive display.

[0095] (C12) In some embodiments of the method described in C6, the method further includes detecting a selection of at least one affordance of the predicted place category, and in response to detecting the selection, (i) receiving data corresponding to the at least one nearby place, and (ii) displaying the received data corresponding to the at least one nearby place on the touch-sensitive display.

[0096] (C13) In some embodiments of the method described in C7, the method further includes detecting a selection of at least one affordance of the predicted news article, and in response to detecting the selection, displaying the predicted news article on the touch-sensitive display.

[0097] (C14) In some embodiments of the methods described in any one of C1 to C13, the search activation gesture is available from at least two different user interfaces, and a first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of the series of home screen pages on the touch-sensitive display.

[0098] (C15) In some embodiments of the method described in C14, when the corresponding home screen page is the first home screen page of a series of home screen pages, the search activation gesture includes one of the following: (i) a gesture moving substantially downward relative to a user of the electronic device, or (ii) a continuous gesture moving substantially in a left-to-right direction relative to the user and substantially perpendicular to the downward direction.

[0099] (C16) In some embodiments of the method described in C15, when the corresponding home screen page is a second home screen page of a series of home screen pages, the search activation gesture includes a continuous gesture moving substantially downward relative to a user of the electronic device.

[0100] (C17) In some embodiments of the method described in C14, the second user interface of the at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display.

[0101] (C18) In some embodiments of the method described in C17, the search activation gesture comprises contact on a touch-sensitive display at a default search activation portion of the application switching interface.

[0102] (C19) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of C1 to C18.

[0103] (C20) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for performing the method of any one of C1 to C18.

[0104] (C21) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform a method according to any one of C1 to C18.

[0105] (C22) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of C1 to C18.

[0106] (C23) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4301, FIG. 43), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4303, FIG. 43), and a processing unit (e.g., processing unit 4305, FIG. 43). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 43 shows display unit 4301 and touch-sensitive surface unit 4303 as being integrated into electronic device 4300, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a display unit (e.g., display unit 4309, FIG. 43), a detection unit (e.g., detection unit 4307, FIG. 43), a retrieval unit (e.g., retrieval unit 4311, FIG. 43), a populating unit (e.g., populating unit 4313, FIG. 43), a scrolling unit (e.g., scrolling unit 4315, FIG. 43), a revealing unit (e.g., revealing unit 4317, FIG. 43), a selection unit (e.g., selection unit 4319, FIG. 43), a communication unit (e.g., communication unit 4321, FIG. 43), a receiving unit (e.g., receiving unit 4323, FIG. 43), and an execution unit (e.g., execution unit 4325, FIG. 43). The processing unit (or one or more components thereof, such as units 4307 to 4225) is configured to detect a search activation gesture of a user of the electronic device on the touch-sensitive display (e.g., by detection unit 4307 and / or touch-sensitive surface unit 4303), and in response to detecting the search activation gesture, display a search interface on the touch-sensitive display (e.g., by display unit 4309 and / or display unit 4301), the search interface including (i) a search entry portion and (ii) a prediction portion that is displayed before receiving any user input in the search entry portion.The prediction portion is populated with one or more of: (a) at least one affordance for contacting a person from a plurality of previously contacted people, where the person is automatically selected (e.g., by the selection unit 4319) from among the plurality of previously contacted people based at least in part on the current time; and (b) at least one affordance for performing a predicted action within an application from a plurality of applications available on the electronic device, where the predicted action is automatically selected (e.g., by the selection unit 4319) based at least in part on an application usage history associated with a user of the electronic device.

[0107] (C24) In some embodiments of the electronic device described in C23, the person is further selected (eg, by the selection unit 4319) based at least in part on location data corresponding to the electronic device.

[0108] (C25) In some embodiments of the electronic device described in C23 or C24, the application usage history and contact information for the person are retrieved from the memory of the electronic device (eg, by retrieval unit 4311).

[0109] (C26) In some embodiments of the electronic device described in C23 or C24, the application usage history and contact information for the person are retrieved (eg, by retrieval unit 4311) from a server located remotely from the electronic device.

[0110] (C27) In some embodiments of the electronic device described in any one of C23 to C26, the predicted portion is further populated (e.g., by a populating unit 4313) with at least one affordance for executing the predicted application, and the predicted application is automatically selected (e.g., by a selection unit 4319) based at least in part on application usage history.

[0111] (C28) In some embodiments of the electronic device described in any one of C23 to C27, the predicted portion is further populated (e.g., by a populating unit 4313) with at least one affordance of a predicted place category, and the predicted place category is automatically selected (e.g., by a selection unit 4319) based at least in part on one or more of the current time and location data corresponding to the electronic device.

[0112] (C29) In some embodiments of the electronic device described in any one of C23 to C28, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a user input to scroll the predicted portion. In response to detecting the user input to scroll the predicted portion, the processing unit is configured to scroll (e.g., by the scrolling unit 4319) the predicted portion according to the user input. In response to scrolling, the processing unit is configured to reveal (e.g., by the revealing unit 4317) at least one affordance of a predicted news article in the predicted portion (e.g., the predicted news article is one that is expected to be of interest to the user).

[0113] (C30) In some embodiments of the electronic device described in C7, the predicted news articles are automatically selected (eg, by the selection unit 4319) based at least in part on location data corresponding to the electronic device.

[0114] (C31) In some embodiments of the electronic device described in any one of C23 to C30, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance for performing a predicted action within the application. In response to detecting the selection, the processing unit is configured to display (e.g., by the display unit 4309) the application on the touch-sensitive display (e.g., the display unit 4301) and perform (e.g., by the execution unit 4325) the predicted action within the application.

[0115] (C32) In some embodiments of the electronic device described in C25 or C26, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance for contacting the person. In response to detecting the selection, the processing unit is configured to contact the person (e.g., by the contact unit 4321) using contact information for the person.

[0116] (C33) In some embodiments of the electronic device described in C27, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance for executing the predicted application. In response to detecting the selection, the processing unit is configured to display (e.g., by the display unit 4307) the predicted application on the touch-sensitive display (e.g., by the display unit 4301).

[0117] (C34) In some embodiments of the electronic device described in C28, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance of the predicted location category. In response to detecting the selection, the processing unit is configured to (i) receive (e.g., by the receiving unit 4323) data corresponding to the at least one nearby location, and (ii) display (e.g., by the display unit 4307) the received data corresponding to the at least one nearby location on the touch-sensitive display (e.g., the display unit 4301).

[0118] (C35) In some embodiments of the electronic device described in C29, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance of the predicted news article. In response to detecting the selection, the processing unit is configured to display (e.g., by the display unit 4307) the predicted news article on the touch-sensitive display (e.g., the display unit 4301).

[0119] (C36) In some embodiments of the electronic device described in any one of C23 to C35, the search activation gesture is available from at least two different user interfaces, and a first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of a series of home screen pages on the touch-sensitive display.

[0120] (C37) In some embodiments of the electronic device described in C36, if the corresponding home screen page is the first home screen page of a series of home screen pages, the search activation gesture includes one of the following: (i) a gesture moving substantially downward relative to a user of the electronic device, or (ii) a continuous gesture moving substantially in a left-to-right direction relative to the user and substantially perpendicular to the downward direction.

[0121] (C38) In some embodiments of the electronic device described in C37, if the corresponding home screen page is a second home screen page of a series of home screen pages, the search activation gesture includes a continuous gesture moving substantially downward relative to a user of the electronic device.

[0122] (C39) In some embodiments of the electronic device described in C36, a second user interface of the at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display.

[0123] (C40) In some embodiments of the electronic device described in C39, the search activation gesture comprises contact on the touch-sensitive display at a default search activation portion of the application switching interface.

[0124] Thus, electronic devices having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface are provided with faster and more efficient methods and interfaces for proactively accessing applications and proactively executing functions within applications, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may complement or replace conventional methods for accessing applications and the functionality associated with such methods.

[0125] (D1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A through D ( FIG. 1E )) having a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ) and a display (e.g., display 194, FIG. 1D ). The method includes displaying content associated with an application running on the electronic device on the display. The method further includes detecting a swipe gesture via the touch-sensitive surface that, when detected, causes the electronic device to transition to a search mode distinct from the application. The method also includes, in response to detecting the swipe gesture, transitioning to a search mode, the search mode including a search interface displayed on the display. In conjunction with transitioning to the search mode, the method includes determining at least one suggested search query based, at least in part, on information associated with the content. The method includes populating the displayed search interface with the at least one suggested search query before receiving any user input in the search interface. In this way, instead of having to remember and re-enter information into a search interface, the device provides the user with relevant suggestions based on the application content the user is viewing, and the user simply needs to select one of the suggestions without having to type anything.

[0126] (D2) In some embodiments of the method described in D1, detecting the swipe gesture includes detecting the swipe gesture over at least a portion of the currently displayed content.

[0127] (D3) In some embodiments of the method described in D1 or D2, the method further includes, before detecting the swipe gesture, detecting an input corresponding to a request to view a home screen of the electronic device, and in response to detecting the input, ceasing to display content associated with the application and displaying a corresponding page of the home screen of the electronic device. In some embodiments, the corresponding page is an initial page of a series of home screen pages, and the swipe gesture is detected while the initial page of the home screen is displayed on the display.

[0128] (D4) In some embodiments of the method according to any one of D1 to D3, the search interface is displayed semi-transparently overlaid on the application.

[0129] (D5) In some embodiments of the method of any one of D1 to D4, the method further includes, pursuant to a determination that the content includes textual content, determining at least one suggested search query based, at least in part, on the textual content.

[0130] (D6) In some embodiments of the method described in D5, determining the at least one suggested search query based at least in part on the textual content includes analyzing the textual content to detect one or more defined keywords used to determine the at least one suggested search query.

[0131] (D7) In some embodiments of the method described in any one of D1 to D6, determining at least one suggested search query includes determining a plurality of suggested search queries, and populating the search interface includes populating the search interface with the plurality of suggested search queries.

[0132] (D8) In some embodiments of the method of any one of D1 to D7, the method further includes detecting, via the touch-sensitive surface, a new swipe gesture on currently displayed new content; and in response to detecting the new swipe gesture, transitioning to a search mode, where transitioning to the search mode includes displaying a search interface on the display; and in conjunction with transitioning to the search mode and in accordance with determining that the new content does not include textual content, populating the search interface with suggested search queries based on a set of selected past search queries by a user of the electronic device.

[0133] (D9) In some embodiments of the method described in D8, the search interface is displayed with points of interest based on location information provided by a second application different from the application.

[0134] (D10) In some embodiments of the method according to any one of D8 or D9, the search interface further includes one or more suggested applications.

[0135] (D11) In some embodiments of the method of any one of D8 to D10, the set of past search queries is selected based at least in part on the frequency of recent search queries.

[0136] (D12) In some embodiments of the method of any one of D1 to D11, the method further includes, in conjunction with entering a search mode, obtaining information associated with the content by using one or more accessibility features available on the electronic device.

[0137] (D13) In some embodiments of the method described in D12, using the one or more accessibility features includes generating information associated with the content using the one or more accessibility features by (i) applying a natural language processing algorithm to text content currently displayed within the application, and (ii) using data obtained from the natural language processing algorithm to determine one or more keywords that describe the content, wherein at least one suggested search query is determined based on the one or more keywords.

[0138] (D14) In some embodiments of the method described in D13, determining one or more keywords that describe the content also includes (i) retrieving metadata that corresponds to non-text content currently displayed within the application, and (ii) determining the one or more keywords using the retrieved metadata in addition to data obtained from the natural language processing algorithm.

[0139] (D15) In some embodiments of the method according to any one of D1 to D14, the search interface further includes one or more trend queries.

[0140] (D16) In some embodiments of the method described in D15, the search interface further includes one or more applications that are expected to be of interest to a user of the electronic device.

[0141] (D17) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of D1 to D16.

[0142] (D18) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing a method according to any one of D1 to D16.

[0143] (D19) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of D1 to D16.

[0144] (D20) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of D1 to D16. In a further aspect, an information processing apparatus is provided for use in an electronic device including a touch-sensitive surface and a display. The information processing apparatus comprises means for performing a method described in any one of D1 to D16.

[0145] (D21) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4401, FIG. 44), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4403, FIG. 44), and a processing unit (e.g., processing unit 4405, FIG. 44). The processing unit is coupled to the touch-sensitive surface unit and the display unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 44 shows display unit 4401 and touch-sensitive surface unit 4403 as being integrated into electronic device 4400, although in some embodiments, one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into a single touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a detection unit (e.g., detection unit 4407, FIG. 44), a display unit (e.g., display unit 4409, FIG. 44), an extraction unit (e.g., extraction unit 4411, FIG. 44), a search mode transition unit (e.g., search mode transition unit 4412, FIG. 44), a populating unit (e.g., populating unit 4413, FIG. 44), an acquisition unit (e.g., acquisition unit 4415, FIG. 44), a judgment unit (e.g., judgment unit 4417, FIG. 44), and a selection unit (e.g., selection unit 4419, FIG. 44).The processing unit (or one or more components thereof, such as units 1007 to 1029) is configured to: display (e.g., by display unit 4407) on a display unit (e.g., display unit 4407) content associated with an application running on the electronic device; detect (e.g., by detection unit 4407) via the touch-sensitive surface unit (e.g., touch-sensitive surface unit 4403) a swipe gesture that, when detected, causes the electronic device to transition to a search mode distinct from the application; in response to detecting the swipe gesture, transition (e.g., by search mode transition unit 4412) to a search mode including a search interface displayed on the display unit (e.g., display unit 4407); and, in conjunction with transitioning to the search mode, determine (e.g., by determining unit 4417) at least one suggested search query based, at least in part, on information associated with the content; and populate (e.g., by populating unit 4413) the displayed search interface with the at least one suggested search query before receiving any user input in the search interface.

[0146] (D22) In some embodiments of the electronic device described in D21, detecting the swipe gesture includes detecting (e.g., by the detection unit 4407) the swipe gesture over at least a portion of the currently displayed content.

[0147] (D23) In some embodiments of the electronic device described in D21 or D22, the processing unit is further configured to detect (e.g., by the detection unit 4407) an input corresponding to a request to view a home screen of the electronic device before detecting the swipe gesture, and in response to detecting the input (e.g., by the detection unit 4407), stop displaying content associated with the application and display (e.g., by the display unit 4409) a corresponding page of the home screen of the electronic device, the corresponding page being an initial page of a series of home screen pages, and the swipe gesture is detected (e.g., by the detection unit 4407) while the initial page of the home screen is displayed on the display unit.

[0148] (D24) In some embodiments of the electronic device according to any one of D21 to D23, the search interface is displayed semi-transparently overlaid on the application (e.g., on the display unit 4409 and / or the display unit 4401).

[0149] (D25) In some embodiments of the electronic device described in any one of D21 to D24, the processing unit is further configured to determine (e.g., by the determining unit 4417) at least one suggested search query based, at least in part, on the text content in accordance with a determination that the content includes text content.

[0150] (D26) In some embodiments of the electronic device described in D25, determining the at least one suggested search query based at least in part on the textual content includes analyzing the textual content to detect (e.g., by the detection unit 4407) one or more predefined keywords used to determine (e.g., by the determination unit 4417) the at least one suggested search query.

[0151] (D27) In some embodiments of the electronic device described in any one of D21 to D26, determining at least one suggested search query includes determining a plurality of suggested search queries (e.g., by the determining unit 4417), and populating the search interface includes populating the search interface with the plurality of suggested search queries (e.g., by the populating unit 4413).

[0152] (D28) In some embodiments of the electronic device described in any one of D21 to D27, the processing unit is further configured to detect (e.g., by detection unit 4407) a new swipe gesture on the currently displayed new content via the touch-sensitive surface unit (e.g., by touch-sensitive surface unit 4403), and in response to detecting the new swipe gesture, transition to a search mode (e.g., by search mode transition unit 4412), wherein transitioning to the search mode includes displaying a search interface on the display unit (e.g., by display unit 4409), and in conjunction with transitioning to the search mode and in accordance with determining that the new content does not include text content, populate (e.g., by populating unit 4413) the search interface with suggested search queries based on a set of past search queries selected by a user of the electronic device.

[0153] (D29) In some embodiments of the electronic device described in D28, points of interest are displayed in the search interface based on location information provided by a second application different from the application (e.g., display unit 4409).

[0154] (D30) In some embodiments of the electronic device described in D28 or D29, the search interface further comprises one or more suggested applications.

[0155] (D31) In some embodiments of the electronic device described in any one of D28 to D30, the set of past search queries is selected (e.g., by the selection unit 4419) based at least in part on the frequency of recent search queries.

[0156] (D32) In some embodiments of the electronic device described in any one of D21 to D31, the processing unit is further configured to, in conjunction with entering the search mode, obtain (e.g., by the obtaining unit 4415) information associated with the content by using one or more accessibility features available on the electronic device.

[0157] (D33) In some embodiments of the electronic device described in D32, using the one or more accessibility features includes (i) applying a natural language processing algorithm to text content currently displayed within the application, and (ii) using data obtained (e.g., by obtaining unit 4415) from the natural language processing algorithm to determine (e.g., by determining unit 4417) one or more keywords that describe the content, wherein at least one suggested search query is determined (e.g., by determining unit 4417) based on the one or more keywords, thereby generating information associated with the content using the one or more accessibility features.

[0158] (D34) In some embodiments of the electronic device described in D33, determining one or more keywords that describe the content also includes (i) retrieving metadata corresponding to non-text content currently displayed within the application (e.g., by retrieval unit 4411), and (ii) determining one or more keywords using the retrieved metadata in addition to data obtained from the natural language processing algorithm (e.g., by determination unit 4417).

[0159] (D35) In some embodiments of the electronic device of any one of D21 to D34, the search interface further comprises one or more trend queries.

[0160] (D36) In some embodiments of the electronic device described in D35, the search interface further includes one or more applications expected to be of interest to a user of the electronic device.

[0161] (E1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ) and a display (e.g., display 194, FIG. 1D ). The method includes detecting a swipe gesture on a user interface via the touch-sensitive surface, the swipe gesture causing the electronic device to transition to a search mode upon detection of the swipe gesture. The method further includes transitioning to the search mode in response to detecting the swipe gesture, wherein transitioning to the search mode includes populating a search interface, distinct from the user interface, with the first content item before receiving any user input in the search interface. In some embodiments, populating the search interface with the first content item in accordance with a determination that the user interface includes content associated with an application different from a home screen that includes a selectable icon for invoking the application includes populating the search interface with at least one suggested search query based, at least in part, on content associated with the application, and populating the search interface with the first content item in accordance with a determination that the user interface is associated with a home screen page includes populating the search interface with affordances that include selectable descriptions of at least one point of interest within a threshold distance of a current location of the electronic device.

[0162] (E2) In some embodiments of the method described in E1, populating the search interface with affordances includes displaying a search entry portion of the search interface on the touch-sensitive surface, the method further including detecting input in the search entry portion, and in response to detecting input in the search entry portion, ceasing to display the affordances and displaying at least one suggested search query in the search interface.

[0163] (E3) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform the method described in E1 or E2.

[0164] (E4) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method of E1 or E2.

[0165] (E5) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in E1 or E2.

[0166] (E6) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in E1 or E2. In a further aspect, an information processing apparatus is provided for use in an electronic device including a touch-sensitive surface and a display. The information processing apparatus comprises means for performing the method described in E1 or E2.

[0167] (E7) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4501, FIG. 45), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4503, FIG. 45), and a processing unit (e.g., processing unit 4505, FIG. 45). The processing unit is coupled to the touch-sensitive surface unit and the display unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 45 shows display unit 4501 and touch-sensitive surface unit 4503 as being integrated into electronic device 4500, although in some embodiments, one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into a single touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a detection unit (e.g., detection unit 4507, FIG. 45), a display unit (e.g., display unit 4509, FIG. 45), a populating unit (e.g., populating unit 4511, FIG. 45), and a search mode transition unit (e.g., search mode transition unit 4513, FIG. 45). The processing unit (or one or more components thereof, such as units 4507 to 4513) is configured to detect (e.g., by detection unit 4507) a swipe gesture on the user interface via the touch-sensitive surface unit (e.g., touch-sensitive surface unit 4503), which swipe gesture causes the electronic device to transition to a search mode when the swipe gesture is detected, and to transition to the search mode (e.g., by search mode transition unit 4513) in response to detecting the swipe gesture, wherein transitioning to the search mode includes populating (e.g., by populating unit 4511) a search interface different from the user interface with a first content item before receiving any user input in the search interface.In some embodiments, populating the search interface with the first content item in accordance with a determination that the user interface includes content associated with an application different from a home screen that includes a selectable icon for invoking the application includes populating the search interface with at least one suggested search query based, at least in part, on content associated with the application (e.g., by the populating unit 4511), and populating the search interface with the first content item in accordance with a determination that the user interface is associated with a home screen page includes populating the search interface with affordances that include selectable descriptions of at least one point of interest within a threshold distance of a current location of the electronic device (e.g., by the populating unit 4511).

[0168] (E8) In some embodiments of the electronic device described in E7, populating the search interface with affordances includes displaying a search entry portion of the search interface (e.g., by the display unit 4507 and / or the display unit 4501), and the processing unit is further configured to detect input in the search entry portion (e.g., by the detection unit 4507), and in response to detecting the input in the search entry portion, stop displaying the affordances (e.g., by the display unit 4507 and / or the display unit 4501), and display at least one suggested search query in the search interface (e.g., by the display unit 4507 and / or the display unit 4501).

[0169] (F1) In some embodiments, a method is performed in an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A through D ( FIG. 1E )) having a position sensor, a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ), and a display (e.g., display 194, FIG. 1D ). The method includes automatically and without instructions from a user determining that a user of the electronic device is in a stopped vehicle at a geographic location, and upon determining that the user has left the vehicle at the geographic location, determining whether location information retrieved from the position sensor to identify the geographic location meets an accuracy criterion. The method further includes, upon determining that the location information does not meet the accuracy criterion, providing a prompt to the user to input information about the geographic location. The method also includes, in response to providing the prompt, receiving information about the geographic location from the user and storing the information as vehicle location information.

[0170] (F2) In some embodiments of the method described in F1, the method further includes, upon determining that the location information meets the accuracy criteria, automatically and without instruction from a user, storing the location information as vehicle location information.

[0171] (F3) In some embodiments of the method of claim F2, the method further includes, in accordance with a determination that the user is heading to the geographic location, displaying a user interface object that includes vehicle location information.

[0172] (F4) In some embodiments of the method of claim F3, the user interface object is a map object that includes an identifier for the user's current location and another identifier for the geographic location.

[0173] (F5) In some embodiments of the method of claim F4, the user interface object is displayed on a lock screen of the electronic device.

[0174] (F6) In some embodiments of the method of claim F4, the user interface object is displayed in response to a swipe gesture that causes the electronic device to enter a search mode.

[0175] (F7) In some embodiments of the method of claim F6, determining whether the user is heading towards the geographic location is performed in response to receiving a swipe gesture.

[0176] (F8) In some embodiments of the methods described in any one of F1 to F7, the prompt is an audio prompt provided by a virtual assistant available via the electronic device, and receiving information from the user includes receiving a verbal description from the user that identifies the geographic location, and displaying the user interface object includes displaying a selectable affordance that, when selected, causes the device to play the verbal description.

[0177] (F9) In some embodiments of the methods described in any one of F1 to F7, the prompt is displayed on a display of the electronic device, receiving information from the user includes receiving a text description from the user that identifies the geographic location, and displaying the user interface object includes displaying the text description from the user.

[0178] (F10) In some embodiments of the methods described in any one of F1 to F7, determining whether the user is moving towards the geographic location includes determining that the electronic device is moving towards the geographic location using new location information received from the location sensor.

[0179] (F11) In some embodiments of the method described in F10, determining whether the user is heading towards the geographic location includes (i) determining that the electronic device remains at the different geographic location for more than a threshold time, and (ii) determining that new location information indicates that the electronic device is leaving the different geographic location and heading towards the geographic location.

[0180] (F12) In some embodiments of the method of any one of F1 to F11, determining that the user is in a vehicle stopped at the geographic location includes (i) determining that the user is in the vehicle by determining that the electronic device is moving above a threshold speed; and (ii) determining that the vehicle has stopped at the geographic location by one or more of: (a) determining that the electronic device remains at the geographic location for more than a threshold time; (b) determining that a communication link between the electronic device and the vehicle has been disconnected; and (c) determining that the geographic location corresponds to a location in a parking lot.

[0181] (F13) In some embodiments of the method of claim F12, determining that the vehicle has stopped at the geographic location includes determining that the electronic device has remained at the geographic location for more than a threshold time.

[0182] (F14) In some embodiments of the method described in F12 or F13, determining that the vehicle has stopped at the geographic location includes determining that a communication link between the electronic device and the vehicle has been disconnected.

[0183] (F15) In some embodiments of the method according to any one of F12 to F14, determining that the vehicle has stopped at the geographic location includes determining that the geographic location corresponds to a location within a parking lot.

[0184] (F16) In some embodiments of the method according to any one of F1 to F15, the accuracy criterion comprises a criterion that is met if the accuracy of the GPS readings associated with the location information exceeds a threshold level of accuracy.

[0185] (F17) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, a position sensor, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of F1 to F16.

[0186] (F18) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, a position sensor, and means for performing the method of any one of F1 to F16.

[0187] (F19) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface, a display, and a position sensor, cause the electronic device to perform a method according to any one of F1 to F16.

[0188] (F20) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface, a display, and a position sensor is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method of any one of F1 to F16. In a further aspect, an information processing apparatus is provided for use in an electronic device including a touch-sensitive surface, a display, and a position sensor. The information processing apparatus comprises means for performing the method of any one of F1 to F16.

[0189] (F21) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4601, FIG. 46), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4603, FIG. 46), a position sensor unit (e.g., position sensor unit 4607, FIG. 46), and a processing unit (e.g., processing unit 4605, FIG. 46). The processing unit is coupled to the touch-sensitive surface unit, the display unit, and the position sensor unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 46 depicts display unit 4601 and touch-sensitive surface unit 4603 as being integrated into electronic device 4600, although in some embodiments, one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into a single touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a display unit (e.g., display unit 4609, FIG. 46), an extraction unit (e.g., extraction unit 4611, FIG. 46), a judgment unit (e.g., judgment unit 4613, FIG. 46), a storage unit (e.g., storage unit 4615, FIG. 46), an identification unit (e.g., identification unit 4617, FIG. 46), a selection unit (e.g., selection unit 4619, FIG. 46), a receiving unit (e.g., receiving unit 4621, FIG. 46), a providing unit (e.g., providing unit 4623, FIG. 46), and a playback unit (e.g., playback unit 4625, FIG. 46).The processing unit (or one or more components thereof, such as units 4607 to 4625) is configured to automatically and without instructions from the user determine (e.g., by the determining unit 4613) that a user of the electronic device is in a vehicle stopped at the geographical location, and when it determines that the user has left the vehicle at the geographical location, determine (e.g., by the determining unit 4613) whether location information retrieved (e.g., by the retrieving unit 4611) from the location sensor unit (e.g., the location sensor unit 4607) satisfies an accuracy criterion to identify the geographical location (e.g., by the identifying unit 4617), and when it determines (e.g., by the determining unit 4613) that the location information does not satisfy the accuracy criterion, provide (e.g., by the providing unit 4623) a prompt to the user to input information about the geographical location, and in response to providing the prompt, receive (e.g., by the receiving unit 4621) information about the geographical location from the user and store (e.g., by the storage unit 4615) this information as vehicle location information.

[0190] (F22) In some embodiments of the electronic device described in F21, the processing unit is further configured to, upon determining that the location information meets the accuracy criteria, automatically and without instruction from the user, store the location information as vehicle location information (e.g., by the storage unit 4615).

[0191] (F23) In some embodiments of the electronic device described in F22, the processing unit is further configured to display (e.g., by the display unit 4609 in conjunction with the display unit 4601) a user interface object including vehicle location information in accordance with a determination that the user is heading to the geographic location.

[0192] (F24) In some embodiments of the electronic device described in F23, the user interface object is a map object that includes an identifier for the user's current location and another identifier for the geographic location.

[0193] (F25) In some embodiments of the electronic device described in F24, the user interface object is displayed on a lock screen of the electronic device (eg, by display unit 4609 in conjunction with display unit 4601).

[0194] (F26) In some embodiments of the electronic device described in F24, the user interface object is displayed (e.g., by display unit 4609 in conjunction with display unit 4601) in response to a swipe gesture that causes the electronic device to enter a search mode.

[0195] (F27) In some embodiments of the electronic device described in F26, determining whether the user is heading towards the geographic location is performed in response to receiving a swipe gesture.

[0196] (F28) In some embodiments of the electronic device described in any one of F21 to F27, the prompt is an audio prompt provided by a virtual assistant available via the electronic device, and receiving information from the user includes receiving (e.g., by the receiving unit 4621) a verbal description from the user that identifies the geographic location, and displaying the user interface object includes displaying (e.g., by the display unit 4609 in conjunction with the display unit 4601) a selectable affordance that, when selected (e.g., by the selection unit 4619), causes the device to play (e.g., by the playback unit 4625) the verbal description.

[0197] (F29) In some embodiments of the electronic device described in any one of F21 to F27, the prompt is displayed on a display of the electronic device (e.g., by a display unit 4609 in conjunction with a display unit 4601), receiving information from the user includes receiving a text description from the user that identifies the geographic location (e.g., by a receiving unit 4621), and displaying the user interface object includes displaying the text description from the user.

[0198] (F30) In some embodiments of the electronic device described in any one of F21 to F27, determining whether the user is moving towards the geographic location includes determining (e.g., by the determining unit 4613) that the electronic device is moving towards the geographic location using new location information received (e.g., by the receiving unit 4621) from a location sensor unit (e.g., the location sensor unit 4607).

[0199] (F31) In some embodiments of the electronic device described in F30, determining whether the user is heading towards the geographic location includes (i) determining (e.g., by determining unit 4613) that the electronic device has remained at a different geographic location for more than a threshold time, and (ii) determining (e.g., by determining unit 4613) that new location information indicates that the electronic device is leaving the different geographic location and heading towards the geographic location.

[0200] (F32) In some embodiments of the electronic device described in any one of F21 to F31, determining that the user is in a vehicle stopped at the geographical location includes: (i) determining that the user is in the vehicle by determining (e.g., by the determining unit 4613) that the electronic device is moving at above a threshold speed; and (ii) determining that the vehicle has stopped at the geographical location by one or more of: (a) determining (e.g., by the determining unit 4613) that the electronic device remains at the geographical location for more than a threshold time; (b) determining that a communication link between the electronic device and the vehicle has been disconnected (e.g., by the determining unit 4613); and (c) determining that the geographical location corresponds to a location in a parking lot (e.g., by the determining unit 4613).

[0201] (F33) In some embodiments of the electronic device described in F32, determining that the vehicle has stopped at the geographic location includes determining (e.g., by the determining unit 4613) that the electronic device has remained at the geographic location for more than a threshold time.

[0202] (F34) In some embodiments of the electronic device described in F32 or F33, determining that the vehicle has stopped at the geographic location includes determining (e.g., by determining unit 4613) that a communication link between the electronic device and the vehicle has been disconnected.

[0203] (F35) In some embodiments of the electronic device described in any one of F32 to F34, determining that the vehicle has stopped at the geographic location includes determining (e.g., by the determining unit 4613) that the geographic location corresponds to a location in a parking lot.

[0204] (F36) In some embodiments of the electronic device described in any one of F21 to F35, the accuracy criterion includes a criterion that is met if the accuracy of the GPS readings associated with the location information exceeds a threshold level of accuracy.

[0205] (G1) In some embodiments, a method is performed in an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A through D ( FIG. 1E )) having a position sensor, a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ), and a display (e.g., display 194, FIG. 1D ). The method includes monitoring a geographic location of the electronic device using the position sensor. The method further includes determining, based on the monitored geographic location, that the electronic device is within a threshold distance of a predetermined type of point of interest. The method also includes, in accordance with determining that the electronic device is within the threshold distance of the point of interest, identifying at least one activity that is currently popular at the point of interest and retrieving information about the point of interest, and retrieving information about the at least one activity that is currently popular at the point of interest. The method further includes detecting, via the touch-sensitive surface, a first input that, when detected, causes the electronic device to transition to a search mode; and in response to detecting the first input, transitioning to the search mode, wherein transitioning to the search mode includes presenting, via the display, before receiving any user input at the search interface, an affordance including: (i) information regarding at least one activity; and (ii) an indication that the at least one activity has been identified as currently popular in the location of interest.

[0206] (G2) In some embodiments of the method of claim G1, the method includes detecting a second input; and, in response to detecting the second input, updating the affordance to include available information about a current activity at a second point of interest that is different from the point of interest, the point of interest being within a threshold distance of the electronic device.

[0207] (G3) In some embodiments of the method described in G1 or G2, the affordance further includes a selectable category of points of interest, the method further including detecting a selection of the corresponding selectable category, and in response to detecting the selection, updating the affordance to include information about additional points of interest located within a second threshold distance of the device.

[0208] (G4) In some embodiments of the method of any one of G1 to G3, the point of interest is an amusement park and the retrieved information includes current wait times for rides at the amusement park.

[0209] (G5) In some embodiments of the method of claim G4, the retrieved information includes information regarding waiting times for rides located within a predetermined distance of the electronic device.

[0210] (G6) In some embodiments of the method of any one of G1 to G3, the point of interest is a restaurant and the retrieved information includes information about popular menu items at the restaurant.

[0211] (G7) In some embodiments of the method of claim G6, the retrieved information is retrieved from a social network associated with the user of the electronic device.

[0212] (G8) In some embodiments of the method according to any one of G1 to G3, the point of interest is a movie theater, and the retrieved information includes information about movie theater showtimes.

[0213] (G9) In some embodiments of the method of claim G8, the retrieved information is retrieved from a social network associated with the user of the electronic device.

[0214] (G10) In some embodiments of the methods described in any one of G1 to G9, after the electronic device is unlocked, the affordance is available in response to a substantially horizontal swipe on an initial page of a home screen of the electronic device.

[0215] (G11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, a position sensor, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of G1 to G10.

[0216] (G12) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, a position sensor, and means for performing the method of any one of G1 to G10.

[0217] (G13) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface, a display, and a position sensor, cause the electronic device to perform a method according to any one of G1 to G10.

[0218] (G14) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface, a display, and a position sensor is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of G1 to G10. In a further aspect, an information processing apparatus is provided for use in an electronic device including a touch-sensitive surface, a display, and a position sensor. The information processing apparatus comprises means for performing a method described in any one of G1 to G10.

[0219] (G15) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4701, FIG. 47), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4703, FIG. 47), a position sensor unit 4707, and a processing unit (e.g., processing unit 4705, FIG. 47). The processing unit is coupled to the touch-sensitive surface unit, the display unit, and the position sensor unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 47 shows display unit 4701 and touch-sensitive surface unit 4703 as being integrated into electronic device 4700, although in some embodiments, one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into a single touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a detection unit (e.g., detection unit 4709, FIG. 47), a display unit (e.g., display unit 4711, FIG. 47), an extraction unit (e.g., extraction unit 4713, FIG. 47), a judgment unit (e.g., judgment unit 4715, FIG. 47), an identification unit (e.g., identification unit 4717, FIG. 47), an unlocking unit (e.g., unlocking unit 4719, FIG. 47), and a search mode transition unit (e.g., search mode transition unit 4721, FIG. 47).The processing unit (or one or more components thereof, such as units 4709 to 4721), without receiving any instructions from a user of the electronic device, monitors a geographical location of the electronic device using a location sensor unit (e.g., location sensor unit 4707), determines (e.g., by determining unit 4715) that the electronic device is within a threshold distance of a predetermined type of point of interest based on the monitored geographical location, identifies (e.g., by identifying unit 4717) at least one activity that is currently popular at the point of interest according to determining that the electronic device is within the threshold distance of the point of interest, and retrieves information about the at least one activity that is currently popular at the point of interest (e.g., by retrieving unit 4713), including retrieving information about the at least one activity that is currently popular at the point of interest. ), configured to detect (e.g., by detection unit 4709) via a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4703) a first input that, when detected, causes the electronic device to transition to a search mode, and in response to detecting the first input, transition to the search mode (e.g., by search mode transition unit 4721), where transitioning to the search mode includes presenting (e.g., by display unit 4711) via a display unit (e.g., display unit 4701) an affordance including (i) information regarding at least one activity and (ii) an indication that the at least one activity has been identified as currently popular in the location of interest, before receiving any user input at the search interface.

[0220] (G16) In some embodiments of the electronic device described in G15, the processing unit is further configured to detect a second input (e.g., by the detection unit 4709) and, in response to detecting the second input, update the affordance (e.g., by the display unit 4711) to include available information about a current activity at a second point of interest different from the point of interest, where the point of interest is within a threshold distance of the electronic device.

[0221] (G17) In some embodiments of the electronic device described in G15 or G16, the affordance further includes a selectable category of points of interest, and the processing unit is further configured to detect (e.g., by the detection unit 4709) a selection of the corresponding selectable category, and, in response to detecting the selection, update (e.g., by the display unit 4711) the affordance to include information about additional points of interest located within a second threshold distance of the device.

[0222] (G18) In some embodiments of the electronic device according to any one of G15 to G17, the point of interest is an amusement park and the retrieved information includes current wait times for rides at the amusement park.

[0223] (G19) In some embodiments of the electronic device described in G18, the retrieved information includes information about waiting times for rides located within a predetermined distance of the electronic device.

[0224] (G20) In some embodiments of the electronic device described in any one of G15 to G17, the point of interest is a restaurant and the retrieved information includes information about popular menu items of the restaurant.

[0225] (G21) In some embodiments of the electronic device described in G20, the retrieved information is retrieved from a social network associated with a user of the electronic device.

[0226] (G22) In some embodiments of the electronic device according to any one of G15 to G17, the point of interest is a movie theater and the retrieved information includes information about movie theater show times.

[0227] (G23) In some embodiments of the electronic device described in G22, the retrieved information is retrieved from a social network associated with a user of the electronic device.

[0228] (G24) In some embodiments of the electronic device described in any one of G15 to G23, after the electronic device is unlocked (e.g., by the unlocking unit 4719), the affordance is available in response to a substantially horizontal swipe on an initial page of a home screen of the electronic device.

[0229] (H1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (touchscreen 112; in some embodiments, the touch-sensitive surface and the display are integrated, as shown in FIG. 1C ). The method includes receiving at least a portion of a voice communication (e.g., a live call or a pre-recorded voicemail of 10 seconds or less), the portion of the voice communication including speech provided by a remote user of a remote device that is different from a user of the electronic device. The method also includes extracting a content item based, at least in part, on the speech provided by the remote user of the remote device. The method further includes determining whether the content item is currently available on the electronic device. In accordance with a determination that the content item is not currently available on the electronic device, the method includes: (i) identifying an application associated with the content item; and (ii) displaying a selectable description of the content item on the display. The method includes storing the content item for presentation in the identified application (e.g., storing the content item so that it is available for presentation by the identified application) in response to detecting a selection of the selectable description. In this manner, a user can remember content items mentioned or spoken in a voice communication without having to remember all of the "spoken" details and later enter those details to create the appropriate content item. Instead, the electronic device can detect and extract the content items based on the speech describing various corresponding content items, then provide selectable descriptions of the content items from which the user can select, and store the corresponding content items on the electronic device.

[0230] (H2) In some embodiments of the method described in H1, the content item is a new event.

[0231] (H3) In some embodiments of the method described in H1, the content item is new event details for an event currently associated with a calendar application on the electronic device.

[0232] (H4) In some embodiments of the method described in H1, the content item is a new contact.

[0233] (H5) In some embodiments of the method described in H1, the content item is new contact information for an existing contact associated with a phone application on the electronic device.

[0234] (H6) In some embodiments of the method according to any one of H1 to H5, the voice communication is a live call.

[0235] (H7) In some embodiments of the method according to any one of H1 to H5, the voice communication is a live FaceTime call.

[0236] (H8) In some embodiments of the method of any one of H1 to H5, the voice communication is a pre-recorded voicemail.

[0237] (H9) In some embodiments of the methods described in any one of H1 to H8, displaying the selectable descriptions includes displaying, within the user interface, the selectable descriptions including recent calls made using a phone application. In this manner, a user can easily and conveniently access extracted content items (e.g., extracted during corresponding voice communications) including recent calls directly from the user interface.

[0238] (H10) In some embodiments of the method described in H9, the selectable description is displayed with an indication that the content item is associated with a voice communication.

[0239] (H11) In some embodiments of the method described in H9, detecting the selection includes receiving the selection while a user interface including the recent calls is displayed.

[0240] (H12) In some embodiments of the method of any one of H1 to H11, the method further includes, in conjunction with displaying the selectable description of the content item, providing feedback to a user of the electronic device that the content item has been detected (e.g., haptic feedback generated by the electronic device or presentation of a user interface object on the second device, so that the user does not have to remove the phone from their ear during a call). In this way, during a voice communication, the user is provided with a concise indication that the content item has been detected / extracted, and the user can then decide whether to store the content item.

[0241] (H13) In some embodiments of the method described in H12, providing feedback includes transmitting information about the detection of the content item to a different electronic device (e.g., a laptop computer, a television monitor, a smart watch, etc.) that is in the vicinity of the electronic device. In this way, the user does not need to interrupt the voice communication and can still view details about the detected / extracted content item on a different device.

[0242] (H14) In some embodiments of the method of any one of H1 to H13, the method further includes determining that the voice communication includes information about a first physical location (e.g., an address mentioned during the call or the name of a restaurant spoken during the call, etc., as described in further detail below). The method also includes detecting an input and, in response to detecting the input, opening an application capable of accepting location data and populating the application with information about the first physical location. In this way, the electronic device, in addition to detecting and extracting event and contact information, can extract location information spoken in the voice communication and provide this location information to the user in an appropriate application (e.g., so that the user is not burdened with remembering specific location details spoken in the call, especially new details that may be unfamiliar to the user, but rather the device extracts these location details and displays them for use by the user).

[0243] (H15) In some embodiments of the method described in H14, the application is a map application, and populating the map application with information about the first physical location includes populating a map displayed within the map application with a location identifier corresponding to the first physical location. In this way, a user can easily use the newly extracted location details to navigate to new destinations, view how far away a particular location is, and easily use other functionality provided by the map application.

[0244] (H16) In some embodiments of the method of any one of H1 to H13, the method further includes determining that the voice communication includes information about the first physical location. The method also includes detecting an input (e.g., a search activation gesture such as a swipe gesture, described in more detail below) and, in response to detecting the input, populating a search interface with information about the first physical location. In this way, in addition to (or instead of) providing the location information to the user for use in a particular application, the electronic device can also provide the location information for use in the search interface (e.g., to search for related points of interest or to search for additional details about the first physical location, such as phone numbers, menus, etc.).

[0245] (H17) In some embodiments of the methods described in any one of H1 to H16, extracting the content item includes analyzing a portion of the voice communication for detection of a predetermined type of content, and the analysis is performed while outputting the voice communication via an audio system in communication with the electronic device (e.g., the voice communication is analyzed in real time while the voice communication is being output to a user of the electronic device).

[0246] (H18) In some embodiments of the method described in H17, analyzing the voice communication includes (i) converting speech provided by a remote user of the remote device into text; (ii) applying a natural language processing algorithm to the text to determine whether the text includes one or more defined keywords; and (iii) determining that the voice communication includes speech describing the content item according to a determination that the text includes the corresponding defined keywords.

[0247] (H19) In some embodiments of the method described in any one of H1 to H18, receiving at least a portion of the voice communication includes receiving an indication from a user of the electronic device that the portion of the voice communication should be analyzed.

[0248] (H20) In some embodiments of the method described in H19, the indication corresponds to the selection of a hardware button (e.g., a user selects a hardware button while a voice communication is being output by an audio system to indicate that a predetermined number of seconds (e.g., the previous 10, 15, or 20 seconds) of the voice communication should be analyzed). In some embodiments, the button may also be presented on a display of the electronic device for user selection (e.g., a button that appears during a voice communication and reads, "Tap here to analyze new content of this voice communication").

[0249] (H21) In some embodiments of the method described in H19, the indication corresponds to a command from a user of the electronic device that includes the words "hey Siri." Thus, the user can simply instruct the electronic device to begin analyzing portions of the voice communication to detect content items (such as information about events, contact information, and physical locations) spoken in the voice communication.

[0250] (H22) In some embodiments of the method of any one of H1 to H21, the method further includes receiving a second portion of the audio communication. The second portion includes speech provided by a remote user of the remote device and speech provided by a user of the electronic device (e.g., the audio communication is a live call and the second portion includes a conversation between the user and the remote user). The method also includes extracting a second content item based, at least in part, on the speech provided by the remote user of the remote device and the speech provided by the user of the electronic device. In accordance with determining that the second content item is not currently available on the electronic device, the method includes (i) identifying a second application associated with the second content item and (ii) displaying a second selectable description of the second content item on the display. In response to detecting a selection of the second selectable description, the method includes storing the second content item for presentation in the identified second application.

[0251] (H23) In some embodiments of the method described in H22, a selectable description and a second selectable description are displayed within a user interface that includes recent calls made using a phone application. In this manner, the user is provided with one interface that conveniently includes content items found in several voice communications (e.g., several calls, voicemails, or calls and voicemails).

[0252] (H24) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of H1 to H23.

[0253] (H25) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of any one of H1 to H23.

[0254] (H26) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of H1 to H23.

[0255] (H27) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive surface and a display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of H1 to H23.

[0256] (H28) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of H1 to H23.

[0257] (H29) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4801, FIG. 48), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4803, FIG. 48), and a processing unit (e.g., processing unit 4805, FIG. 48). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 48 shows display unit 4801 and touch-sensitive surface unit 4803 as being integrated into electronic device 4800, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an audio communication receiving unit (e.g., audio communication receiving unit 4807, FIG. 48), a content item extracting unit (e.g., content item extracting unit 4809, FIG. 48), an availability determining unit (e.g., availability determining unit 4811, FIG. 48), an application identifying unit (e.g., application identifying unit 4813, FIG. 48), a displaying unit (e.g., displaying unit 4815, FIG. 48), a content item storing unit (e.g., content item storing unit 4817, FIG. 48), a feedback providing unit (e.g., feedback providing unit 4819, FIG. 48), an input detecting unit (e.g., input detecting unit 4821, FIG. 48), an application opening unit (e.g., application opening unit 4823, FIG. 48), a populating unit (e.g., populating unit 4825, FIG. 48), and an audio communication analyzing unit (e.g., audio communication analyzing unit 4827, FIG. 48). The processing unit (or one or more components thereof, such as units 4807 to 4827) is configured to receive (e.g., by the audio communication receiving unit 4807) at least a portion of an audio communication, the portion of the audio communication including speech provided by a remote user of a remote device different from the user of the electronic device.The processing unit is further configured to extract a content item based, at least in part, on speech provided by a remote user of the remote device (e.g., by the content item extraction unit 4809) and determine whether the content item is currently available on the electronic device (e.g., by the availability determination unit 4811). In accordance with a determination that the content item is not currently available on the electronic device, the processing unit is further configured to (i) identify an application associated with the content item (e.g., by the application identification unit 4813), and (ii) cause a selectable description of the content item to be displayed on the display (e.g., by the display unit 4815 and / or the display unit 4801). In response to detecting a selection of the selectable description (e.g., by the input detection unit 4821 and / or the touch-sensitive surface unit 4803), the processing unit is configured to store (e.g., by the content item storage unit 4817) the content item for presentation in the identified application.

[0258] (H30) In some embodiments of the electronic device described in H29, the processing unit (or one or more components thereof, such as units 4907 to 4927) is further configured to perform the method described in any one of H2 to H23.

[0259] (I1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A through D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes receiving at least a portion of a voice communication, the portion of the voice communication (e.g., a live call, a pre-recorded voicemail) including speech provided by a remote user of a remote device that is different from a user of the electronic device. The method also includes determining that the voice communication includes speech identifying a physical location. In response to determining that the voice communication includes speech identifying the physical location, the method includes providing an indication that information about the physical location has been detected (e.g., providing haptic feedback to the user, displaying a user interface object with information about the physical location, or transmitting information about the physical location to a nearby device for display on the nearby device). The method further includes detecting input via the touch-sensitive surface. The method includes, in response to detecting the input, (i) opening an application that accepts geographic location data and (ii) populating the application with information about the physical location. In this way, a user can remember information about a physical location mentioned or spoken in a voice communication without having to remember all of the spoken details and later enter those details in an appropriate application. Instead, the electronic device can detect and extract information about the physical location based on speech describing the physical location (e.g., a description of a restaurant, driving directions to the physical location, etc.), and then provide an indication that information about the corresponding physical location has been detected.

[0260] (I2) In some embodiments of the method described in I1, the voice communication is a live call.

[0261] (I3) In some embodiments of the method described in I1, the voice communication is a live FaceTime call.

[0262] (I4) In some embodiments of the method described in I1, the voice communication is a pre-recorded voicemail.

[0263] (I5) In some embodiments of the methods of any one of I1 to I4, providing the indication includes displaying within the user interface selectable descriptions of physical locations that include recent calls made using a phone application.

[0264] (I6) In some embodiments of the method described in I5, the selectable description indicates that the content item is associated with a voice communication.

[0265] (I7) In some embodiments of the method described in I5 or I6, detecting the input includes detecting the input on a selectable description while a user interface including the recent calls is displayed.

[0266] (I8) In some embodiments of the methods of any one of I1 to I7, providing the indication includes providing tactile feedback to a user of the electronic device.

[0267] (I9) In some embodiments of the method of any one of I1 to I8, providing the indication includes transmitting information about the physical location to a different electronic device in the vicinity of the electronic device.

[0268] (I10) In some embodiments of the method of any one of I1 to I9, determining that the voice communication includes speech describing a physical location includes analyzing a portion of the voice communication to detect information related to the physical location, the analysis being performed while outputting the voice communication via an audio system in communication with the electronic device.

[0269] (I11) In some embodiments of the methods described in any one of I1 to I10, receiving at least a portion of the voice communication includes receiving an instruction from a user of the electronic device that the portion of the voice communication should be analyzed.

[0270] (I12) In some embodiments of the method described in I11, the instruction corresponds to selection of a hardware button. In some embodiments, the button may also be a button presented on a display of the electronic device for user selection (e.g., a button that appears during a voice communication and reads, "Tap here to analyze new content in this voice communication").

[0271] (I13) In some embodiments of the method described in I11, the instruction corresponds to a command from a user of the electronic device that includes the words "hey Siri."

[0272] (I14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of I1 to I13.

[0273] (I15) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of any one of I1 to I13.

[0274] (I16) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of I1 to I13.

[0275] (I17) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of I1 to I13.

[0276] (I18) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of I1 to I13.

[0277] (I19) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4901, FIG. 49), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4903, FIG. 49), and a processing unit (e.g., processing unit 4905, FIG. 49). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 49 shows display unit 4901 and touch-sensitive surface unit 4903 as being integrated into electronic device 4900, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an audio communication receiving unit (e.g., audio communication receiving unit 4907, FIG. 49), a content item extraction unit (e.g., content item extraction unit 4909, FIG. 49), an indication providing unit (e.g., indication providing unit 4911, FIG. 49), an input detection unit (e.g., input detection unit 4913, FIG. 49), an application opening unit (e.g., application opening unit 4915, FIG. 49), an application populating unit (e.g., application populating unit 4917, FIG. 49), a feedback providing unit (e.g., feedback providing unit 4919, FIG. 49), and an audio communication analysis unit (e.g., audio communication analysis unit 4921, FIG. 49). The processing unit (or one or more components thereof, such as units 4907 to 4921) is configured to receive at least a portion of an audio communication, the portion of the audio communication including speech provided (e.g., by the audio communication receiving unit 4907) by a remote user of a remote device different from a user of the electronic device. The processing unit is further configured to determine (e.g., by the content item extraction unit 4909) that the audio communication includes speech identifying a physical location.In response to determining that the voice communication includes speech identifying a physical location, the processing unit is configured to provide an indication that information related to the physical location has been detected (e.g., by content item extraction unit 4909). The processing unit is also configured to detect input via the touch-sensitive surface unit (e.g., by input detection unit 4911). In response to detecting the input, the processing unit is configured to (i) open an application that accepts geographic location data (e.g., by application opening unit 4913), and (ii) populate the application with information related to the physical location (e.g., by application populating unit 4915).

[0278] (I20) In some embodiments of the electronic device described in I19, the processing unit (or one or more components thereof, such as units 4907 to 4921) is further configured to perform the method described in any one of I2 to I13.

[0279] (J1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes presenting a text input field and a conversation record in a message application on the display. The method also includes determining, while the message application is presented on the display, that a next possible input from a user of the electronic device is information related to a physical location. The method further includes analyzing content associated with the text input field and the conversation record to determine a suggested physical location based, at least in part, on a portion of the analyzed content. The method further includes presenting, within the message application on the display, a selectable user interface element that identifies the suggested physical location and receiving a selection of the selectable user interface element. The method includes, in response to receiving the selection, presenting a representation of the suggested physical location in the text input field. In this way, users of electronic devices are conveniently provided with the content they need, without the user having to type anything or search for the content (e.g., a user can enter their current location by simply selecting a selectable user interface element, rather than having to access a maps application to reveal their exact location, return to a messaging application, and explicitly type to send the location information).

[0280] (J2) In some embodiments of the method described in J1, the message application includes a virtual keyboard, and the selectable user interface elements are displayed in a suggestions portion adjacent to and above the virtual keyboard.

[0281] (J3) In some embodiments of the method described in J1 or J2, determining that a next possible input from a user of the electronic device is information regarding a physical location includes processing content associated with the text entry field and the conversation record to detect that the conversation transcription includes a question regarding the user's current location. In this manner, the user is provided with a suggested physical location that is directly related to a description in the conversation transcription (e.g., in response to a second user asking "Where are you?", the user is presented with a user interface object that, when selected, causes the device to send information regarding the user's current location to the second user).

[0282] (J4) In some embodiments of the method described in J3, processing the content includes applying a natural language processing algorithm to detect one or more predefined keywords that form a question (e.g., "Where are you?" or "What is your address?").

[0283] (J5) In some embodiments of the method described in J3 or J4, the question is included in a message received from a second user different from the user.

[0284] (J6) In some embodiments of the method according to any one of J1 to J5, determining that a next possible input from a user of the electronic device is information regarding a physical location includes monitoring typing input received from the user in a text input portion of a messaging application.

[0285] (J7) In some embodiments of the method of any one of J1 to J6, the method further includes ceasing presentation of the selectable user interface element in accordance with a determination that the user is typing and has not selected a selectable user interface element. In this manner, the device does not continue to present the selectable user interface object if it can be determined that the user is not interested in selecting an object.

[0286] (J8) In some embodiments of the method of any one of J1 to J7, the method further includes ceasing presentation of the selectable user interface element in accordance with a determination that the user has provided additional input indicating that the user will not select the selectable user interface element. In this manner, the device does not continue to present the selectable user interface object if it can be determined that the user is not interested in selecting the object.

[0287] (J9) In some embodiments of the method according to any one of J1 to J5, the representation of the proposed physical location includes information identifying a current geographic location of the electronic device.

[0288] (J10) In some embodiments of the method according to any one of J1 to J9, the proposed representation of the physical location is an address.

[0289] (J11) In some embodiments of the method according to any one of J1 to J9, the proposed physical location is a map object that includes an identifier for the proposed physical location.

[0290] (J12) In some embodiments of the method according to any one of J1 to J11, the suggested physical location corresponds to a location recently viewed by the user in an application other than a messaging application.

[0291] (J13) In some embodiments of the method according to any one of J1 to J12, the message application is an email application.

[0292] (J14) In some embodiments of the method according to any one of J1 to J12, the message application is a text message application.

[0293] (J15) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of J1 to J14.

[0294] (J16) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of any one of J1 to J14.

[0295] (J17) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of J1 to J14.

[0296] (J18) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of J1 to J14.

[0297] (J19) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of J1 to J14.

[0298] (J20) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5001, FIG. 50), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5003, FIG. 50), and a processing unit (e.g., processing unit 5005, FIG. 50). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 50 shows display unit 5001 and touch-sensitive surface unit 5003 as being integrated into electronic device 5000, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5007, FIG. 50), a next input determination unit (e.g., next input determination unit 5009, FIG. 50), a content analysis unit (e.g., content analysis unit 5011, FIG. 50), a selection receiving unit (e.g., selection receiving unit 5013, FIG. 50), a typing input monitoring unit (e.g., typing input monitoring unit 5015, FIG. 50), and a presentation termination unit (e.g., presentation termination unit 5017, FIG. 50). The processing unit (or one or more components thereof, such as units 5007 to 5017) is configured to present (e.g., by the presentation unit 5007 and / or the display unit 5001) a text input field and a conversation record in a message application on the display. While the message application is presented on the display, the processing unit is also configured to determine (e.g., by the next input determination unit 5009) that a next possible input from a user of the electronic device is information about a physical location.The processing unit is further configured to analyze content associated with the text entry field and the conversation record to determine a suggested physical location based at least in part on a portion of the analyzed content (e.g., by the content analysis unit 5011), present a selectable user interface element within the message application on the display (e.g., by the presentation unit 5007) that identifies the suggested physical location, receive a selection of the selectable user interface element (e.g., by the selection receiving unit 5013 and / or the touch-sensitive surface unit 5003), and, in response to receiving the selection, present a representation of the suggested physical location within the text entry field (e.g., by the presentation unit 5007).

[0299] (J21) In some embodiments of the electronic device described in J20, the processing unit (or one or more components thereof, such as units 5007 to 5017) is further configured to perform a method described in any one of J2 to J14.

[0300] (K1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured in accordance with any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes, while displaying the first application, obtaining information identifying a first physical location viewed by a user in the first application (e.g., a restaurant searched for by a user in an application that enables searching for local businesses). The method also includes closing the first application and, after closing the first application, receiving a request from the user to open a second application that is different from the first application. In some embodiments, the request is received without receiving any input in the first application (e.g., the request does not include clicking a link or button in the first application). In response to receiving the request and following a determination that the second application is capable of accepting geographic location information, the method includes presenting the second application, where presenting the second application includes populating the second application with information based, at least in part, on information identifying the first physical location. In this manner, a user does not need to manually transfer information between two different applications. Instead, the device intelligently determines that the second application is capable of accepting geographic location information and then populates the second application directly with information related to the physical location viewed in the first application (e.g., populating a map object in the second application to include an identifier related to the physical location).

[0301] (K2) In some embodiments of the method described in K1, receiving the request to open the second application includes detecting an input on an affordance of the second application after the first application has closed. In other words, the request does not correspond to clicking on a link in the first application, but instead explicitly and directly requests that the user open the second application, and the device then determines to populate the second application with information about a previously viewed physical location (previously viewed in a different first application) so that the user can further explore or consider this previously viewed physical location in the second application.

[0302] (K3) In some embodiments of the method described in K2, the affordance of the second application is an icon displayed within a home screen of the electronic device. In some embodiments, the home screen is a system-level component of an operating system and includes icons for invoking applications available on the electronic device.

[0303] (K4) In some embodiments of the method described in K2, detecting the input includes (i) detecting a double tap on a physical home button; (ii) displaying an application switching user interface in response to detecting the double tap; and (iii) detecting a selection of an affordance from within the application switching user interface.

[0304] (K5) In some embodiments of the methods described in any one of K1 to K4, populating the second application includes displaying a user interface object including information based, at least in part, on information identifying the first physical location.

[0305] (K6) In some embodiments of the method described in K5, the user interface object includes a textual description informing the user that the first physical location was recently viewed in the first application.

[0306] (K7) In some embodiments of the method described in K6, the user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location.

[0307] (K8) In some embodiments of the method described in K6 or K7, the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard.

[0308] (K9) In some embodiments of the method described in any one of K6 to K8, obtaining the information includes obtaining information regarding the second physical location, and displaying the user interface object includes displaying a user interface object having information regarding the second physical location.

[0309] (K10) In some embodiments of the method described in any one of K1 to K9, determining that the second application is capable of accepting geographic location information includes one or more of: (i) determining that the second application includes an input receiving field capable of accepting and processing geographic location data; (ii) determining that the second application is capable of displaying the geographic location information on a map; (iii) determining that the second application is capable of using the geographic location information to facilitate route guidance; and (iv) determining that the second application is capable of using the geographic location information to locate and provide transportation services.

[0310] (K11) In some embodiments of the method described in K10, determining that the second application is capable of accepting geographic location information includes determining that the second application includes an input-receiving field capable of accepting and processing geographic location data, the input-receiving field being a search box that enables searching within a map displayed within the second application.

[0311] (K12) In some embodiments of the method described in any one of K1 to K11, the method further includes, in response to receiving the request, determining whether the second application is associated with the first application (e.g., opened a threshold number of times after opening the first application) based on the user's application usage history.

[0312] (K13) In some embodiments of the method described in K12, the method further includes, before presenting the second application, providing the second application with access to information identifying the first physical location, where before the access is provided, the second application does not have access to information identifying the first physical location. In this manner, the second application can receive information regarding actions taken by the user in the first application, and the user is then provided with a way to use this information within the second application (e.g., to retrieve more information about the first physical location or to use the first physical location for any service, such as a ride-sharing service, available through the second application).

[0313] (K14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform the method of any one of K1 to K13.

[0314] (K15) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method of any one of K1 to K13.

[0315] (K16) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of K1 to K13.

[0316] (K17) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive surface and a display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of K1 to K13.

[0317] (K18) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of K1 to K13.

[0318] (K19) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5101, FIG. 51), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5103, FIG. 51), and a processing unit (e.g., processing unit 5105, FIG. 51). In some embodiments, the electronic device is configured in accordance with any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 51 shows display unit 5101 and touch-sensitive surface unit 5103 as being integrated into electronic device 5100, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an information acquisition unit (e.g., information acquisition unit 5107, FIG. 51), an application termination unit (e.g., application termination unit 5109, FIG. 51), a request receiving unit (e.g., request receiving unit 5111, FIG. 51), an application performance determination unit (e.g., application performance determination unit 5113, FIG. 51), an application presentation unit (e.g., application presentation unit 5115, FIG. 51), an application populating unit (e.g., application populating unit 5117, FIG. 51), an input detection unit (e.g., input detection unit 5119, FIG. 51), an application switching user interface display unit (e.g., application switching user interface display unit 5121, FIG. 51), an application association determination unit (e.g., application association determination unit 5123, FIG. 51), and an access providing unit (e.g., access providing unit 5125, FIG. 51). The processing unit (or one or more components thereof, such as units 5107 to 5125) is configured to acquire (e.g., by information acquisition unit 5107) information identifying a first physical location viewed by the user in the first application while the first application is being displayed.The processing unit is also configured to terminate the first application (e.g., by application termination unit 5109), and, after the first application is terminated, receive a request from the user to open a second application different from the first application (e.g., by request receiving unit 5111). In response to receiving the request and in accordance with a determination that the second application can accept geographic location information (e.g., a determination processed or performed by application capabilities determining unit 5113), present the second application (e.g., by application presenting unit 5115), where presenting the second application includes populating the second application with information based at least in part on the information identifying the first physical location (e.g., by application populating unit 5117).

[0319] (K20) In some embodiments of the electronic device described in K19, the processing unit (or one or more components thereof, such as units 5107 to 5125) is further configured to perform a method described in any one of K2 to K13.

[0320] (L1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes obtaining information identifying a first physical location viewed by a user in a first application and detecting a first input. In response to detecting the first input, the method includes: (i) identifying a second application capable of accepting geographic location information; and (ii) presenting, on at least a portion of the display, an affordance having a suggestion to open the second application, the second application having information related to the first physical location, different from the first application. The method also includes detecting the second input in the affordance. In response to detecting the second input in the affordance, (i) opening the second application; and (ii) populating the second application to include information based, at least in part, on the information identifying the first physical location.

[0321] Compared to the operations associated with K1, the operations associated with L1 do not receive a specific request from the user to open the second application before providing the user with a proposal to open the second application having information about the first physical location. Thus, by providing operations associated with both K1 and L1 (and combinations thereof using several processing steps of each of these methods), an electronic device can provide an efficient user experience that enables the user to predictively use location data either before or after the user opens an application capable of accepting geographic location information. Additionally, in embodiments of L1 in which a determination that the second application can accept geographic location information is made even before the second application is opened, and thus an input corresponds to a request to open an application switching user interface, the application switching user interface simply displays a proposal to open an application (e.g., the second application) having information about the first physical location if it is known that the application can accept location data.

[0322] (L2) In some embodiments of the method described in L1, the first input corresponds to a request to open an application switching user interface (e.g., the first input is a double tap on the physical home button of the electronic device).

[0323] (L3) In some embodiments of the method described in L2, the affordance is presented within an application switching user interface.

[0324] (L4) In some embodiments of the method described in L3, presenting the affordance includes, in conjunction with presenting the affordance, presenting a representation of the application running on the electronic device (e.g., a snapshot of application content of the application) within the application switching user interface, and presenting the affordance within an area of ​​the display located below the representation of the application.

[0325] (L5) In some embodiments of the method described in L1, the first input corresponds to a request to open a home screen of the electronic device (e.g., the first input is a single tap on a physical home button of the electronic device).

[0326] (L6) In some embodiments of the method described in L5, the affordance is presented on a portion of the home screen.

[0327] (L7) In some embodiments of the method of any one of L1 to L6, the suggestion includes a type-specific textual description associated with the second application.

[0328] (L8) In some embodiments of the methods described in any one of L1 to L7, populating the second application includes displaying a user interface object including information based, at least in part, on information identifying the first physical location.

[0329] (L9) In some embodiments of the method described in L8, the user interface object includes a textual description informing the user that the first physical location was recently viewed in the first application.

[0330] (L10) In some embodiments of the method described in L9, the user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location.

[0331] (L11) In some embodiments of the method described in L9 or L10, the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard.

[0332] (L12) In some embodiments of the methods described in any one of L1 to L11, identifying the second application capable of accepting geographic location information includes one or more of: (i) determining that the second application includes an input-receiving field capable of accepting and processing geographic location data; (ii) determining that the second application is capable of displaying the geographic location information on a map; (iii) determining that the second application is capable of using the geographic location information to facilitate route guidance; and (iv) determining that the second application is capable of using the geographic location information to locate and provide transportation services.

[0333] (L13) In some embodiments of the method described in L12, identifying that the second application is capable of accepting geographic location information includes determining that the second application includes an input-receiving field capable of accepting and processing geographic location data, wherein the input-receiving field is a search box that enables searching within a map displayed within the second application.

[0334] (L14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of L1 to L13.

[0335] (L15) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of any one of L1 to L13.

[0336] (L16) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of L1 to L13.

[0337] (L17) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of L1 to L13.

[0338] (L18) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of L1 to L13.

[0339] (L19) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5201, FIG. 52), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5203, FIG. 52), and a processing unit (e.g., processing unit 5205, FIG. 52). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 52 shows display unit 5201 and touch-sensitive surface unit 5203 as being integrated into electronic device 5200, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an information acquisition unit (e.g., information acquisition unit 5207, FIG. 52), an input detection unit (e.g., input detection unit 5209, FIG. 52), an application identification unit (e.g., application identification unit 5211, FIG. 52), an affordance presentation unit (e.g., affordance presentation unit 5213, FIG. 52), an application opening unit (e.g., application opening unit 5215, FIG. 52), an application populating unit (e.g., application populating unit 5217, FIG. 52), an application switching user interface presentation unit (e.g., application switching user interface presentation unit 5219, FIG. 52), and an application performance determination unit (e.g., application performance determination unit 5221, FIG. 52). The processing unit (or one or more components thereof, such as units 5207 to 5221) is configured to obtain information identifying a first physical location viewed by a user in a first application (e.g., by the information obtaining unit 5207) and detect a first input (e.g., by the input detecting unit 5209).In response to detecting the first input, the processing unit is configured to (i) identify (e.g., via the application identifying unit 5209) a second application that can accept geographic location information, and (ii) present (e.g., via the affordance presenting unit 5213) on at least a portion of the display an affordance having a proposal to open the second application, different from the first application, the second application having information about the first physical location. The processing unit is also configured to detect (e.g., via the input detecting unit 5209) the second input in the affordance. In response to detecting the second input in the affordance, the processing unit is configured to (i) open the second application (e.g., via the application opening unit 5215) and (ii) populate (e.g., via the application populating unit 5217) the second application to include information based, at least in part, on the information identifying the first physical location.

[0340] (L20) In some embodiments of the electronic device described in L19, the processing unit (or one or more components thereof, such as units 5207 to 5221) is further configured to perform a method according to any one of L2 to L13.

[0341] (M1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes obtaining information identifying a first physical location viewed by a user in a first application running on the electronic device. The method also includes determining that the user has entered a vehicle. In response to determining that the user has entered the vehicle, providing a prompt to the user to use the first physical location as a route guidance destination. The method includes receiving an instruction from the user in response to providing the prompt to use the first physical location as a route guidance destination. The method further includes facilitating route guidance to the first physical location. In this manner, the user is conveniently provided with routing suggestions for the destination based on the previously viewed physical location in an application on the electronic device.

[0342] (M2) In some embodiments of the method described in M1, the method further includes detecting that the electronic device has received a message, the message including detecting that the message includes information identifying a second physical location, and in response to the detection, providing a new prompt to the user to use the second physical location as a new destination for route directions. In this manner, the user can dynamically add a waypoint or a new destination for route directions based on information included in the message (e.g., text, email, voicemail, etc.).

[0343] (M3) In some embodiments of the method described in M2, the method further includes facilitating route guidance to the second physical location in response to receiving an instruction from the user to use the second physical location as a new destination.

[0344] (M4) In some embodiments of the method described in M2 or M3, detecting that the message includes information identifying the second physical location includes performing the detection while a virtual assistant available on the electronic device is reading the message to the user via an audio system in communication with the electronic device. In this manner, when the user is listening to the message read by the audio system (e.g., via a personal assistant available via the electronic device), the electronic device detects the information identifying the second physical location and uses the detected information to suggest using the second physical location as a new destination. Thus, the user does not need to divert their attention from the road while driving, yet can dynamically adjust route guidance settings and destinations.

[0345] (M5) In some embodiments of the method according to any one of M2 to M4, determining that the user has entered the vehicle includes detecting that the electronic device has established a communication link with the vehicle.

[0346] (M6) In some embodiments of the method of any one of M2 to M5, facilitating route guidance includes providing route guidance via a display of the electronic device.

[0347] (M7) In some embodiments of the method described in any one of M2 to M5, facilitating route guidance includes transmitting, to the vehicle, information identifying the first physical location.

[0348] (M8) In some embodiments of the methods described in any one of M2 to M7, facilitating route guidance includes providing route guidance through an audio system in communication with the electronic device (e.g., a car speaker or the device's own built-in speaker).

[0349] (M9) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of M1 to M8.

[0350] (M10) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method of any one of M1 to M8.

[0351] (M11) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of M1 to M8.

[0352] (M12) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of M1 to M8.

[0353] (M13) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of M1 to M8.

[0354] (M14) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5301, FIG. 53), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5303, FIG. 53), and a processing unit (e.g., processing unit 5305, FIG. 53). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 53 shows display unit 5301 and touch-sensitive surface unit 5303 as being integrated into electronic device 5300, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an information acquiring unit (e.g., information acquiring unit 5307, FIG. 53), a vehicle entry determining unit (e.g., vehicle entry determining unit 5309, FIG. 53), a prompt providing unit (e.g., prompt providing unit 5311, FIG. 53), an instruction receiving unit (e.g., instruction receiving unit 5313, FIG. 53), a route guidance facilitating unit (e.g., route guidance facilitating unit 5315, FIG. 53), and a message detecting unit (e.g., message detecting unit 5317, FIG. 53). The processing unit (or one or more components thereof, such as units 5307 to 5317) is configured to acquire (e.g., by the information acquiring unit 5307) information identifying a first physical location viewed by a user in a first application running on the electronic device. The processing unit is also configured to determine (e.g., by the vehicle entry determining unit 5309) that the user has entered a vehicle. In response to determining that the user has entered the vehicle, the processing unit is configured to provide a prompt to the user to use the first physical location as a destination for route guidance (e.g., by the prompt providing unit 5311). In response to providing the prompt, receive an instruction from the user to use the first physical location as a destination for route guidance (e.g., by the instruction receiving unit 5313).The processing unit is further configured to facilitate route guidance to the first physical location (eg, by route guidance facilitation unit 5307).

[0355] (M15) In some embodiments of the electronic device described in M14, the processing unit (or one or more components thereof, such as units 5307 to 5317) is further configured to perform a method according to any one of M2 to M8.

[0356] (N1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured in accordance with any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes presenting content in a first application. The method also includes receiving a request from a user to open a second application different from the first application, the second application including an input-receiving field. The method includes presenting an input-receiving field in the second application in response to receiving the request. The method includes providing a selectable user interface object to enable a user to paste at least a portion of the content into the input-receiving field before receiving any user input in the input-receiving field. The method includes pasting the portion of the content into the input-receiving field in response to detecting a selection of the selectable user interface object. In this way, the user is provided with a proactive paste action in the second application-based content that was previously viewed with the first action (e.g., this allows the user to paste content into the second application without having to reopen the first application, perform an explicit copy action, reopen the second application, and then explicitly request that the copied content be pasted into the second application).

[0357] (N2) According to some embodiments of the method described in N1, the method includes identifying an input-receiving field as a field capable of accepting a portion of content before providing a selectable user interface object.

[0358] (N3) According to some embodiments of the method described in N2, identifying the input-receiving field as a field capable of accepting the portion of content is performed in response to detecting a selection of the input-receiving field.

[0359] (N4) According to some embodiments of the method according to any one of N1 to N3, the part of the content corresponds to an image.

[0360] (N5) According to some embodiments of the method according to any one of N1 to N3, the part of the content corresponds to text content.

[0361] (N6) According to some embodiments of the method according to any one of N1 to N3, the part of the content corresponds to text content and an image.

[0362] (N7) According to some embodiments of the method according to any one of N1 to N6, the first application is a web browsing application and the second application is a messaging application.

[0363] (N8) According to some embodiments of the method according to any one of N1 to N6, the first application is a photo browsing application and the second application is a message application.

[0364] (N9) According to some embodiments of the method described in any one of N1 to N8, the method includes receiving a request to copy at least a portion of the content before receiving the request to open the second application.

[0365] (N10) According to some embodiments of the methods described in any one of N1 to N9, the selectable user interface object is displayed with an indication that the piece of content was recently viewed in the first application, thus providing the user with a clear indication as to why the paste suggestion was made.

[0366] (N11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of N1 to N10.

[0367] (N12) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method of any one of N1 to N10.

[0368] (N13) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of N1 to N10.

[0369] (N14) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of N1 to N10.

[0370] (N15) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of N1 to N10.

[0371] (N16) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5401, FIG. 54), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5403, FIG. 54), and a processing unit (e.g., processing unit 5405, FIG. 54). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 54 shows display unit 5401 and touch-sensitive surface unit 5403 as being integrated into electronic device 5400, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5407, FIG. 54), a request receiving unit (e.g., request receiving unit 5409, FIG. 54), a user interface object providing unit (e.g., user interface object providing unit 5411, FIG. 54), a proactive pasting unit (e.g., proactive pasting unit 5413, FIG. 54), and a performance determination unit (e.g., performance determination unit 5415, FIG. 54).The processing unit (or one or more components thereof, such as units 5407 to 5415) is configured to present content in a first application (e.g., by the presentation unit 5407 and / or the display unit 5401), receive a request from a user to open a second application different from the first application (e.g., by the request receiving unit and / or the touch-sensitive surface unit 5403), the second application including an input receiving field, and in response to receiving the request, present the input receiving field in the second application (e.g., by the presentation unit 5407 and / or the display unit 5401), provide a selectable user interface object (e.g., by the user interface object providing unit 5411 and / or the display unit 5401) to enable the user to paste at least a portion of the content into the input receiving field before receiving any user input in the input receiving field, and in response to detecting selection of the selectable user interface object, paste a portion of the content into the input receiving field (e.g., by the proactive pasting unit 5413).

[0372] (N17) In some embodiments of the electronic device described in N16, the processing unit (or one or more components thereof, such as units 5407 to 5415) is further configured to perform the method described in any one of N1 to N10.

[0373] (O1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touchscreen 112; as shown in FIG. 1C , the touch-sensitive surface and the display are integrated). The method includes presenting textual content associated with an application on the display. The method also includes determining that a portion of the textual content is related to (i) a location, (ii) a contact, or (iii) an event. If the method determines that the portion of the textual content is related to a location, it includes obtaining location information from a location sensor on the electronic device and preparing the obtained location information for display as a predicted content item. If the method determines that the portion of the textual content is related to a contact, it includes performing a search on the electronic device for contact information associated with the portion of the textual content and preparing information associated with at least one contact retrieved from the search for display as a predicted content item. Upon determining that the portion of the textual content is related to an event, the method includes conducting a new search on the electronic device for event information related to the portion of the textual content, and preparing information based at least in part on the at least one event retrieved from the new search for display as a predicted content item. The method further includes displaying an affordance within the application that includes the predicted content item, detecting a selection of the affordance via the touch-sensitive surface, and, in response to detecting the selection, displaying information associated with the predicted content item on the display adjacent to the textual content. In this manner, the user is conveniently provided with predicted content items that can be used to complete a sentence (or, for example, to answer a question posed by another user in a messaging application), without the user having to type anything or search information available on the electronic device to find the desired information.For example, the electronic device may provide phone numbers, current location, availability for scheduling new events, and details related to current events, all without any explicit request or extra effort by the user, thereby saving time while still providing the user with the desired information efficiently.

[0374] (O2) According to some embodiments of the method described in O1, the portion of the textual content corresponds to the most recently presented textual content within the application.

[0375] (O3) According to some embodiments of the method described in O1 or O2, the application is a message application and part of the text content is a question received within the message application from a remote user of a remote device different from the electronic device.

[0376] (O4) According to some embodiments of the method described in O1 or O2, the part of the textual content is an input provided by a user of the electronic device in an input receiving field in the application.

[0377] (O5) According to some embodiments of the method described in O1, the portion of the textual content is identified in response to a user input selecting a user interface object that includes the portion of the textual content.

[0378] (O6) According to some embodiments of the method described in O5, the application is a messaging application and the user interface object is a message bubble in a conversation displayed within the messaging application.

[0379] (O7) According to some embodiments of the method described in O5 or O6, the method further includes detecting a selection of the second user interface object, and in response to detecting the selection, (i) ceasing to display the affordance with the predicted content item and determining that textual content associated with the second user interface object is related to a location, a contact, or an event, and displaying a new predicted content item within the application in accordance with the determination. In this manner, a user can easily return to a message conversation and select a previously received message, and still be provided with an appropriately predicted content item.

[0380] (O8) According to some embodiments of the methods described in any one of O1 to O7, the affordance is displayed in an input-receiving field adjacent to a virtual keyboard within the application.

[0381] (O9) According to some embodiments of the method described in O8, the input receiving field is a field that displays typing input received on a virtual keyboard.

[0382] (O10) According to some embodiments of the methods described in any one of O1 to O9, determining includes parsing the text content as it is received by the application and detecting stored patterns known to be associated with contacts, events and / or locations.

[0383] (O11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method described in any one of O1 to O10.

[0384] (O12) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method of any one of O1 to O10.

[0385] (O13) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method described in any one of O1 to O10.

[0386] (O14) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of O1 to O10.

[0387] (O15) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of O1 to O10.

[0388] (O16) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5501, FIG. 55), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5503, FIG. 55), and a processing unit (e.g., processing unit 5505, FIG. 55). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 55 shows display unit 5501 and touch-sensitive surface unit 5503 as being integrated into electronic device 5500, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5507, FIG. 55), a determination unit (e.g., determination unit 5509, FIG. 55), an acquisition unit (e.g., acquisition unit 5511, FIG. 55), a search implementation unit (e.g., search implementation unit 5513, FIG. 55), an information preparation unit (e.g., information preparation unit 5515, FIG. 55), an affordance display unit (e.g., affordance display unit 5517, FIG. 55), and a detection unit (e.g., detection unit 5519, FIG. 55).The processing unit (or one or more components thereof, such as units 5507 to 5519) presents text content associated with the application on a display (e.g., by the presentation unit 5507 and / or the display unit 5501); determines (e.g., by the determining unit 5509) that a portion of the text content is related to (i) a location, (ii) a contact, or (iii) an event; upon determining that the portion of the text content is related to a location, acquires location information from a location sensor on the electronic device (e.g., by the acquiring unit 5511); prepares the acquired location information for display as a predicted content item (e.g., by the information preparing unit 5515); upon determining that the portion of the text content is related to a contact, performs a search on the electronic device for contact information related to the portion of the text content (e.g., by the search conducting unit 5513); and selects information associated with at least one contact retrieved by the search for display as a predicted content item. the electronic device (e.g., by the information preparing unit 5515), and upon determining that the portion of the textual content is related to the event, performs a new search on the electronic device (e.g., by the search performing unit 5513) for event information related to the portion of the textual content, prepares (e.g., by the information preparing unit 5515) information based at least in part on the at least one event retrieved by the new search for display as a predicted content item, displays (e.g., by the affordance display unit 5517 and / or the display unit 5501) in the application an affordance including the predicted content item, detects (e.g., by the detection unit 5519) a selection of the affordance via the touch-sensitive surface, and in response to detecting the selection, displays (e.g., by the presentation unit 5507 and / or the display unit 5501) information associated with the predicted content item on the display adjacent to the textual content.

[0389] (O17) In some embodiments of the electronic device described in O16, the processing unit (or one or more components thereof, such as units 5507 to 5519) is further configured to perform a method according to any one of O1 to O10.

[0390] As described above (and in more detail below), one aspect of the present technology is the collection and use of data available from various sources (e.g., based on speech provided during voice communications) to improve the delivery to users of content that may be of interest to them. This disclosure contemplates that, in some examples, this collected data may include personal information data that uniquely identifies a particular person or that can be used to contact or locate a particular person. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, home addresses, or any other identifiable information.

[0391] This disclosure recognizes that the use of such personal information data may be used to the benefit of the user. For example, personal information data may be used to deliver targeted content that is more likely to interest the user. Thus, use of such personal information data allows for intentional control of the content that is delivered. Additionally, other uses of personal information data that benefit the user are contemplated by this disclosure.

[0392] This disclosure further contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other uses of such personal information data will comply with appropriately defined privacy policies and / or privacy practices. Specifically, such entities should implement and always use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for maintaining the strict confidentiality of personal information data. For example, personal information from users should be collected for the entity's lawful and legitimate uses and should not be shared or sold except for those lawful uses. Furthermore, such collection should only be carried out after the user's informed consent is obtained. Furthermore, such entities may take all necessary measures to protect and secure access to such personal information data and to ensure that others with access to that personal information data comply with their privacy policies and procedures. Furthermore, such entities may submit themselves to third-party assessment to attest to their compliance with widely accepted privacy policies and practices.

[0393] Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which a user selectively prevents use of or access to personal information data. That is, the present disclosure contemplates that hardware and / or software elements may be provided to prevent or block access to such personal information data. For example, when monitoring voice communications or actions performed by a user within an application, the present technology may be configured to allow a user to "opt in" or "opt out" of participating in the collection of personal information data during registration for the service. In another example, a user may choose not to provide location information to a given content distribution service. In yet another example, a user may choose not to provide precise location information but allow the transfer of location zone information.

[0394] Thus, while this disclosure broadly encompasses the use of personal information data to implement one or more various disclosed embodiments, this disclosure also contemplates that the various embodiments can also be implemented without requiring access to such personal information data. That is, various embodiments of the present technology are not rendered inoperable by the absence of all or part of such personal information data. For example, content can be selected and delivered to a user by inferring preferences based on non-personal information or minimal personal information, such as content requested by a device associated with the user, other non-personal information available to the content delivery service, or publicly available information.

[0395] It should be noted that the various embodiments described above can be combined with any other embodiment described herein. The features and advantages described herein are not exhaustive, and many additional features and advantages will be apparent to those skilled in the art, particularly in light of the drawings, the specification, and the claims. Furthermore, it should be noted that the language used in this specification has been selected solely for the purposes of readability and explanation, and not to define or limit the subject matter of the present invention. [Brief explanation of the drawings]

[0396] For a better understanding of the various described embodiments, please refer to the Detailed Description section below in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout the drawings:

[0397] [Figure 1A] 1 is a high-level block diagram of a computing device with a touch-sensitive display in accordance with some embodiments.

[0398] [Figure 1B] FIG. 1 is a block diagram of example components for event processing according to some embodiments.

[0399] [Figure 1C] 1 is an overview of a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

[0400] [Figure 1D] 1 is a schematic used to illustrate a computing device with a touch-sensitive surface that is separate from the display, according to some embodiments. [Figure 1E] 1 is an overview of an example configuration of a computing device that constitutes an electronic device, according to some embodiments.

[0401] [Figure 2] 1 is an overview of a touch screen used to present a user interface for a menu of applications, according to some embodiments.

[0402] [Figure 3A] FIG. 2 is a block diagram illustrating a data structure for storing application usage data according to some embodiments. [Figure 3B] FIG. 2 is a block diagram illustrating a data structure for storing application usage data according to some embodiments.

[0403] [Figure 4A] FIG. 10 is a block diagram illustrating a data structure for storing trigger conditions, according to some embodiments. [Figure 4B] FIG. 10 is a block diagram illustrating a data structure for storing trigger conditions, according to some embodiments.

[0404] [Figure 5] 1 is a block diagram illustrating an example of a trigger condition establishment system, according to some embodiments.

[0405] [Figure 6A]1 is a flowchart illustrating a method for proactively identifying and surfacing (e.g., surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments. [Figure 6B] 1 is a flowchart illustrating a method for proactively identifying and surfacing (e.g., surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments.

[0406] [Figure 7A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 7B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments.

[0407] [Figure 8A] 1 is a flowchart illustrating a method for proactively identifying and surfacing (e.g., surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments. [Figure 8B] 1 is a flowchart illustrating a method for proactively identifying and surfacing (e.g., surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments.

[0408] [Figure 9A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 9B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 9C]1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 9D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments.

[0409] [Figure 10A] 1 is a flowchart illustrating a method for proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display, according to some embodiments. [Figure 10B] 1 is a flowchart illustrating a method for proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display, according to some embodiments. [Figure 10C] 1 is a flowchart illustrating a method for proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display, according to some embodiments.

[0410] [Figure 11A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11D]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11E] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11F] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11G] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11H] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11I] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11J] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments.

[0411] [Figure 12] 1 is a flow chart illustrating a method for entering a search mode based on heuristics, according to some embodiments.

[0412] [Figure 13A] 1 is an overview of a touch-sensitive display used to present a user interface for entering a search mode based on heuristics, according to some embodiments. [Figure 13B] 1 is an overview of a touch-sensitive display used to present a user interface for entering a search mode based on heuristics, according to some embodiments.

[0413] [Figure 14] 1 is a flowchart illustrating a method for proactively providing vehicle location on an electronic device with a touch-sensitive display according to some embodiments.

[0414] [Figure 15A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing vehicle location, according to some embodiments. [Figure 15B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing vehicle location, according to some embodiments.

[0415] [Figure 16A] 1 is a flowchart illustrating a method for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. [Figure 16B] 1 is a flowchart illustrating a method for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments.

[0416] [Figure 17A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. [Figure 17B]1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. [Figure 17C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. [Figure 17D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. [Figure 17E] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments.

[0417] [Figure 18A] 1 is a flow chart illustrating a method for extracting a content item from an audio communication and interacting with the extracted content item according to some embodiments. [Figure 18B] 1 is a flow chart illustrating a method for extracting a content item from an audio communication and interacting with the extracted content item according to some embodiments.

[0418] [Figure 19A] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications, according to some embodiments. [Figure 19B] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications, according to some embodiments. [Figure 19C]1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications, according to some embodiments. [Figure 19D] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications, according to some embodiments. [Figure 19E] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications, according to some embodiments. [Figure 19F] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications, according to some embodiments.

[0419] [Figure 20] 1 is a flowchart illustrating a method of determining that an audio communication includes speech that identifies a physical location and populating an application with information about the physical location, according to some embodiments.

[0420] [Figure 21A] 1 is an overview of a touch-sensitive display used to present a user interface for determining that an audio communication includes speech identifying a physical location and populating an application with information about that physical location, according to some embodiments. [Figure 21B] 1 is an overview of a touch-sensitive display used to present a user interface for determining that an audio communication includes speech identifying a physical location and populating an application with information about that physical location, according to some embodiments.

[0421] [Figure 22A]1 is a flowchart illustrating a method for proactively suggesting physical locations for use in a messaging application, according to some embodiments. [Figure 22B] 1 is a flowchart illustrating a method for proactively suggesting physical locations for use in a messaging application, according to some embodiments.

[0422] [Figure 22C] 1 is a flow chart illustrating a method for proactively suggesting information about locations, events, or contacts according to some embodiments;

[0423] [Figure 23A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23E]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23F] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23G] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23H] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23I] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23J] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23K]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23L] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23M] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23N] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23O] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments.

[0424] [Figure 24A] 1 is a flow chart illustrating a method for proactively populating an application with information previously viewed by a user in a different application, according to some embodiments. [Figure 24B] 1 is a flow chart illustrating a method for proactively populating an application with information previously viewed by a user in a different application, according to some embodiments.

[0425] [Figure 25A] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25B] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25C] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25D] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25E] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25F] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25G] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25H] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25I] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25J] FIG. 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments.

[0426] [Figure 26A] 1 is a flow chart illustrating a method of proactively suggesting information previously viewed by a user in a first application for use in a second application according to some embodiments; [Figure 26B] 1 is a flow chart illustrating a method of proactively suggesting information previously viewed by a user in a first application for use in a second application according to some embodiments;

[0427] [Figure 27] 1 is a flowchart illustrating a method for proactively suggesting physical locations for use as destinations for vehicular routing, according to some embodiments.

[0428] [Figure 28] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting physical locations to use as destinations for vehicular route guidance, according to some embodiments.

[0429] [Figure 29] 1 is a flowchart illustrating a method for proactively suggesting paste actions according to some embodiments.

[0430] [Figure 30A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments; [Figure 30B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments; [Figure 30C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments; [Figure 30D]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments;

[0431] [Figure 31-1] 1 illustrates a mobile device configured to perform dynamic adjustment of the mobile device, according to some embodiments.

[0432] [Figure 31-2] 1 illustrates an example process for invoking a heuristic process according to some embodiments.

[0433] [Figure 31-3] 1 illustrates a process for adjusting settings on a mobile device using a heuristic process, according to some embodiments.

[0434] [Figure 31-4] 1 illustrates an example system for performing background fetch updates for an application, according to some embodiments.

[0435] [Figure 31-5] 1 illustrates peer prediction for determining the likelihood of a user invocation of an application on a mobile device 100 according to some embodiments.

[0436] [Figure 31-6] FIG. 1 is a flow diagram of an example process for predictively launching applications and performing background updates, according to some embodiments.

[0437] [Figure 31-7] FIG. 1 is a flow diagram of an example process for determining when to launch an application on a mobile device, according to some embodiments.

[0438] [Figure 31-8]FIG. 10 is a flow diagram illustrating state transitions for populating a trend table according to some embodiments.

[0439] [Figure 31-9] 1 is a block diagram illustrating a system for providing push notifications to a mobile device according to some embodiments.

[0440] [Figure 31-10] FIG. 10 is a flow diagram of an example process for performing a non-waking push at a push notification server, according to some embodiments.

[0441] [Figure 31-11] FIG. 10 is a flow diagram of an example process for performing a background update of an application in response to a low-priority push notification, according to some embodiments.

[0442] [Figure 31-12] FIG. 10 is a flow diagram of an example process for performing a background update of an application in response to a high priority push notification, according to some embodiments.

[0443] [Figure 31-13] 1 is a block diagram of an example system for performing background downloading and / or uploading of data on a mobile device, according to some embodiments.

[0444] [Figure 31-14] FIG. 10 is a flow diagram of an example process for performing background downloads and uploads, according to some embodiments.

[0445] [Figure 31-15] 1 illustrates an example graphical user interface (GUI) for enabling and / or disabling background updates for applications on a mobile device, according to some embodiments.

[0446] [Figure 31-16] 1 illustrates an example system for sharing data between peer devices according to some embodiments.

[0447] [Figure 31-17] 1 illustrates an example process for sharing data between peer devices according to some embodiments.

[0448] [Figure 32-1] FIG. 1 is a block diagram of an embodiment of a system for returning search results based on an entered query prefix, according to some embodiments.

[0449] [Figure 32-2] 1 is a flowchart of an embodiment of a process for determining query completion and relevant results based on an entered query prefix, according to some embodiments.

[0450] [Figure 32-3] FIG. 2 is a block diagram of an embodiment of an aggregator and multiple search domains, according to some embodiments.

[0451] [Figure 32-4] FIG. 1 is a diagram of an embodiment for a query completion search domain, according to some embodiments.

[0452] [Figure 32-5] FIG. 1 is a diagram of one embodiment of a map search domain. [Figure 32-6] 1 is a flowchart of an embodiment of a process for determining query completions from multiple search domains, according to some embodiments.

[0453] [Figure 32-7] 1 is a flowchart of an embodiment of a process for determining relevant results across multiple search domains from determined query completion, according to some embodiments.

[0454] [Figure 32-8]FIG. 1 is a block diagram of an embodiment of a system for incorporating user feedback into a feedback search index, according to some embodiments.

[0455] [Figure 32-9] 1 is a flowchart of an embodiment of a process for incorporating user feedback into a citation search index, according to some embodiments.

[0456] [Figure 32-10] 1 is a flowchart of an embodiment of a process for collecting user feedback during a user search session, according to some embodiments.

[0457] [Figure 32-11] 10 is a flowchart of an embodiment of a process for incorporating user feedback into a feedback index, according to some embodiments.

[0458] [Figure 32-12] 10 is a flowchart of an embodiment of a process for updating a results cache using user feedback, according to some embodiments.

[0459] [Figure 32-13] FIG. 2 is a block diagram of an embodiment of a federator that performs multi-domain searches using characterized query completion, according to some embodiments.

[0460] [Figure 32-14] 10 is a flowchart of an embodiment of a process for determining relevant results using a vocabulary service, according to some embodiments.

[0461] [Figure 32-15] 1 is a flowchart of an embodiment of a process for characterizing query completion, according to some embodiments.

[0462] [Figure 32-16]FIG. 10 is a block diagram of an embodiment of a completion module that determines query completions from multiple search domains, according to some embodiments.

[0463] [Figure 32-17] FIG. 10 is a block diagram of an embodiment of a results module that determines relevant results across multiple search domains from determined query completions, according to some embodiments.

[0464] [Figure 32-18] FIG. 10 is a block diagram of an embodiment of a feedback collection module that collects user feedback during a user search session, according to some embodiments.

[0465] [Figure 32-19] FIG. 10 is a block diagram of an embodiment of a feedback processing module that incorporates user feedback into a feedback index, according to some embodiments.

[0466] [Figure 32-20] FIG. 10 is a block diagram of an embodiment of a query results update module that uses user feedback to update a results cache, according to some embodiments.

[0467] [Figure 32-21] FIG. 10 is a block diagram of an embodiment of a feedback processing module that incorporates user feedback into a feedback index, according to some embodiments.

[0468] [Figure 32-22] FIG. 10 is a block diagram of an embodiment of a query results update module that uses user feedback to update a results cache, according to some embodiments.

[0469] [Figure 33-1] 1 illustrates, in block diagram form, a local search subsystem and a remote search subsystem on a computing device as known in the prior art.

[0470] [Figure 33-2] 1 illustrates, in block diagram form, a local search subsystem with local learning capabilities that can be used to improve results returned from a remote search application on a computing device, according to some embodiments.

[0471] [Figure 33-3] 1 illustrates, in block diagram form, a method for locally learning query features using local search queries, local results, and local feedback based on the local results, according to some embodiments.

[0472] [Figure 33-4] 1 illustrates, in block diagram form, a method for locally learning query features using search results returned from both local and remote search queries and local feedback on both local and remote search query results, according to some embodiments.

[0473] [Figure 33-5] 1 illustrates, in block diagram form, a method for locally learning query features sent by a remote service to a local device in response to a query passed to the remote service, according to some embodiments.

[0474] [Figure 33-6] 1 illustrates, in block diagram form, a method for receiving or determining, locally training, and using new features according to some embodiments.

[0475] [Figure 33-7] 1 illustrates an exemplary embodiment of a software stack usable in some embodiments of the present invention, according to some embodiments.

[0476] [Figure 34-5A] FIG. 1 illustrates a block diagram of an exemplary data architecture for suggested contacts, according to some embodiments.

[0477] [Figure 34-5B] FIG. 1 illustrates a block diagram of an exemplary data architecture for suggested calendar events, according to some embodiments.

[0478] [Figure 34-6A] 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6A illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate processes described below, including the processes illustrated in Figures 34-7A-34-13. [Figure 34-6B]

[0033] Figures 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6B illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in Figures 34-7A-34-13. [Figure 34-6C]

[0033] Figures 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6C illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate processes described below, including the processes illustrated in Figures 34-7A-34-13. [Figure 34-6D]

[0033] Figures 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6D illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in Figures 34-7A-34-13. [Figure 34-6E]

[0033] Figures 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6E illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate processes described below, including the processes illustrated in Figures 34-7A-34-13. [Figure 34-6F]

[0033] Figures 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6F illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in Figures 34-7A-34-13. [Figure 34-6G]

[0033] Figures 1A-1B, 2, and 3 illustrate exemplary user interfaces for providing suggested contacts and calendar events, according to some embodiments. Figures 1A-1B, 2, and 3 provide illustrations of exemplary devices for implementing the techniques for suggesting contact and event information described in this section. Figure 34-6G illustrates an exemplary user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate processes described below, including the processes illustrated in Figures 34-7A-34-13.

[0479] [Figure 34-7A] 1 illustrates a flow diagram of an exemplary process for generating suggested contacts according to some embodiments. [Figure 34-7B] 1 illustrates a flow diagram of an exemplary process for generating suggested contacts according to some embodiments.

[0480] [Figure 34-8A] 1 illustrates a flow diagram of an exemplary process for updating an existing contact with suggested items of contact information according to some embodiments. [Figure 34-8B] 1 illustrates a flow diagram of an exemplary process for updating an existing contact with suggested items of contact information according to some embodiments.

[0481] [Figure 34-9A] 1 illustrates a flow diagram of an exemplary process for displaying contacts with suggested contact information according to some embodiments. [Figure 34-9B] 1 illustrates a flow diagram of an exemplary process for displaying contacts with suggested contact information according to some embodiments.

[0482] [Figure 34-10] 1 illustrates a flow diagram of an exemplary process for displaying suggested contact information along with a message according to some embodiments.

[0483] [Figure 34-11A]1 illustrates a flow diagram of an exemplary process for generating suggested calendar events, according to some embodiments. [Figure 34-11B] 1 illustrates a flow diagram of an exemplary process for generating suggested calendar events, according to some embodiments.

[0484] [Figure 34-12] 1 illustrates a flow diagram of an exemplary process for displaying suggested event information along with a message according to some embodiments.

[0485] [Figure 34-13] 1 illustrates a flow diagram of an exemplary process for displaying multiple suggested contact or event information along with a message according to some embodiments.

[0486] [Figure 34-14] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments.

[0487] [Figure 34-15] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments.

[0488] [Figure 35-1] 3 is a flowchart of a method 35_100 for suggesting applications based on detected events according to some embodiments.

[0489] [Figure 35-2] 1 illustrates a segmentation process 35_200 according to some embodiments.

[0490] [Figure 35-3] 3 shows a decision tree 35_300 that may be generated by some embodiments.

[0491] [Figure 35-4]3 is a flowchart of a method 35_400 for suggesting applications to a user of a computing device based on an event, according to some embodiments.

[0492] [Figure 35-5A] 10 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments. [Figure 35-5B] 10 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments. [Figure 35-5C] 10 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments. [Figure 35-5D] 10 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments.

[0493] [Figure 35-6A] 10 illustrates parent models and sub-models resulting from segmentation according to some embodiments; [Figure 35-6B] 10 illustrates parent models and sub-models resulting from segmentation according to some embodiments;

[0494] [Figure 35-7] 3 illustrates an example architecture 35_700 for providing a user with a user interface for interacting with one or more applications, according to some embodiments.

[0495] [Figure 36-1] 1 is a flowchart of a method for identifying an application based on a trigger event, according to some embodiments.

[0496] [Figure 36-2] 1 illustrates a block diagram of a system for determining a trigger event, according to some embodiments.

[0497] [Figure 36-3]1 illustrates a block diagram of a system for identifying applications for a user based on a trigger event, according to some embodiments.

[0498] [Figure 36-4] 1 illustrates a block diagram of a system for identifying applications using multiple predictive models, according to some embodiments.

[0499] [Figure 36-5] 1 is a flowchart of a method for identifying an application based on a trigger event on a device according to some embodiments.

[0500] [Figure 36-6] FIG. 1 is a simplified diagram of a device having a user interface for a music application according to some embodiments.

[0501] [Figure 36-7A] 1 is a flowchart of a method for removing an identified application from a user interface according to some embodiments. [Figure 36-7B] 1 is a flowchart of a method for removing an identified application from a user interface according to some embodiments.

[0502] [Figure 37-1] 1 is a flowchart of a method 100 for suggesting destinations based on detected events according to some embodiments.

[0503] [Figure 37-2] 1 illustrates a block diagram of a system for determining a trigger event, according to some embodiments.

[0504] [Figure 37-3] 1 illustrates a block diagram of a system for identifying a destination based on a trigger event, according to some embodiments.

[0505] [Figure 37-4] 1 illustrates an example of suggesting destinations within a user interface for an email application according to some embodiments.

[0506] [Figure 37-5] 10 illustrates an example of suggesting destinations in a user interface for a search application according to some embodiments.

[0507] [Figure 37-6] 3 is a flowchart of a method 37-600 for suggesting recipients to a user of a computing device based on an event, according to some embodiments.

[0508] [Figure 37-7] 1 illustrates an example data flow for suggesting a destination according to some embodiments.

[0509] [Figure 37-8] 1 illustrates a block diagram of an interaction module according to some embodiments.

[0510] [Figure 37-9] 3 illustrates an example architecture 37-900 for providing a user with a user interface for suggesting destinations, according to some embodiments.

[0511] [Figure 38-1] 1 illustrates a block diagram of different components of a mobile computing device configured to implement various techniques described herein, according to some embodiments.

[0512] [Figure 38-2] 38-1 illustrates a method performed by the application prediction engine of FIG. 38-1 according to some embodiments.

[0513] [Figure 38-3] 2 illustrates a method performed by the search application of FIG. 1 according to some embodiments.

[0514] [Figure 38-4] 38-1 illustrates a conceptual diagram of an example user interface of the search application of FIG. 38-1, according to some embodiments.

[0515] [Figure 39-1] 1 illustrates a block diagram of different components of a mobile computing device configured to implement various techniques described herein, according to some embodiments.

[0516] [Figure 39-2] 39-1 (or FIG. 1A) in accordance with some embodiments.

[0517] [Figure 39-3A] 1 illustrates a method for high-level initialization and operation of a prediction engine according to some embodiments.

[0518] [Figure 39-3B] 1 illustrates a method for synchronously providing predictions in a prediction engine according to some embodiments.

[0519] [Figure 39-3C] 1 illustrates a method for asynchronously providing predictions in a prediction engine according to some embodiments.

[0520] [Figure 39-4A] 1 illustrates a method for a consumer application requesting to receive predictions synchronously, according to some embodiments.

[0521] [Figure 39-4B] 1 illustrates a method for a consumer application to register to receive predictions asynchronously, according to some embodiments.

[0522] [Figure 39-5A]1 illustrates a method for managing prediction engine registrations at a prediction engine center according to some embodiments.

[0523] [Figure 39-5B] 1 illustrates a method for synchronously providing predictions to consumer applications at a prediction engine center, according to some embodiments.

[0524] [Figure 39-5C] 1 illustrates a method for asynchronously providing predictions to consumer applications at a prediction engine center, according to some embodiments.

[0525] [Figure 40-1] 1 is a block diagram of an example system for monitoring, predicting, and notifying a context client of changes in a current context of a computing device, according to some embodiments.

[0526] [Figure 40-2A] 1 illustrates an example of context items that may constitute a current context according to some embodiments.

[0527] [Figure 40-2B] 10 illustrates an example of a new context item being added to a current context according to some embodiments.

[0528] [Figure 40-3] 1 illustrates an example callback predicate database according to some embodiments.

[0529] [Figure 40-4] 10 is a graph illustrating example state changes over time associated with a context item according to some embodiments;

[0530] [Figure 40-5] 1 is a graph illustrating an example event stream associated with a context item according to some embodiments;

[0531] [Figure 40-6] 1 illustrates an example historical event stream database according to some embodiments.

[0532] [Figure 40-7] 1 is a block diagram of an example system for providing context callback notifications to a requesting client according to some embodiments.

[0533] [Figure 40-8A] FIG. 1 is a block diagram of an example system illustrating restarting a terminated request client, according to some embodiments.

[0534] [Figure 40-8B] FIG. 1 is a block diagram of an example system illustrating restarting a terminated request client, according to some embodiments.

[0535] [Figure 40-9A] FIG. 1 is a block diagram of an example system illustrating restarting a terminated context daemon, according to some embodiments.

[0536] [Figure 40-9B] FIG. 1 is a block diagram of an example system illustrating restarting a terminated context daemon, according to some embodiments.

[0537] [Figure 40-10A] FIG. 10 is a block diagram of an example system illustrating restarting a terminated context daemon and request client, according to some embodiments.

[0538] [Figure 40-10B] FIG. 10 is a block diagram of an example system illustrating restarting a terminated context daemon and request client, according to some embodiments.

[0539] [Figure 40-11]FIG. 1 is a block diagram of an example system configured to restart a client and / or a context daemon based on device state information received by an activation daemon, according to some embodiments.

[0540] [Figure 40-12A] FIG. 1 is a block diagram of an example system illustrating restarting a context daemon using a launch daemon, according to some embodiments.

[0541] [Figure 40-12B] FIG. 1 is a block diagram of an example system illustrating restarting a context daemon using a launch daemon, according to some embodiments.

[0542] [Figure 40-13A] FIG. 1 is a block diagram of an example system illustrating restarting a requesting client using a launch daemon, according to some embodiments.

[0543] [Figure 40-13B] FIG. 1 is a block diagram of an example system illustrating resuming a requesting client using an activation daemon, according to some embodiments.

[0544] [Figure 40-14] 1 is a graph illustrating an example of slot-wise averaging to predict future events, according to some embodiments;

[0545] [Figure 40-15] 10 illustrates an example graph showing slot weighting, according to some embodiments.

[0546] [Figure 40-16A] 1 is a graph illustrating an example method for predicting future context according to some embodiments;

[0547] [Figure 40-16B]1 is a graph illustrating an example method for converting slot-wise probabilities into a probability curve, according to some embodiments.

[0548] [Figure 40-17] 1 illustrates an example event stream including predicted future events according to some embodiments.

[0549] [Figure 40-18] FIG. 10 is a flow diagram of an example process for notifying a client of a context change on a computing device, according to some embodiments.

[0550] [Figure 40-19] FIG. 10 is a flow diagram of an example process for restarting a context daemon to service a callback request according to some embodiments.

[0551] [Figure 40-20] FIG. 10 is a flow diagram of an example process for restarting a callback client to receive callback notifications, according to some embodiments.

[0552] [Figure 40-21] FIG. 1 is a flow diagram of an example process for predicting future events based on past context information, according to some embodiments.

[0553] [Figure 40-22] FIG. 10 is a flow diagram of an example process for servicing a sleep context callback request according to some embodiments.

[0554] [Figure 41-1] FIG. 1 is a block diagram of one embodiment of a system for indexing application state for use in a local device search index.

[0555] [Figure 41-2]FIG. 1 is a block diagram of an embodiment of a system for searching application state using an on-device application state search index.

[0556] [Figure 41-3] FIG. 10 is a block diagram of an embodiment of a user interface displaying application status query results among other query results.

[0557] [Figure 41-4A] FIG. 10 is a flow diagram of one embodiment of a process for indexing application state received from multiple different applications on a device.

[0558] [Figure 41-4B] FIG. 1 is a flow diagram of one embodiment of a process for determining query results for a query using an application state index.

[0559] [Figure 41-5] FIG. 1 is a flow diagram of one embodiment of a process for receiving and presenting application state as part of a query result.

[0560] [Figure 41-6] FIG. 1 is a block diagram of one embodiment of a system for indexing application state for use in a remote search index.

[0561] [Figure 41-7] FIG. 1 is a block diagram of one embodiment of a system for searching application state using a remote application state search index.

[0562] [Figure 41-8] FIG. 10 is a flow diagram of one embodiment of a process for adding application state to an application state index.

[0563] [Figure 41-9]FIG. 10 is a flow diagram of one embodiment of a process for exporting application state to an application state indexing service.

[0564] [Figure 41-10] 1 is a flow diagram of one embodiment of a process for performing a query search using an application state index.

[0565] [Figure 41-11] FIG. 1 is a flow diagram of one embodiment of a process for receiving and presenting application state as part of a query result.

[0566] [Figure 41-12] FIG. 1 is a block diagram of one embodiment of a system for indexing application state views for use in a remote search index.

[0567] [Figure 41-13] FIG. 2 is a block diagram of one embodiment of an application view.

[0568] [Figure 41-14] 1 is a flowchart of one embodiment of a process for generating an application state view using application state.

[0569] [Figure 41-15] 1 is a flowchart of one embodiment of a process for receiving and presenting application state, including application state views, as part of a query result.

[0570] [Figure 42] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 43] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 44] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 45]FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 46] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 47] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 48] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 49] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 50] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 51] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 52] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 53] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 54] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 55] FIG. 1 is a functional block diagram of an electronic device in accordance with some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0571] As described above and in more detail below, there is a growing demand for electronic devices with faster, more efficient methods and interfaces for quickly accessing applications and desired functions within those applications. In particular, there is a growing demand for devices that assist users in providing proactive assistance by eliminating repetitive tasks and identifying and providing relevant information before the user explicitly requests it. Furthermore, there is a growing demand for quickly accessing applications and desired functions within those applications at specific time periods (e.g., accessing a calendar application after waking up each morning) and specific locations (e.g., accessing a music application at the gym). Disclosed herein are new methods and interfaces that address these demands and provide users with a way to quickly access applications and functions within those applications at those specific locations and time periods. Such methods and interfaces optionally complement or replace existing methods for accessing applications. Such methods and interfaces reduce the cognitive burden on users and create a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and extend the time between battery charges. Furthermore, such methods and interfaces help extend the life of touch-sensitive displays by reducing the number of inputs required (e.g., instead of continuously and aimlessly tapping a touch-sensitive display to identify desired information, the methods and interfaces disclosed herein proactively provide information without requiring user input).

[0572] Below, FIGS. 1A-1E and 2 provide descriptions of example devices. FIGS. 10 and 11 provide functional block diagrams of example electronic devices. FIGS. 3A-3B and 4A-4B are block diagrams of example data structures used to proactively identify and surface related content (these data structures are used in the methods described in connection with FIGS. 6A-6B and with reference to FIGS. 8A-8B). FIG. 5 is a block diagram illustrating an example system for establishing trigger conditions used to proactively identify and surface related content (this example system is used in the methods described in connection with FIGS. 6A-6B and with reference to FIGS. 8A-8B). FIGS. 6A-6B are a flowchart illustrating a method for proactively identifying and surfacing related content. FIGS. 7A-7B are schematics of a touch-sensitive display used to illustrate example user interfaces and gestures for proactively identifying and surfacing related content. FIGS. 8A-8B are a flowchart illustrating a method for proactively identifying and surfacing related content. Figures 9A-9D are schematics of touch-sensitive displays used to illustrate additional user interfaces for proactively identifying and surfacing related content. Figures 3A-3B, 4A-4B, 5, and 7A-7B are used to illustrate the methods and / or processes of Figures 6A-6B. Figures 3A-3B, 4A-4B, 5, and 9A-9D are used to illustrate the methods and / or processes of Figures 8A-8B.

[0573] Figures 10A-10C are flowcharts illustrating methods for proactively suggesting search queries based on content currently displayed on an electronic device with a touch-sensitive display. Figures 11A-11J are schematics of a touch-sensitive display used to illustrate a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display. Figures 11A-11J are used to illustrate the methods and / or processes of Figures 10A-10C.

[0574] Figure 12 is a flow chart illustrating a method for transitioning to a search mode based on heuristics. Figures 13A-13B are schematics of a touch-sensitive display used to illustrate a user interface for transitioning to a search mode based on heuristics. Figures 13A-13B are used to illustrate the method and / or process of Figure 12.

[0575] Figure 14 is a flowchart illustrating a method for proactively providing vehicle location on an electronic device with a touch-sensitive display, according to some embodiments. Figures 15A-15B are schematics of a touch-sensitive display used to illustrate a user interface for proactively providing vehicle location, according to some embodiments. Figures 15A-15B are used to illustrate the method and / or process of Figure 14.

[0576] 16A-16B are flowcharts illustrating a method for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. Figures 17A-17E are schematics of a touch-sensitive display used to illustrate a user interface for proactively providing nearby points of interest (POI) information in response to a search query, according to some embodiments. Figures 16A-16B are used to illustrate the methods and / or processes of Figures 17A-17E.

[0577] Figures 18A-18B are flowcharts illustrating a method for extracting content items from an audio communication and interacting with the extracted content items, according to some embodiments. Figures 19A-19F are schematics of a touch-sensitive display used to illustrate a user interface for displaying and interacting with content items extracted from an audio communication, according to some embodiments. Figures 19A-19F are used to illustrate the methods and / or processes of Figures 18A-18B.

[0578] Figure 20 is a flowchart illustrating a method for determining that an audio communication includes speech identifying a physical location and populating an application with information about the physical location, according to some embodiments. Figures 21A-21B are schematics of a touch-sensitive display used to show a user interface for determining that an audio communication includes speech identifying a physical location and populating an application with information about the physical location, according to some embodiments. Figures 19A-19F and 21A-21B are used to illustrate the method and / or process of Figure 20.

[0579] Figures 22A-22B are flowcharts illustrating a method for proactively suggesting physical locations for use in a messaging application, according to some embodiments. Figures 23A-23O are schematics of a touch-sensitive display used to show a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. Figures 23A-23O are used to illustrate the methods and / or processes of Figures 22A-22B.

[0580] Figure 22C is a flow chart illustrating a method for proactively suggesting information about locations, events, or contacts, according to some embodiments. Figures 23A-23O are used to illustrate the method and / or process of Figure 22C.

[0581] Figures 24A-24B are flowcharts illustrating a method for proactively populating an application with information previously viewed by a user in a different application, according to some embodiments. Figures 25A-25J are schematics of a touch-sensitive display used to show a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. Figures 25A-25J are used to illustrate the methods and / or processes of Figures 24A-24B.

[0582] 26A-26B are flow charts illustrating a method for proactively suggesting information previously viewed by a user in a first application for use in a second application, according to some embodiments. Figures 25A-25J are used to illustrate the method and / or process of Figures 26A-26B.

[0583] Figure 27 is a flowchart illustrating a method for proactively suggesting physical locations for use as destinations for vehicular route guidance, according to some embodiments. Figure 28 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting physical locations for use as destinations for vehicular route guidance, according to some embodiments. Figure 28 is used to illustrate the method and / or process of Figure 27.

[0584] Figure 29 is a flowchart illustrating a method for proactively suggesting paste actions, according to some embodiments. Figures 30A-30D are schematics of a touch-sensitive display used to show a user interface for proactively suggesting paste actions, according to some embodiments. Figures 30A-30D are used to illustrate the method and / or process of Figure 29.

[0585] Sections 1-11 within the "Additional Description of the Invention" section provide additional details that complement the description provided in connection with Figures 1A-30D.

[0586] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments being described. However, it will be apparent to those skilled in the art that the various embodiments being described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail as not to unnecessarily obscure aspects of the embodiments.

[0587] In this specification, terms such as first, second, etc. are used to describe various elements in some examples, but it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first contact can be referred to as a second contact, and similarly, a second contact can be referred to as a first contact, without departing from the scope of the various embodiments being described. Although a first contact and a second contact are both contacts, they are not the same contact.

[0588] The terminology used in the description of the various embodiments set forth herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the various described embodiments and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "and / or" should also be understood to refer to and include any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0589] As used herein, the term "if" is, optionally, interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are, optionally, interpreted to mean "upon determining," "in response to determining," "upon detecting (the stated condition or event)," or "in response to detecting (the stated condition or event)," depending on the context.

[0590] The disclosure herein refers interchangeably to detecting a touch input on, at, across, on, or substantially within a particular user interface element or a particular portion of a touch-sensitive display. As used herein, a touch input detected "at" a particular user interface element may also be a touch input detected "on," "across," "on," or "substantially within" the same user interface element, depending on the context. In some embodiments, and as described in more detail below, the desired sensitivity level for detecting a touch input is configured by a user of the electronic device (e.g., the user may determine (and configure and operate the electronic device) that a touch input should be detected only if the touch input is entirely within a user interface element).

[0591] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile telephone, that also includes other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, IPHONE®, IPOD TOUCH®, and IPAD® devices from APPLE Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch-sensitive displays and / or touchpads), are also optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touch-sensitive displays and / or touchpads).

[0592] In the following description, we describe an electronic device that includes a display and a touch-sensitive surface, but it should be understood that the electronic device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0593] The device typically supports a variety of applications such as one or more of a drawing application, a presentation application, a word processing application, a website creation application, a disc authoring application, a spreadsheet application, a gaming application, a phone application, a video conferencing application, an email application, an instant messaging application, a health / fitness application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0594] Various applications running on the device optionally use at least one common physical user-interface device, such as a touch-sensitive surface. One or more features of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or changed for each application and / or within the corresponding application. In this way, the common physical architecture (such as the touch-sensitive surface) of the device optionally supports various applications with user interfaces that are intuitive and transparent to the user.

[0595] Attention is now directed to embodiments of portable electronic devices with touch-sensitive displays. FIG. 1A is a block diagram illustrating portable multifunction device 100 (also referred to herein, interchangeably, as electronic device 100 or device 100) with touch-sensitive display 112, according to some embodiments. Touch-sensitive display 112 may conveniently be referred to as a "touchscreen" and may also be known or referred to as a touch-sensitive display system. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), controller 120, one or more processing units (CPUs) 122, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more light sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of a contact on device 100 (e.g., a touch-sensitive surface such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating a tactile output on device 100 (e.g., generating a tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or a touchpad of device 100). These components optionally communicate via one or more communication buses or signal lines 103.

[0596] As used in this specification and in the claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or a surrogate (proxy) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four distinct values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors positioned beneath or adjacent to the touch-sensitive surface are optionally used to measure force at various points on the touch-sensitive surface. In some implementations, the force measurements of multiple force sensors are combined (e.g., weighted average) to determine an estimate of the contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or change in the contact area detected on the touch-sensitive surface, the capacitance and / or change in the capacitance of the touch-sensitive surface proximate the contact, and / or the resistance and / or change in the capacitance of the touch-sensitive surface proximate the contact are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measure of the force or pressure of the contact is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is stated in units corresponding to the surrogate measure). In some implementations, the surrogate measure of the force or pressure of the contact is converted to an estimated force or pressure, and this estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).

[0597] As used herein and in the claims, the term “tactile output” refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of gravity of the device, that is detected by a user with the user's sense of touch. For example, in a situation where a device or a component of the device is in contact with a touch-sensitive surface of a user (e.g., the fingers, palm, or other part of the user's hand), the tactile output produced by the physical displacement is interpreted by the user as a tactile sensation corresponding to a perceived change in a physical property of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a “downclick” or “upclick” of a physical actuator button. In some cases, a user will feel a tactile sensation such as a “downclick” or “upclick” even when no movement of a physical actuator button associated with the touch-sensitive surface is physically pressed (e.g., displaced) by the user's action. As another example, movement of a touch-sensitive surface is optionally interpreted or perceived by a user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. While such user interpretation of touch depends on the user's personal sensory perception, there are many sensory perceptions of touch that are common to the majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., "upclick," "downclick," "roughness"), unless otherwise specified, the generated tactile output corresponds to a physical displacement of the device, or a component of the device, that produces the described sensory perception for a typical (or average) user.

[0598] It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of components. The various components shown in Figure 1A are implemented in hardware, software, or a combination of hardware and software, including one or more signal processing circuits and / or application specific integrated circuits.

[0599] Memory 102 optionally includes high-speed random-access memory (e.g., DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices), and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory 102 optionally includes one or more storage devices located remotely from processor(s) 122. Access to memory 102 by other components of device 100, such as CPU 122 and peripherals interface 118, is optionally controlled by controller 120.

[0600] A peripherals interface 118 can be used to couple input and output peripherals of the device to the CPU 122 and memory 102. The one or more processors 122 operate or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions for the device 100 and to process data.

[0601] In some embodiments, peripheral interface 118, CPU 122, and controller 120 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0602] RF (radio frequency) circuitry 108 transmits and receives RF signals, also known as electromagnetic signals. RF circuitry 108 converts electrical signals to or from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. RF circuitry 108 optionally includes well-known circuitry for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, etc. RF circuitry 108 optionally communicates via wireless communication with the Internet, also known as the World Wide Web (WWW), an intranet, and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs), and other devices. Wireless communication optionally uses any of a number of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM)®, Enhanced Data GSM Environment (EDGE)®, High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth®, and / or Wireless Fidelity (Wi-Fi)® (e.g., IEEE 802.1 la, IEEE 802.1 lb, IEEE 802.1 lg, and / or IEEE 802.1 ln).

[0603] Audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and device 100. Audio circuit 110 receives audio data from peripherals interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves audible to humans. Audio circuit 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuit 110 converts the electrical signals into audio data and transmits the audio data to peripherals interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to memory 102 and / or RF circuit 108 by peripherals interface 118. In some embodiments, audio circuit 110 further comprises a headset jack. The headset jack provides an interface between audio circuitry 110 and a removable audio input / output peripheral, such as an output-only headphone or a headset with both an output (e.g., single or double ear headphones) and an input (e.g., a microphone).

[0604] I / O subsystem 106 connects input / output peripherals of device 100, such as touchscreen 112 and other input control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes one or more input controllers 160 for display controller 156, light sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and other input or control devices. One or more input controllers 160 receive electrical signals from and send electrical signals to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, input controller(s) 160 are optionally coupled to any (or none) of a keyboard, infrared port, USB port, and pointer device such as a mouse. The one or more buttons optionally include up / down buttons for adjusting the volume of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button.

[0605] Touch-sensitive display 112 provides an input and output interface between the device and a user. Display controller 156 receives and / or sends electrical signals to touchscreen 112. Touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

[0606] Touchscreen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic and / or tactile contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction sets in memory 102) detects contacts (and any movement or cessation of contact) on touchscreen 112 and translates the detected contacts into interactions with user interface objects (e.g., one or more softkeys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the point of contact between touchscreen 112 and the user corresponds to the area under the user's finger.

[0607] Touchscreen 112 optionally uses LCD (liquid crystal display), LPD (light emitting polymer display), or LED (light emitting diode) or OLED (organic light emitting diode) technology, although other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally detect contact and any movement or cessation of contact using any of a number of now known or later developed touch sensing technologies, including, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112. In one example embodiment, projected mutual capacitance sensing technology is used, such as that found in the IPHONE®, IPOD TOUCH®, and IPAD® from APPLE Inc. of Cupertino, California.

[0608] Touchscreen 112 optionally has a video resolution greater than 400 dpi. In some embodiments, touchscreen 112 has a video resolution of at least 600 dpi. In other embodiments, touchscreen 112 has a video resolution of at least 1000 dpi. A user optionally contacts touchscreen 112 using any suitable object, such as a stylus, finger, or finger. In some embodiments, the user interface is designed to function primarily with finger-based touches and gestures. In some embodiments, the device translates finger-based input into precise pointer / cursor position or commands to perform the user's desired action.

[0609] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating certain functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touchscreen 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0610] Device 100 also includes a power system 162 for providing power to the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with generating, managing, and distributing electrical power within a portable device.

[0611] Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor coupled to light sensor controller 158 in I / O subsystem 106. Light sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. Light sensor 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), light sensor 164 optionally captures still images or video. In some embodiments, a light sensor is located on the back of device 100, opposite touchscreen 112 on the front of the device, to enable the touch-sensitive display as a viewfinder for still image and / or video capture. In some embodiments, another light sensor is located on the front of the device to optionally obtain an image of the user for video conferencing while the user views other video conference participants on the touch-sensitive display.

[0612] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows contact intensity sensors coupled to intensity sensor controller 159 in I / O subsystem 106. Contact intensity sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensors 165 receive contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is located on or proximate to the touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite touchscreen 112, which is located on the front of device 100.

[0613] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I / O subsystem 106. In some embodiments, when the multifunction device is placed near a user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and touchscreen 112 is disabled.

[0614] Device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows tactile output generators coupled to haptic feedback controller 161 in I / O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output-generating components (e.g., components that convert electrical signals into tactile output on the device). Contact intensity sensor 165 receives haptic feedback generation instructions from haptic feedback module 133 and generates tactile outputs on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is located on or proximate to the touch-sensitive surface (e.g., touch-sensitive display system 112) and generates a tactile output, optionally by moving the touch-sensitive surface vertically (e.g., in / out of the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100, opposite touch-sensitive display 112, which is located on the front of device 100.

[0615] Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows accelerometer 168 coupled to peripherals interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on the touch-sensitive display in portrait or landscape orientation based on analysis of data received from the one or more accelerometers. In addition to accelerometer(s) 168, device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the location and orientation (e.g., portrait or landscape) of device 100.

[0616] In some embodiments, the software components stored in memory 102 include operating system 126, proactive module 163 (optionally including one or more of application usage data table 335, trigger condition table 402, trigger establishment module 163-1, and / or usage data collection module 163-2), communication module (or instruction set) 128, contact / motion module (or instruction set) 130, graphics module (or instruction set) 132, text input module (or instruction set) 134, global positioning system (GPS) module (or instruction set) 135, and applications (or instruction sets) 136. Further, in some embodiments, as shown in FIG. 1A , memory 102 stores device / global internal state 157. Device / global internal state 157 includes one or more of the following: active application state, which indicates which application, if any, is currently active; display state, which indicates which applications, views, or other information are occupying various regions of touch-sensitive display 112; sensor state, which includes information obtained from the device's various sensors and input control devices 116; and position information regarding the device's position and / or attitude (e.g., the device's orientation).

[0617] Operating system 126 (e.g., an embedded operating system such as Darwin®, RTXC®, LINUX®, UNIX®, OS X®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0618] Communications module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by RF circuitry 108 and / or external port 124. External port 124 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) is adapted to couple to other devices directly or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, similar to, and / or compatible with the 30-pin connector used on some embodiments of IPOD devices from APPLE Inc. In other embodiments, the external port is a multi-pin (e.g., 8-pin) connector that is the same as, similar to, and / or compatible with the 8-pin connector used on the LIGHTNING connector from APPLE Inc.

[0619] Contact / motion module 130, optionally in conjunction with display controller 156, detects contact with touch screen 112 and other touch-sensing devices (e.g., a touchpad or physical click wheel). Contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact occurs (e.g., detecting a finger-down event), determining the intensity of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is contact movement and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-drag events), and determining whether the contact has stopped (e.g., detecting a finger-up event or an interruption of the contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by the series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions are optionally applied to a single contact (e.g., a single finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, contact / motion module 130 and display controller 156 detect contacts on the touchpad.

[0620] In some embodiments, contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has selected or “clicked” on an affordance). In some embodiments, at least a subset of the intensity thresholds are determined according to software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and may be adjusted without modifying the physical hardware of device 100). For example, the mouse “click” threshold of a trackpad or touch-sensitive display can be set to any of a wide range of pre-defined thresholds without modifying the trackpad or touch-sensitive display hardware. Furthermore, in some implementations, a user of the device is provided with a software setting to adjust one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once via a system-level click “intensity” parameter).

[0621] Contact / motion module 130 optionally detects gesture input by a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timing, and / or intensity of detected contact). Thus, a gesture is optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger down event (e.g., at an icon location), followed by detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger down event. As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger down event, followed by detecting one or more finger drag events, followed, in some embodiments, by detecting a finger lift (lift-off) event.

[0622] Graphics module 132 includes various known software components for rendering and displaying graphics on touchscreen 112 or other display, including components for modifying the visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, video, animation, etc.

[0623] In some embodiments, graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. Graphics module 132 receives one or more codes specifying the graphics to be displayed, including coordinate data and other graphic characteristic data, as needed, from an application or the like, and then generates screen image data to output to display controller 156.

[0624] The tactile feedback module 133 includes various software components for generating instructions used by the tactile output generator(s) 167 to generate tactile outputs at one or more locations on the device 100 in response to user interaction with the device 100.

[0625] Text input module 134 is optionally a component of graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts module 137, email client module 140, IM module 141, browser module 147, and any other application requiring text input).

[0626] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to a phone 138 application for use in location-based dialing, to a camera 143 application as image / video metadata, and to applications that provide location-based services such as a weather widget, a local yellow pages widget, and a maps / navigation widget).

[0627] Applications (“apps”) 136 optionally include the following modules (or sets of instructions), or a subset or superset thereof: • a contacts module 137 (sometimes called an address book or contact list); ●Telephone module 138, ●Videoconferencing module 139, ● an email client module 140; ● Instant messaging (IM) module 141; ● Health module 142, a camera module 143 for still and / or video images, ● Image management module 144; ● Browser module 147, ●Calendar module 148, a widget module 149, optionally including one or more of a weather widget 149-1, a stocks widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widgets 149-6; ● Search module 151, a video and music player module 152, optionally consisting of a video player module and a music player module; ● Memo module 153, Map module 154, and / or ●Online video module 155.

[0628] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, website creation applications, disk authoring applications, spreadsheet applications, JAVA® enabled applications, encryption, digital rights management, voice recognition, a widget creator module 149-6 for creating user created widgets, and voice duplication.

[0629] Contacts module 137, in conjunction with touch screen 112, display controller 156, touch module 130, graphics module 132, and text input module 134, is optionally used to manage an address book or contact list (e.g., stored in memory 102 or in contacts module 137 in memory 370), including adding name(s) to the address book, deleting name(s) from the address book, associating phone number(s), email address(es), postal address(es), or other information with names, associating images with names, categorizing and sorting names, providing phone numbers or email addresses to initiate and / or facilitate communication via telephone module 138, videoconferencing module 139, email module 140, or IM module 141, etc.

[0630] Telephone module 138, in conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, is optionally used to enter a series of characters corresponding to a telephone number, access one or more telephone numbers in address book 137, modify an entered telephone number, dial a corresponding telephone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As noted above, wireless communication optionally uses any of a number of communication standards, protocols, and technologies.

[0631] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, light sensor 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, videoconferencing module 139 contains executable instructions for initiating, conducting, and terminating a videoconference between a user and one or more other participants in accordance with the user's instructions.

[0632] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 contains executable instructions for composing, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to compose and send emails with still or video images captured by camera module 143.

[0633] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant message module 141 includes executable instructions for entering characters corresponding to an instant message, modifying previously entered characters, sending the corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), and receiving and displaying the received instant message. In some embodiments, sent and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0634] In conjunction with the RF circuitry 108, touchscreen 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and video and music player module 146, the health module 142 includes executable instructions for creating workouts (e.g., in terms of time, distance, and / or calorie burn goals), communicating with workout sensors (such as a watch or sports device such as a pedometer), receiving workout sensor data, calibrating sensors used to monitor workouts, selecting and playing music for workouts, and displaying, storing, and transmitting workout data.

[0635] Camera module 143, in conjunction with touch screen 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, and image management module 144, contains executable instructions to capture still images or video (including video streams) and store them in memory 102, change characteristics of the still images or video, or delete the still images or video from memory 102.

[0636] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing) or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.

[0637] Browser module 147, in conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, contains executable instructions for browsing the Internet according to user instructions, including retrieving, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0638] Calendar module 148, in conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, includes executable instructions for creating, displaying, modifying, and storing calendars and data associated with calendars (e.g., calendar items, to-do lists, etc.) in accordance with user instructions.

[0639] In conjunction with RF circuitry 108, touchscreen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget modules 149 are mini-applications (e.g., weather widget 149-1, stocks widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) that are optionally downloaded and used by a user, or mini-applications (e.g., user-created widget 149-6) that are created by a user. In some embodiments, a widget includes an HTML (HyperText Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, a widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0640] A widget creator module (not shown), in conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, is optionally used by a user to create a widget (e.g., turn a user-specified portion of a web page into a widget).

[0641] In conjunction with touchscreen 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 includes executable instructions for searching memory 102 for text, music, sound, images, video, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with a user's instructions. In some embodiments, search module 151 further includes executable instructions for displaying a search entry portion and a prediction portion (e.g., search entry portion 920 and prediction portion 930, as shown in FIG. 9B and described in more detail below in conjunction with FIGS. 6A-9C). In some embodiments, search module 151, in conjunction with proactive module 163, also populates the prediction portion with affordances for suggested or predicted people, actions within applications, applications, nearby places, and / or news articles before receiving user input in the search entry portion (as described in more detail below in conjunction with FIGS. 3A-9C).

[0642] Video and music player module 152, in conjunction with touchscreen 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, contains executable instructions that enable a user to download and play pre-recorded music or other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing videos (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an IPOD® from APPLE Inc.

[0643] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, notes module 153 contains executable instructions for creating and managing notes, to-do lists, and the like, in accordance with user instructions.

[0644] In conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 can optionally be used to receive, display, modify, and store maps and data associated with maps (e.g., driving directions, data about businesses and other points of interest at or near a particular location, and other location-based data) in accordance with user instructions.

[0645] Online video module 155, in conjunction with touchscreen 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, contains instructions that enable a user to access, browse, receive (e.g., by streaming and / or downloading), and play (e.g., on the touchscreen or on an external display connected via external port 124) online videos in one or more file formats, such as H.264, and send and otherwise manage emails containing links to particular online videos. In some embodiments, instant messaging module 141 is used to send links to particular online videos, rather than email client module 140.

[0646] 1A , portable multifunction device 100 also includes proactive module 163 for proactively identifying and surfacing relevant content (e.g., surfacing user interface objects corresponding to actions within an application (e.g., a UI object for playing a playlist in a music app) on the lock screen or in a search interface). Proactive module 163 optionally includes the following modules (or sets of instructions), or subsets or supersets: ● Application Usage Table 335 ●Trigger condition table 402 ●Trigger establishment module 163-1 ●Usage data collection module 163-2 ● Proactive Suggestion Module 163-3 ●(Voice communication) content extraction module 163-4

[0647] The application usage table 335 and usage data collection module 163-2, in conjunction with the applications 136, GPS module 135, operating system 126, I / O subsystem 106, RF circuitry 108, external port 124, proximity sensor 166, audio circuitry 110, accelerometer 168, speaker 111, microphone 113, and peripherals interface 118, receive and / or store application usage data (e.g., from the components of device 100 identified in FIG. 1A above). In some embodiments, application usage is reported to the usage data collection module 163-2, which is then stored in the application usage table 335. In some embodiments, the application usage data includes all (or most importantly, related, or predictive) contextual usage information corresponding to a user's use of a particular application 136. In some embodiments, each particular application stores usage data while a user interacts with that application, and the usage data is then reported to application usage data table 335 for storage (e.g., in FIG. 3B , usage data 193 for particular application 136-1 includes all sensor readings, in-application actions performed, device connectivity information, etc., and this usage data 193 is sent to application usage table 335 for storage as records in the table). For example, while a user interacts with browser module 147, application usage data is received by application usage table 335, which receives and stores all contextual usage information including device 100's current...

Claims

1. In an electronic device including one or more processors, a memory, a touch-sensitive surface, a display, and a position sensor, without receiving any instructions from a user of the electronic device, monitoring the geographic location of the electronic device using the location sensor; determining, based on the monitored geographic location, that the electronic device is within a threshold distance of a predetermined type of point of interest; In response to determining that the electronic device is within the threshold distance of the point of interest, identifying at least one activity that is currently popular at the point of interest; obtaining information about the point of interest, the information including obtaining information about at least one currently popular activity at the point of interest; detecting a first input via the touch-sensitive surface, the first input, when detected, causing the electronic device to enter a search mode; transitioning to the search mode in response to detecting the first input; and transitioning to the search mode comprises presenting, prior to receiving any user input in a search interface of the search mode, via the display, an affordance including: (i) the information regarding the at least one activity; and (ii) an indication that the at least one activity has been identified as currently popular in the point of interest; A method comprising:

2. Detecting a second input; In response to detecting the second input, updating the affordance to include available information regarding a current activity at a second point of interest that is different from the point of interest; wherein the point of interest is within the threshold distance of the electronic device. The method of claim 1.

3. The affordance further includes a selectable category of points of interest, and the method further comprises: Detecting a selection of a corresponding selectable category; In response to detecting the selection, updating the affordance to include information about additional points of interest located within a second threshold distance of the electronic device; The method of claim 1 or 2, further comprising:

4. The method of claim 1 , wherein the point of interest is an amusement park and the obtained information includes current wait times for rides at the amusement park.

5. The method of claim 4 , wherein the obtained information includes information regarding waiting times for rides located within a predetermined distance of the electronic device.

6. The method of claim 1 , wherein the point of interest is a restaurant and the obtained information includes information about popular menu items at the restaurant.

7. The method of claim 6 , wherein the obtained information is obtained from a social network associated with the user of the electronic device.

8. The method of claim 1 , wherein the point of interest is a movie theater and the obtained information includes information about showtimes at the movie theater.

9. The method of claim 8 , wherein the obtained information is obtained from a social network associated with the user of the electronic device.

10. 10. The method of claim 1, wherein after the electronic device is unlocked, the affordance is available in response to a substantially horizontal swipe on an initial page of a home screen of the electronic device.

11. A computer program for causing an electronic device to carry out the method of any one of claims 1 to 10.

12. 1. An electronic device comprising: a memory storing the computer program according to claim 11; one or more processors capable of executing the computer programs stored in the memory; and configured to communicate with the touch-sensitive surface, the display, and the position sensor.

13. An electronic device configured to communicate with a touch-sensitive surface, a display, and a position sensor, the electronic device comprising means for performing the method of any one of claims 1 to 10.

Images