Systems and methods for proactively identifying and surfacing relevant content on touch-sensitive devices

Abstract

To proactively identify and surface relevant content on an electronic device with a touch-sensitive display.SOLUTION: A method includes executing, on an electronic device, an application in response to an instruction from a user. While executing the application, the method further includes collecting usage data. The usage data at least includes one or more actions performed by the user within the application. The method also includes automatically, without human intervention, obtaining at least one trigger condition based on the collected usage data and associating the at least one trigger condition with a particular action of the one or more actions performed by the user within the application. Upon determining that the at least one trigger condition has been satisfied, the method includes providing an indication to the user that the particular action associated with the trigger condition is available.SELECTED DRAWING: Figure 11D

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE

[0002] The embodiments disclosed herein relate generally to electronic devices having touch-sensitive displays, and more particularly to systems and methods for proactively identifying and surfacing relevant content on electronic devices in communication with a display and a touch-sensitive surface. [Background technology]

[0002] Handheld electronic devices with touch-sensitive displays are becoming widespread. Users of these widespread handheld electronic devices currently install numerous applications on their devices and use these applications to perform their daily activities more efficiently. However, to access these applications, users must typically unlock their devices, locate the desired application (e.g., by locating an icon associated with the desired application by looking around a home screen or by searching for the desired application in a search interface), and then also locate the desired function within the desired application. Thus, users often spend a significant amount of time locating the desired application and the desired function within these applications, instead of simply being able to instantly execute the desired application (e.g., by single-touch input) and / or execute the desired function.

[0003] Furthermore, having multiple installed applications inundates users with a continuous stream of information that cannot all be viewed instantly. As a result, users often want to return to view specific information that they have previously noticed or that they wish to use at a later time. However, users often are unable to locate specific information or are unable to remember how to locate specific information.

[0004] It is therefore desirable to provide an intuitive and easy-to-use system and method for proactively identifying and surfacing relevant content (e.g., specific information) on an electronic device in communication with a display and a touch-sensitive surface. Summary of the Invention

[0005] Thus, there is a need for electronic devices with faster and more efficient methods and interfaces for quickly accessing applications and desired functions within those applications. Additionally, there is a need for electronic devices that assist users in managing the continuous flow of information they receive daily by proactively identifying and providing relevant information (e.g., contacts, nearby places, applications, news articles, addresses, and other information available on the device) before this information is explicitly requested by the user. Such methods and interfaces, optionally, complement or replace traditional methods for accessing applications. Such methods and interfaces result in more efficient human-machine interfaces by requiring less input from the user to locate desired information. For battery-powered devices, such methods and interfaces conserve power and extend the time between battery charges (e.g., by requiring fewer touch inputs to perform various functions). Furthermore, such methods and interfaces increase the lifespan of touch-sensitive displays by requiring fewer touch inputs (e.g., instead of having to continually and aimlessly tap a touch-sensitive display to locate desired information, the methods and interfaces disclosed herein proactively provide this information without requiring user input).

[0006] The above deficiencies and other problems associated with user interfaces for electronic devices with touch-sensitive surfaces are addressed by the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or sets of instructions stored in the memory for performing a number of functions. In some embodiments, a user interacts with the GUI primarily through stylus and / or finger contacts and gestures on the touch-sensitive surface. In some embodiments, the functions optionally include image editing, drawing, presenting, word processing, website creation, disc authoring, spreadsheet creation, game playing, making phone calls, video conferencing, emailing, instant messaging, fitness support, digital photography, digital videography, web browsing, digital music playback, and / or digital video playback. Executable instructions to perform these functions are optionally contained in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0007] (A1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive display (touch screen 112, FIG. 1C). The method includes executing an application on the electronic device in response to instructions from a user of the electronic device. The method further includes collecting usage data while the application is running. The usage data includes at least one or more actions (or types of actions) performed by the user within the application. The method also includes (i) automatically, and without human intervention, obtaining at least one trigger condition based on the collected usage data, and (ii) associating the at least one trigger condition with a particular one of the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that a particular action associated with the trigger condition is available.

[0008] (A2) In some embodiments of the method described in A1, obtaining the at least one trigger condition includes transmitting usage data from the electronic device to one or more servers remotely located, and receiving the at least one trigger condition from the one or more servers.

[0009] (A3) In some embodiments of the methods described in A1 or A2, providing the indication includes displaying a user interface object on a lock screen of the touch-sensitive display corresponding to a particular action associated with the trigger condition.

[0010] (A4) In some embodiments of the method according to A3, the user interface object includes a description of a particular action associated with the trigger condition.

[0011] (A5) In some embodiments of the method according to A4, the user interface objects further include an icon associated with the application.

[0012] (A6) In some embodiments of the method of any one of A3 to A5, the method further includes detecting a first gesture at a user interface object, and in response to detecting the first gesture, (i) displaying the application on the touch-sensitive display, and (ii) the method includes performing a particular action associated with the trigger condition while displaying the application.

[0013] (A7) In some embodiments of the method according to A6, the first gesture is a swipe gesture on a user interface object.

[0014] (A8) In some embodiments of the method of any one of A3 to A5, the method further includes detecting a second gesture at the user interface object, and in response to detecting the second gesture and while continuing to display the lock screen on the touch-sensitive display, performing a particular action associated with the trigger condition.

[0015] (A9) In some embodiments of the method according to A8, the second gesture is a single tap in a predefined area of ​​the user interface object.

[0016] (A10) In some embodiments of the methods according to any one of A3 to A9, the user interface object is displayed in a default center portion of the lock screen.

[0017] (A11) In some embodiments of the method according to A1, providing an indication to the user that a particular action associated with the trigger condition is available includes performing the particular action.

[0018] (A12) In some embodiments of the method according to A3, the user interface object is an icon associated with the application, and the user interface object is displayed substantially in a corner of a lock screen of the touch-sensitive display.

[0019] (A13) In some embodiments of the method of any one of A1 to A12, the method further includes receiving an instruction from a user to unlock the electronic device. The method includes, in response to receiving the instruction, displaying, on a touch-sensitive display, a home screen of the electronic device. The method also includes providing, on the home screen, an indication to the user that a particular action associated with the trigger condition is available.

[0020] (A14) In some embodiments of the method described in A13, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion displayed adjacent to the first portion for launching a second set of applications available on the electronic device, the second portion being displayed on all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion.

[0021] (A15) In some embodiments of the method according to A14, the second set of applications is different from and smaller than the first set of applications.

[0022] (A16) In some embodiments of the methods according to any one of A1 to A15, determining that the at least one trigger condition is satisfied includes determining that the electronic device is coupled to a second device different from the electronic device.

[0023] (A17) In some embodiments of the methods described in any one of A1 to A16, determining that at least one trigger condition is satisfied includes determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user.

[0024] (A18) In some embodiments of the method described in A17, determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user includes monitoring motion data from an accelerometer of the electronic device and determining, based on the monitored motion data, that the electronic device has not moved for more than a threshold amount of time.

[0025] (A19) In some embodiments of the method according to any one of A1 to A18, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application continues to run. At least one trigger condition is further based on the verbal command provided to the virtual assistant application.

[0026] (A20) In some embodiments of the method described in A19, the verbal command includes a request to create a reminder that corresponds to a current state of the application, the current state corresponding to the state of the application at the time the verbal command was provided.

[0027] (A21) In some embodiments of the method described in A20, the state of the application when the verbal command is provided is selected from the group consisting of a page displayed within the application when the verbal command is provided, content being played within the application when the verbal command is provided, a notification displayed within the application when the verbal command is provided, and an active portion of a page displayed within the application when the verbal command is provided.

[0028] (A22) In some embodiments of the method described in A20, the spoken command includes the word "this" in reference to the current state of the application.

[0029] (A23) In another aspect, a method is performed on one or more electronic devices (e.g., portable multifunction device 100 (FIG. 5) and one or more servers 502 (FIG. 5)). The method includes executing an application on a first electronic device of the one or more electronic devices in response to an instruction from a user of the first electronic device. The method includes automatically and without human intervention collecting usage data while executing the application, the usage data including at least one or more actions (or types of actions) performed by the user within the application. The method further includes automatically and without human intervention establishing at least one trigger condition based on the collected usage data. The method further includes associating the at least one trigger condition with a particular action of the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that the particular action associated with the trigger condition is available.

[0030] (A24) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method according to any one of A1 to A22.

[0031] (A25) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for performing a method according to any one of A1 to A22.

[0032] (A26) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform a method according to any one of A1 to A22.

[0033] (A27) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of A1 to A22.

[0034] (A28) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4201, FIG. 42), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4203, FIG. 42), and a processing unit (e.g., processing unit 4205, FIG. 42). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 42 illustrates display unit 4201 and touch-sensitive surface unit 4203 as being integrated into electronic device 4200, although in some embodiments, one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit is coupled to the touch-sensitive surface unit and the display unit. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into one touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes an execution unit (e.g., execution unit 4207, FIG. 42), a collection unit (e.g., collection unit 4209, FIG. 42), an acquisition unit (e.g., acquisition unit 4211, FIG. 42), an association unit (e.g., association unit 4213, FIG. 42), a providing unit (e.g., providing unit 4215, FIG. 42), a sending unit (e.g., sending unit 4217, FIG. 42), a receiving unit (e.g., receiving unit 4219, FIG. 42), a display unit (e.g., display unit 4221, FIG. 42), a detection unit (e.g., detection unit 4223, FIG. 42), an implementation unit (e.g., implementation unit 4225, FIG. 42), a judgment unit (e.g., judgment unit 4227, FIG. 42), and a monitoring unit (e.g., monitoring unit 4229, FIG. 42).The processing unit (or one or more components thereof, such as units 4207 to 4229) is configured to execute an application on the electronic device in response to instructions from a user of the electronic device (e.g., by the execution unit 4207), and while executing the application, collect (e.g., by the collection unit 4209) usage data, the usage data including at least one or more actions performed by the user within the application, and automatically and without human intervention obtain (e.g., by the obtaining unit 4211) at least one trigger condition based on the collected usage data, and associate (e.g., by the associating unit 4213) the at least one trigger condition with a particular action among the one or more actions performed by the user within the application, and upon determining that the at least one trigger condition is satisfied, provide (e.g., by the providing unit 4215) an indication to the user that the particular action associated with the trigger condition is available.

[0035] (A29) In some embodiments of the electronic device described in A28, obtaining the at least one trigger condition includes transmitting usage data from the electronic device to one or more servers located remotely (e.g., by the transmitting unit 4217) and receiving the at least one trigger condition from the one or more servers (e.g., by the receiving unit 4219).

[0036] (A30) In some embodiments of the electronic device described in A28 or A29, providing the indication includes displaying (e.g., by display unit 4217 and / or display unit 4201) a user interface object corresponding to a particular action associated with the trigger condition on a lock screen of a touch-sensitive display unit.

[0037] (A31) In some embodiments of the electronic device described in A30, the user interface object includes a description of a particular action associated with the trigger condition.

[0038] (A32) In some embodiments of the electronic device described in A31, the user interface objects further include an icon associated with the application.

[0039] (A33) In some embodiments of the electronic device described in any one of A30 to A32, the processing unit is further configured to detect a first gesture at the user interface object (e.g., by the detection unit 4223 and / or the touch-sensitive surface unit 4203). In response to detecting the first gesture, (i) display the application on the touch-sensitive display unit (e.g., by the display unit 4217 and / or the display unit 4201), and (ii) perform a particular action associated with the trigger condition (e.g., by the implementation unit 4225) while displaying the application.

[0040] (A34) In some embodiments of the electronic device described in A33, the first gesture is a swipe gesture on a user interface object.

[0041] (A35) In some embodiments of the electronic device described in any one of A30 to A33, the processing unit is further configured to detect a second gesture at the user interface object (e.g., by the detection unit 4223 and / or the touch-sensitive surface unit 4203). In response to detecting the second gesture, and while continuing to display the lock screen on the touch-sensitive display unit, the processing unit is configured to perform (e.g., by the implementation unit 4225) a particular action associated with the trigger condition.

[0042] (A36) In some embodiments of the electronic device described in A35, the second gesture is a single tap in a predefined area of ​​the user interface object.

[0043] (A37) In some embodiments of the electronic device according to any one of A30 to A36, the user interface object is displayed in a default center part of the lock screen.

[0044] (A38) In some embodiments of the electronic device described in A28, providing an indication to a user that a particular action associated with the trigger condition is available includes performing the particular action (e.g., by the implementation unit 4225).

[0045] (A39) In some embodiments of the electronic device described in A30, the user interface object is an icon associated with the application, and the user interface object is displayed substantially in a corner of a lock screen of the touch-sensitive display unit.

[0046] (A40) In some embodiments of the electronic device according to any one of A28 to A39, the processing unit is further configured to receive (e.g., by the receiving unit 4219) an instruction from a user to unlock the electronic device. In response to receiving the instruction, the processing unit is configured to display (e.g., by the display unit 4217 and / or the display unit 4201) a home screen of the electronic device on the touch-sensitive display unit. The processing unit is also configured to provide (e.g., by the providing unit 4215) an indication on the home screen to the user that a particular action associated with the trigger condition is available.

[0047] (A41) In some embodiments of the electronic device described in A40, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion for launching a second set of applications available on the electronic device, displayed adjacent to the first portion, the second portion being displayed over all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion (e.g., by the display unit 4217 and / or the display unit 4201).

[0048] (A42) In some embodiments of the electronic device according to A41, the second set of applications is different from and smaller than the first set of applications.

[0049] (A43) In some embodiments of the electronic device described in any one of A28 to A42, determining that at least one trigger condition is satisfied includes determining (e.g., by the determination unit 4227) that the electronic device is coupled to a second device different from the electronic device.

[0050] (A44) In some embodiments of the electronic device described in any one of A28 to A43, determining that at least one trigger condition is satisfied includes determining (e.g., by the determination unit 4227) that the electronic device has arrived at an address corresponding to a home or work address associated with the user.

[0051] (A45) In some embodiments of the electronic device described in A44, determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user includes monitoring motion data from an accelerometer of the electronic device (e.g., by monitoring unit 4229) and determining (e.g., by determination unit 4227) based on the monitored motion data that the electronic device has not moved for more than a threshold amount of time.

[0052] (A46) In some embodiments of the electronic device according to any one of A28 to A45, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application continues to run. At least one trigger condition is further based on the verbal command provided to the virtual assistant application.

[0053] (A47) In some embodiments of the electronic device described in A46, the verbal command includes a request to create a reminder that corresponds to a current state of the application, the current state corresponding to the state of the application at the time the verbal command was provided.

[0054] (A48) In some embodiments of the electronic device described in A47, the state of the application when the verbal command is provided is selected from the group consisting of a page displayed in the application when the verbal command is provided, content being played in the application when the verbal command is provided, a notification displayed in the application when the verbal command is provided, and an active portion of a page displayed in the application when the verbal command is provided.

[0055] (A49) In some embodiments of the electronic device described in A46, the spoken command includes the word "this" in reference to the current state of the application.

[0056] (B1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive display (touch screen 112, FIG. 1C). The method includes obtaining at least one trigger condition based on usage data associated with a user of the electronic device, the usage data including one or more actions (or types of actions) performed by the user within the application while the application is executing on the electronic device. The method also includes associating the at least one trigger condition with a particular one of the one or more actions performed by the user within the application. The method includes, upon determining that the at least one trigger condition is satisfied, providing an indication to the user that the particular action associated with the trigger condition is available.

[0057] (B2) In some embodiments of the method according to B1, the method further comprises the method according to any one of A2 to A22.

[0058] (B3) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory storing one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in B1 or B2.

[0059] (B4) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for performing the method of B1 or B2.

[0060] (B5) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform a method as described in B1 or B2.

[0061] (B6) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive display. In some embodiments, the graphical user interface comprises a user interface displayed according to the method of B1 or B2.

[0062] (B7) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4201, FIG. 42), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4203, FIG. 42), and a processing unit (e.g., processing unit 4205, FIG. 42). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. IE (i.e., computing devices A through D). For ease of explanation, FIG. 42 illustrates display unit 4201 and touch-sensitive surface unit 4203 as being integrated into electronic device 4200, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an execution unit (e.g., execution unit 4207, FIG. 42), a collection unit (e.g., collection unit 4209, FIG. 42), an acquisition unit (e.g., acquisition unit 4211, FIG. 42), an association unit (e.g., association unit 4213, FIG. 42), a providing unit (e.g., providing unit 4215, FIG. 42), a sending unit (e.g., sending unit 4217, FIG. 42), a receiving unit (e.g., receiving unit 4219, FIG. 42), a display unit (e.g., display unit 4221, FIG. 42), a detection unit (e.g., detection unit 4223, FIG. 42), an implementation unit (e.g., implementation unit 4225, FIG. 42), a judgment unit (e.g., judgment unit 4227, FIG. 42), and a monitoring unit (e.g., monitoring unit 4229, FIG. 42).The processing unit (or one or more components thereof, such as units 4207 to 4229) is configured to obtain (e.g., by the obtaining unit 4211) at least one trigger condition based on usage data associated with a user of the electronic device, the usage data including one or more actions performed by the user within the application while the application is running on the electronic device, associate (e.g., by the associating unit 4213) the at least one trigger condition with a particular action among the one or more actions performed by the user within the application, and, upon determining that the at least one trigger condition is satisfied, provide (e.g., by the providing unit 4215) an indication to the user that the particular action associated with the trigger condition is available.

[0063] (B8) In some embodiments of the electronic device described in B7, obtaining the at least one trigger condition includes transmitting usage data to one or more servers located remotely from the electronic device (e.g., by the transmitting unit 4217) and receiving the at least one trigger condition from the one or more servers (e.g., by the receiving unit 4219).

[0064] (B9) In some embodiments of the electronic device described in B7 or B8, providing the indication includes displaying (e.g., by display unit 4217 and / or display unit 4201) a user interface object on a lock screen of the touch-sensitive display that corresponds to a particular action associated with the trigger condition.

[0065] (B10) In some embodiments of the electronic device described in B9, the user interface object includes a description of a particular action associated with the trigger condition.

[0066] (B11) In some embodiments of the electronic device described in B10, the user interface objects further include an icon associated with the application.

[0067] (B12) In some embodiments of the electronic device described in any one of B9 to B11, the processing unit is further configured to detect a first gesture at the user interface object (e.g., by the detection unit 4223 and / or the touch-sensitive surface unit 4203). In response to detecting the first gesture, (i) display the application on the touch-sensitive display (e.g., by the display unit 4217 and / or the display unit 4201), and (ii) perform a particular action associated with the trigger condition (e.g., by the implementation unit 4225) while displaying the application.

[0068] (B13) In some embodiments of the electronic device described in B12, the first gesture is a swipe gesture on a user interface object.

[0069] (B14) In some embodiments of the electronic device described in any one of B9 to B12, the processing unit is further configured to detect a second gesture at the user interface object (e.g., by the detection unit 4223 and / or the touch-sensitive surface unit 4203). In response to detecting the second gesture, and while continuing to display the lock screen on the touch-sensitive display, the processing unit is configured to perform (e.g., by the implementation unit 4225) a particular action associated with the trigger condition.

[0070] (B15) In some embodiments of the electronic device described in B14, the second gesture is a single tap in a predefined area of ​​the user interface object.

[0071] (B16) In some embodiments of the electronic device according to any one of B9 to B15, the user interface object is displayed in a default center portion of the lock screen.

[0072] (B17) In some embodiments of the electronic device described in B7, providing an indication to a user that a particular action associated with the trigger condition is available includes performing the particular action (e.g., by the implementation unit 4225).

[0073] (B18) In some embodiments of the electronic device described in B9, the user interface object is an icon associated with the application, and the user interface object is displayed substantially in a corner of a lock screen of the touch-sensitive display.

[0074] (B19) In some embodiments of the electronic device according to any one of B7 to B18, the processing unit is further configured to receive (e.g., by the receiving unit 4219) an instruction from a user to unlock the electronic device. In response to receiving the instruction, the processing unit is configured to display (e.g., by the display unit 4217 and / or the display unit 4201) a home screen of the electronic device on the touch-sensitive display. The processing unit is also configured to provide (e.g., by the providing unit 4215) an indication on the home screen to the user that a particular action associated with the trigger condition is available.

[0075] (B20) In some embodiments of the electronic device described in B19, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion for launching a second set of applications available on the electronic device, displayed adjacent to the first portion, the second portion being displayed on all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion (e.g., by the display unit 4217 and / or the display unit 4201).

[0076] (B21) In some embodiments of the electronic device according to B20, the second set of applications is different from and smaller than the first set of applications.

[0077] (B22) In some embodiments of the electronic device described in any one of B7 to B21, determining that at least one trigger condition is satisfied includes determining (e.g., by the determination unit 4227) that the electronic device is coupled to a second device different from the electronic device.

[0078] (B23) In some embodiments of the electronic device described in any one of B7 to B22, determining that at least one trigger condition is satisfied includes determining (e.g., by the determination unit 4227) that the electronic device has arrived at an address corresponding to a home or work address associated with the user.

[0079] (B24) In some embodiments of the electronic device described in B23, determining that the electronic device has arrived at an address corresponding to a home or work address associated with the user includes monitoring motion data from an accelerometer of the electronic device (e.g., by monitoring unit 4229) and determining (e.g., by determination unit 4227) based on the monitored motion data that the electronic device has not moved for more than a threshold amount of time.

[0080] (B25) In some embodiments of the electronic device according to any one of B7 to B24, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application continues to run. At least one trigger condition is further based on the verbal command provided to the virtual assistant application.

[0081] (B26) In some embodiments of the electronic device described in B25, the verbal command includes a request to create a reminder that corresponds to a current state of the application, the current state corresponding to the state of the application at the time the verbal command was provided.

[0082] (B27) In some embodiments of the electronic device described in B26, the state of the application when the verbal command is provided is selected from the group consisting of a page displayed in the application when the verbal command is provided, content being played in the application when the verbal command is provided, a notification displayed in the application when the verbal command is provided, and an active portion of a page displayed in the application when the verbal command is provided.

[0083] (B28) In some embodiments of the electronic device described in B26, the spoken command includes the word "this" in reference to a current state of the application.

[0084] (C1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive display (touch screen 112, FIG. 1C). The method includes detecting a search activation gesture of a user of the electronic device on the touch-sensitive display. The method includes, in response to detecting the search activation gesture, causing a search interface to be displayed on the touch-sensitive display, the search interface including (i) a search entry portion and (ii) a prediction portion that is displayed prior to receiving any user input in the search entry portion. The prediction portion is populated with one or more of: (a) at least one affordance for contacting a person from a plurality of previously contacted people, where the person is automatically selected from among the plurality of previously contacted people based at least in part on a current time; and (b) at least one affordance for performing a predicted action within an application from a plurality of applications available on the electronic device, where the predicted action is automatically selected based at least in part on an application usage history associated with a user of the electronic device.

[0085] (C2) In some embodiments of the method of C1, the person is further selected based at least in part on location data corresponding to the electronic device.

[0086] (C3) In some embodiments of the methods described in C1 or C2, application usage history and contact information for the person are retrieved from a memory of the electronic device.

[0087] (C4) In some embodiments of the methods described in C1 or C2, the application usage history and contact information for the person are retrieved from a server located remotely from the electronic device.

[0088] (C5) In some embodiments of the methods of any one of C1 to C4, the prediction portion is further populated with at least one affordance for executing the predicted application, and the predicted application is automatically selected based, at least in part, on application usage history.

[0089] (C6) In some embodiments of the methods of any one of C1 to C5, the prediction portion is further populated with at least one affordance of a predicted place (or nearby place) category, the predicted place category being automatically selected based at least in part on one or more of a current time and location data corresponding to the electronic device.

[0090] (C7) In some embodiments of the method of any one of C1 to C6, the method further includes detecting a user input to scroll the predicted portion. The method includes, in response to detecting the user input to scroll the predicted portion, scrolling the predicted portion according to the user input. The method includes, in response to scrolling, revealing at least one affordance of a predicted news article in the predicted portion (e.g., the predicted news article is expected to be of interest to the user).

[0091] (C8) In some embodiments of the method as set forth in C7, the predicted news articles are automatically selected based, at least in part, on location data corresponding to the electronic device.

[0092] (C9) In some embodiments of the method of any one of C1 to C8, the method further includes detecting a selection of at least one affordance for performing a predicted action within the application, the method including, in response to detecting the selection, displaying the application on the touch-sensitive display and performing the predicted action within the application.

[0093] (C10) In some embodiments of the method of C3 or C4, the method further includes detecting a selection of at least one affordance for contacting the person, the method including, in response to detecting the selection, contacting the person using contact information for the person.

[0094] (C11) In some embodiments of the method of C5, the method further includes detecting a selection of at least one affordance for executing the predicted application, the method including, in response to detecting the selection, displaying the predicted application on the touch-sensitive display.

[0095] (C12) In some embodiments of the method according to C6, the method further includes detecting a selection of at least one affordance of the predicted place category, the method further includes, in response to detecting the selection, (i) receiving data corresponding to the at least one nearby place, and (ii) displaying, on the touch-sensitive display, the received data corresponding to the at least one nearby place.

[0096] (C13) In some embodiments of the method of C7, the method further includes detecting a selection of at least one affordance of the predicted news article. The method includes, in response to detecting the selection, displaying, on the touch-sensitive display, the predicted news article.

[0097] (C14) In some embodiments of the methods described in any one of C1 to C13, the search activation gesture is available from at least two different user interfaces, and a first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of the series of home screen pages on the touch-sensitive display.

[0098] (C15) In some embodiments of the method described in C14, when the corresponding home screen page is a first home screen page in a series of home screen pages, the search activation gesture includes one of the following: (i) a gesture moving substantially in a downward direction relative to a user of the electronic device, or (ii) a continuous gesture moving substantially in a left-to-right direction and substantially perpendicular to the downward direction relative to the user.

[0099] (C16) In some embodiments of the method described in C15, when the corresponding home screen page is a second home screen page of a sequence of home screen pages, the search activation gesture includes a continuous gesture moving substantially downward with respect to a user of the electronic device.

[0100] (C17) In some embodiments of the method according to C14, a second user interface of the at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display.

[0101] (C18) In some embodiments of the method described in C17, the search activation gesture includes contact on a touch-sensitive display at a default search activation portion of the application switching interface.

[0102] (C19) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in any one of C1 to C18.

[0103] (C20) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for performing the method of any one of C1 to C18.

[0104] (C21) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform a method according to any one of C1 to C18.

[0105] (C22) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of C1 to C18.

[0106] (C23) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4301, FIG. 43), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4303, FIG. 43), and a processing unit (e.g., processing unit 4305, FIG. 43). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. IE (i.e., computing devices A through D). For ease of explanation, FIG. 43 illustrates display unit 4301 and touch-sensitive surface unit 4303 as being integrated into electronic device 4300, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a display unit (e.g., display unit 4309, FIG. 43), a detection unit (e.g., detection unit 4307, FIG. 43), a retrieval unit (e.g., retrieval unit 4311, FIG. 43), a populating unit (e.g., populating unit 4313, FIG. 43), a scrolling unit (e.g., scrolling unit 4315, FIG. 43), a revealing unit (e.g., revealing unit 4317, FIG. 43), a selection unit (e.g., selection unit 4319, FIG. 43), a contact unit (e.g., contact unit 4321, FIG. 43), a receiving unit (e.g., receiving unit 4323, FIG. 43), and an execution unit (e.g., execution unit 4325, FIG. 43). The processing unit (or one or more components thereof, such as units 4307 to 4225) is configured to detect a search activation gesture of a user of the electronic device on the touch-sensitive display (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) and, in response to detecting the search activation gesture, display a search interface on the touch-sensitive display (e.g., by the display unit 4309 and / or the display unit 4301), the search interface including (i) a search entry portion and (ii) a prediction portion that is displayed before receiving any user input in the search entry portion.The prediction portion is populated with one or more of: (a) at least one affordance for contacting a person from a plurality of previously contacted people, where the person is automatically selected (e.g., by selection unit 4319) from among the plurality of previously contacted people based at least in part on the current time; and (b) at least one affordance for performing a predicted action within an application from a plurality of applications available on the electronic device, where the predicted action is automatically selected (e.g., by selection unit 4319) based at least in part on an application usage history associated with a user of the electronic device.

[0107] (C24) In some embodiments of the electronic device described in C23, the person is further selected (eg, by the selection unit 4319) based at least in part on location data corresponding to the electronic device.

[0108] (C25) In some embodiments of the electronic device described in C23 or C24, the application usage history and contact information for the person are retrieved from the memory of the electronic device (eg, by retrieval unit 4311).

[0109] (C26) In some embodiments of the electronic device described in C23 or C24, application usage history and contact information for the person are retrieved (eg, by retrieval unit 4311) from a server located remotely from the electronic device.

[0110] (C27) In some embodiments of an electronic device described in any one of C23 to C26, the predicted portion is further populated (e.g., by a populating unit 4313) with at least one affordance for executing the predicted application, and the predicted application is automatically selected (e.g., by a selection unit 4319) based at least in part on application usage history.

[0111] (C28) In some embodiments of the electronic device described in any one of C23 to C27, the predicted portion is further populated (e.g., by a populating unit 4313) with at least one affordance of a predicted place category, and the predicted place category is automatically selected (e.g., by a selection unit 4319) based at least in part on one or more of the current time and location data corresponding to the electronic device.

[0112] (C29) In some embodiments of the electronic device of any one of C23 to C28, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a user input for scrolling the predicted portion. In response to detecting the user input for scrolling the predicted portion, the processing unit is configured to scroll (e.g., by the scrolling unit 4319) the predicted portion according to the user input. In response to scrolling, the processing unit is configured to reveal (e.g., by the revealing unit 4317) at least one affordance of the predicted news article in the predicted portion (e.g., the predicted news article is one expected to be of interest to the user).

[0113] (C30) In some embodiments of the electronic device described in C7, the predicted news articles are automatically selected (eg, by the selection unit 4319) based at least in part on location data corresponding to the electronic device.

[0114] (C31) In some embodiments of the electronic device of any one of C23 to C30, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance for performing a predicted action within the application. In response to detecting the selection, the processing unit is configured to display (e.g., by the display unit 4309) the application on a touch-sensitive display (e.g., the display unit 4301) and perform (e.g., by the execution unit 4325) the predicted action within the application.

[0115] (C32) In some embodiments of the electronic device described in C25 or C26, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of the at least one affordance for contacting the person. In response to detecting the selection, the processing unit is configured to contact the person (e.g., by the contact unit 4321) using contact information for the person.

[0116] (C33) In some embodiments of the electronic device described in C27, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance for executing the predicted application. In response to detecting the selection, the processing unit is configured to display (e.g., by the display unit 4307) the predicted application on the touch-sensitive display (e.g., by the display unit 4301).

[0117] (C34) In some embodiments of the electronic device described in C28, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance of the predicted location category. In response to detecting the selection, the processing unit is configured to (i) receive (e.g., by the receiving unit 4323) data corresponding to the at least one nearby location, and (ii) display (e.g., by the display unit 4307) the received data corresponding to the at least one nearby location on the touch-sensitive display (e.g., the display unit 4301).

[0118] (C35) In some embodiments of the electronic device described in C29, the processing unit is further configured to detect (e.g., by the detection unit 4307 and / or the touch-sensitive surface unit 4303) a selection of at least one affordance of the predicted news article. In response to detecting the selection, the processing unit is configured to display (e.g., by the display unit 4307) the predicted news article on a touch-sensitive display (e.g., the display unit 4301).

[0119] (C36) In some embodiments of the electronic device described in any one of C23 to C35, the search activation gesture is available from at least two different user interfaces, and a first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of the series of home screen pages on the touch-sensitive display.

[0120] (C37) In some embodiments of the electronic device described in C36, when the corresponding home screen page is a first home screen page in a series of home screen pages, the search activation gesture includes one of the following: (i) a gesture moving substantially in a downward direction relative to a user of the electronic device, or (ii) a continuous gesture moving substantially in a left-to-right direction and substantially perpendicular to the downward direction relative to the user.

[0121] (C38) In some embodiments of the electronic device described in C37, if the corresponding home screen page is a second home screen page of a sequence of home screen pages, the search activation gesture includes a continuous gesture moving substantially downward with respect to a user of the electronic device.

[0122] (C39) In some embodiments of the electronic device described in C36, a second user interface of the at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display.

[0123] (C40) In some embodiments of the electronic device described in C39, the search activation gesture comprises contact at a default search activation portion of the application switching interface on a touch-sensitive display.

[0124] Thus, electronic devices having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface are provided with faster and more efficient methods and interfaces for proactively accessing applications and proactively executing functions within applications, thereby increasing the effectiveness, efficiency, and satisfaction of users on such devices. Such methods and interfaces may complement or replace conventional methods and features associated with such methods for accessing applications.

[0125] (D1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ) and a display (e.g., display 194, FIG. 1D ). The method includes displaying, on the display, content associated with an application running on the electronic device. The method further includes detecting, via the touch-sensitive surface, a swipe gesture that, when detected, causes the electronic device to transition to a search mode distinct from the application. The method also includes, in response to detecting the swipe gesture, transitioning to a search mode, the search mode including a search interface displayed on the display. In conjunction with transitioning to the search mode, the method includes determining at least one suggested search query based, at least in part, on information associated with the content. The method includes populating the displayed search interface with the at least one suggested search query prior to receiving any user input at the search interface. In this way, instead of having to remember and re-enter information into a search interface, the device provides the user with relevant suggestions based on the application content the user is viewing, and the user only needs to select one of the suggestions, without having to type anything.

[0126] (D2) In some embodiments of the method according to D1, detecting the swipe gesture includes detecting a swipe gesture over at least a portion of currently displayed content.

[0127] (D3) In some embodiments of the method of D1 or D2, the method further includes detecting an input corresponding to a request to view a home screen of the electronic device before detecting the swipe gesture, and in response to detecting the input, ceasing to display content associated with the application and displaying a corresponding page of the home screen of the electronic device. In some embodiments, the corresponding page is an initial page of a series of home screen pages, and the swipe gesture is detected while the initial page of the home screen is displayed on the display.

[0128] (D4) In some embodiments of the method according to any one of D1 to D3, the search interface is semi-transparently overlaid on the application.

[0129] (D5) In some embodiments of the method of any one of D1 to D4, the method further includes, pursuant to a determination that the content includes textual content, determining at least one suggested search query based, at least in part, on the textual content.

[0130] (D6) In some embodiments of the method described in D5, determining the at least one suggested search query based at least in part on the textual content includes analyzing the textual content to detect one or more defined keywords used to determine the at least one suggested search query.

[0131] (D7) In some embodiments of the methods described in any one of D1 to D6, determining at least one suggested search query includes determining a plurality of suggested search queries, and populating the search interface includes populating the search interface with the plurality of suggested search queries.

[0132] (D8) In some embodiments of the method of any one of D1 to D7, the method further includes detecting a new swipe gesture on currently displayed new content via the touch-sensitive surface, and in response to detecting the new swipe gesture, transitioning to a search mode, where transitioning to the search mode includes displaying a search interface on the display, and in conjunction with transitioning to the search mode and in accordance with a determination that the new content does not include textual content, populating the search interface with suggested search queries based on a set of past search queries selected by a user of the electronic device.

[0133] (D9) In some embodiments of the method described in D8, the search interface is displayed with the points of interest based on location information provided by a second application different from the application.

[0134] (D10) In some embodiments of the method according to any one of D8 or D9, the search interface further includes one or more suggested applications.

[0135] (D11) In some embodiments of the method of any one of D8 to D10, the set of past search queries is selected based at least in part on a frequency of recent search queries.

[0136] (D12) In some embodiments of the method of any one of D1 to D11, the method further includes, in conjunction with entering a search mode, obtaining information associated with the content by using one or more accessibility features available on the electronic device.

[0137] (D13) In some embodiments of the method described in D12, using the one or more accessibility features includes generating information associated with the content using the one or more accessibility features by: (i) applying a natural language processing algorithm to text content currently displayed within the application; and (ii) determining one or more keywords describing the content using data obtained from the natural language processing algorithm, where at least one suggested search query is determined based on the one or more keywords.

[0138] (D14) In some embodiments of the method described in D13, determining one or more keywords that describe the content also includes (i) retrieving metadata that corresponds to non-text content currently displayed within the application, and (ii) determining the one or more keywords using the retrieved metadata in addition to data obtained from the natural language processing algorithm.

[0139] (D15) In some embodiments of the method of any one of D1 to D14, the search interface further includes one or more trend queries.

[0140] (D16) In some embodiments of the method according to D15, the search interface further includes one or more applications expected to be of interest to a user of the electronic device.

[0141] (D17) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in any one of D1 to D16.

[0142] (D18) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of D1 to D16.

[0143] (D19) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method as described in any one of D1 to D16.

[0144] (D20) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of D1 to D16. In a further aspect, an information processing apparatus is provided for use on an electronic device including a touch-sensitive surface and a display. The information processing apparatus includes means for performing a method according to any one of D1 to D16.

[0145] (D21) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4401, FIG. 44), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4403, FIG. 44), and a processing unit (e.g., processing unit 4405, FIG. 44). The processing unit is coupled to the touch-sensitive surface unit and the display unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 44 illustrates display unit 4401 and touch-sensitive surface unit 4403 as being integrated into electronic device 4400, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into one touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a detection unit (e.g., detection unit 4407, FIG. 44), a display unit (e.g., display unit 4409, FIG. 44), an extraction unit (e.g., extraction unit 4411, FIG. 44), a search mode transition unit (e.g., search mode transition unit 4412, FIG. 44), a populating unit (e.g., populating unit 4413, FIG. 44), an acquisition unit (e.g., acquisition unit 4415, FIG. 44), a judgment unit (e.g., judgment unit 4417, FIG. 44), and a selection unit (e.g., selection unit 4419, FIG. 44).The processing unit (or one or more components thereof, such as units 1007 to 1029) is configured to display (e.g., by display unit 4407) on a display unit (e.g., display unit 4407) content associated with an application running on the electronic device; detect (e.g., by detection unit 4407) via the touch-sensitive surface unit (e.g., touch-sensitive surface unit 4403) a swipe gesture that, when detected, causes the electronic device to transition to a search mode distinct from the application; in response to detecting the swipe gesture, transition (e.g., by search mode transition unit 4412) to a search mode including a search interface displayed on the display unit (e.g., display unit 4407); and, in conjunction with transitioning to the search mode, determine (e.g., by determining unit 4417) at least one suggested search query based, at least in part, on information associated with the content; and populate (e.g., by populating unit 4413) the displayed search interface with the at least one suggested search query prior to receiving any user input at the search interface.

[0146] (D22) In some embodiments of the electronic device described in D21, detecting the swipe gesture includes detecting (e.g., by detection unit 4407) the swipe gesture on at least a portion of the currently displayed content.

[0147] (D23) In some embodiments of the electronic device described in D21 or D22, the processing unit is further configured to detect (e.g., by the detection unit 4407) an input corresponding to a request to view a home screen of the electronic device before detecting the swipe gesture, and in response to detecting the input (e.g., by the detection unit 4407), stop displaying content associated with the application and display (e.g., by the display unit 4409) a corresponding page of the home screen of the electronic device, the corresponding page being an initial page of a series of home screen pages, and the swipe gesture is detected (e.g., by the detection unit 4407) while the initial page of the home screen is displayed on the display unit.

[0148] (D24) In some embodiments of the electronic device according to any one of D21 to D23, the search interface is displayed semi-transparently overlaid on the application (eg, on the display unit 4409 and / or the display unit 4401).

[0149] (D25) In some embodiments of the electronic device described in any one of D21 to D24, the processing unit is further configured to determine (e.g., by the determining unit 4417) at least one suggested search query based, at least in part, on the textual content in accordance with a determination that the content includes textual content.

[0150] (D26) In some embodiments of the electronic device described in D25, determining the at least one suggested search query based at least in part on the textual content includes analyzing the textual content to detect (e.g., by detection unit 4407) one or more predefined keywords used to determine (e.g., by determination unit 4417) the at least one suggested search query.

[0151] (D27) In some embodiments of the electronic device described in any one of D21 to D26, determining at least one suggested search query includes determining a plurality of suggested search queries (e.g., by determining unit 4417), and populating the search interface includes populating the search interface with the plurality of suggested search queries (e.g., by populating unit 4413).

[0152] (D28) In some embodiments of the electronic device described in any one of D21 to D27, the processing unit is further configured to detect (e.g., by detection unit 4407) a new swipe gesture on the currently displayed new content via the touch-sensitive surface unit (e.g., by touch-sensitive surface unit 4403), and in response to detecting the new swipe gesture, transition to the search mode (e.g., by search mode transition unit 4412), where transitioning to the search mode includes displaying a search interface on the display unit (e.g., by display unit 4409), and in conjunction with transitioning to the search mode and in accordance with a determination that the new content does not include text content, populate (e.g., by populating unit 4413) the search interface with suggested search queries based on a set of past search queries selected by a user of the electronic device.

[0153] (D29) In some embodiments of the electronic device described in D28, the points of interest are displayed in the search interface based on location information provided by a second application different from the application (eg, display unit 4409).

[0154] (D30) In some embodiments of the electronic device described in D28 or D29, the search interface further comprises one or more suggested applications.

[0155] (D31) In some embodiments of the electronic device described in any one of D28 to D30, the set of past search queries is selected (e.g., by the selection unit 4419) based at least in part on a frequency of recent search queries.

[0156] (D32) In some embodiments of the electronic device described in any one of D21 to D31, the processing unit is further configured to, in conjunction with entering the search mode, obtain (e.g., by the obtaining unit 4415) information associated with the content by using one or more accessibility features available on the electronic device.

[0157] (D33) In some embodiments of the electronic device described in D32, using the one or more accessibility features includes (i) applying a natural language processing algorithm to text content currently displayed within the application, and (ii) using data obtained (e.g., by obtaining unit 4415) from the natural language processing algorithm to determine (e.g., by determining unit 4417) one or more keywords that describe the content, where at least one suggested search query is determined (e.g., by determining unit 4417) based on the one or more keywords, thereby generating information associated with the content using the one or more accessibility features.

[0158] (D34) In some embodiments of the electronic device described in D33, determining one or more keywords describing the content also includes (i) retrieving metadata corresponding to non-text content currently displayed within the application (e.g., by retrieval unit 4411), and (ii) determining the one or more keywords using the retrieved metadata in addition to data obtained from the natural language processing algorithm (e.g., by determination unit 4417).

[0159] (D35) In some embodiments of the electronic device of any one of D21 to D34, the search interface further comprises one or more trend queries.

[0160] (D36) In some embodiments of the electronic device described in D35, the search interface further includes one or more applications in which a user of the electronic device is expected to be of interest.

[0161] (E1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 (FIG. 1A) configured according to any one of computing devices A-D (FIG. 1E)) having a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D) and a display (e.g., display 194, FIG. 1D). The method includes detecting a swipe gesture on a user interface via the touch-sensitive surface, which causes the electronic device to transition to a search mode upon the detection of the swipe gesture. The method further includes transitioning to the search mode in response to detecting the swipe gesture, where transitioning to the search mode includes populating a search interface, distinct from the user interface, with the first content item prior to receiving any user input in the search interface. In some embodiments, populating the search interface with the first content item pursuant to a determination that the user interface includes content associated with an application different from a home screen that includes a selectable icon for invoking the application includes populating the search interface with at least one suggested search query based, at least in part, on the content associated with the application, and populating the search interface with the first content item pursuant to a determination that the user interface is associated with a home screen page includes populating the search interface with affordances including a selectable description of at least one point of interest within a threshold distance of a current location of the electronic device.

[0162] (E2) In some embodiments of the method described in E1, populating the search interface with affordances includes displaying a search entry portion of the search interface on the touch-sensitive surface, the method further including detecting input in the search entry portion, and in response to detecting input in the search entry portion, ceasing to display the affordances and displaying at least one suggested search query in the search interface.

[0163] (E3) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs, the one or more programs, when executed by the one or more processors, cause the electronic device to perform a method as described in E1 or E2.

[0164] (E4) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of E1 or E2.

[0165] (E5) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method as described in E1 or E2.

[0166] (E6) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive surface and a display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method of E1 or E2. In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display. The information processing apparatus comprises means for performing the method of E1 or E2.

[0167] (E7) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4501, FIG. 45), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4503, FIG. 45), and a processing unit (e.g., processing unit 4505, FIG. 45). The processing unit is coupled to the touch-sensitive surface unit and the display unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 45 illustrates display unit 4501 and touch-sensitive surface unit 4503 as being integrated into electronic device 4500, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into one touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a detection unit (e.g., detection unit 4507, FIG. 45), a display unit (e.g., display unit 4509, FIG. 45), a populating unit (e.g., populating unit 4511, FIG. 45), and a search mode transition unit (e.g., search mode transition unit 4513, FIG. 45). The processing unit (or one or more components thereof, such as units 4507 to 4513) is configured to detect (e.g., by detection unit 4507) via the touch-sensitive surface unit (e.g., touch-sensitive surface unit 4503) a swipe gesture on the user interface, which when the swipe gesture is detected, causes the electronic device to transition to a search mode, and in response to detecting the swipe gesture, transition to the search mode (e.g., by search mode transition unit 4513), where transitioning to the search mode includes populating (e.g., by populating unit 4511) a search interface distinct from the user interface with a first content item before receiving any user input in the search interface.In some embodiments, populating the search interface with the first content item in accordance with a determination that the user interface includes content associated with an application different from a home screen that includes a selectable icon for invoking the application includes populating the search interface with at least one suggested search query based, at least in part, on the content associated with the application (e.g., by the populating unit 4511), and populating the search interface with the first content item in accordance with a determination that the user interface is associated with a home screen page includes populating the search interface with affordances including a selectable description of at least one point of interest within a threshold distance of a current location of the electronic device (e.g., by the populating unit 4511).

[0168] (E8) In some embodiments of the electronic device described in E7, populating the search interface with the affordances includes displaying (e.g., by the display unit 4507 and / or the display unit 4501) a search entry portion of the search interface, and the processing unit is further configured to detect (e.g., by the detection unit 4507) input in the search entry portion, and in response to detecting the input in the search entry portion, stop displaying the affordances (e.g., by the display unit 4507 and / or the display unit 4501), and display at least one suggested search query in the search interface (e.g., by the display unit 4507 and / or the display unit 4501).

[0169] (F1) In some embodiments, a method is performed in an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a position sensor, a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ), and a display (e.g., display 194, FIG. 1D ). The method includes determining, automatically and without instructions from a user, that a user of the electronic device is in a stopped vehicle at a geographic location, and upon determining that the user has left the vehicle at the geographic location, determining whether location information retrieved from the position sensor to identify the geographic location satisfies an accuracy criterion. The method further includes, upon determining that the location information does not satisfy the accuracy criterion, providing a prompt to the user to input information regarding the geographic location. The method also includes, in response to providing the prompt, receiving information about the geographic location from the user and storing the information as vehicle location information.

[0170] (F2) In some embodiments of the method described in F1, the method further includes, upon determining that the location information meets the accuracy criteria, automatically and without instruction from a user, storing the location information as vehicle location information.

[0171] (F3) In some embodiments of the method of claim F2, the method further includes, in accordance with a determination that the user is heading to the geographic location, displaying a user interface object that includes vehicle location information.

[0172] (F4) In some embodiments of the method according to claim F3, the user interface object is a map object that includes an identifier for the user's current location and another identifier for the geographic location.

[0173] (F5) In some embodiments of the method of claim F4, the user interface object is displayed on a lock screen of the electronic device.

[0174] (F6) In some embodiments of the method of claim F4, the user interface object is displayed in response to a swipe gesture that causes the electronic device to enter a search mode.

[0175] (F7) In some embodiments of the method of claim F6, determining whether the user is heading towards the geographic location is performed in response to receiving a swipe gesture.

[0176] (F8) In some embodiments of the methods of any one of F1 to F7, the prompt is a voice prompt provided by a virtual assistant available via the electronic device, and receiving information from the user includes receiving a verbal description from the user that identifies the geographic location, and displaying the user interface object includes displaying a selectable affordance that, when selected, causes the device to play the verbal description.

[0177] (F9) In some embodiments of the methods of any one of F1 to F7, the prompt is displayed on a display of the electronic device, receiving information from the user includes receiving a text description from the user that identifies the geographic location, and displaying the user interface object includes displaying the text description from the user.

[0178] (F10) In some embodiments of the methods according to any one of F1 to F7, determining whether the user is moving towards the geographic location includes determining that the electronic device is moving towards the geographic location using new location information received from the location sensor.

[0179] (F11) In some embodiments of the method described in F10, determining whether the user is heading towards the geographic location includes (i) determining that the electronic device remains at the different geographic location for more than a threshold time, and (ii) determining that new location information indicates that the electronic device is leaving the different geographic location and heading towards the geographic location.

[0180] (F12) In some embodiments of the methods described in any one of F1 to F11, determining that the user is in a vehicle stopped at the geographic location includes (i) determining that the user is in the vehicle by determining that the electronic device is moving above a threshold speed, and (ii) determining that the vehicle has stopped at the geographic location by one or more of: (a) determining that the electronic device remains at the geographic location for more than a threshold time, (b) determining that a communications link between the electronic device and the vehicle has been disconnected, and (c) determining that the geographic location corresponds to a location in a parking lot.

[0181] (F13) In some embodiments of the method of claim F12, determining that the vehicle has stopped at the geographic location includes determining that the electronic device has remained at the geographic location for more than a threshold time.

[0182] (F14) In some embodiments of the methods described in F12 or F13, determining that the vehicle has stopped at the geographic location includes determining that a communication link between the electronic device and the vehicle has been disconnected.

[0183] (F15) In some embodiments of the methods according to any one of F12 to F14, determining that the vehicle has stopped at the geographic location includes determining that the geographic location corresponds to a location within a parking lot.

[0184] (F16) In some embodiments of the methods according to any one of F1 to F15, the accuracy criterion comprises a criterion that is met if the accuracy of the GPS readings associated with the location information exceeds a threshold level of accuracy.

[0185] (F17) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, a position sensor, one or more processors, and a memory that stores one or more programs, the one or more programs when executed by the one or more processors causing the electronic device to perform a method as described in any one of F1 to F16.

[0186] (F18) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, a position sensor, and means for performing the method of any one of F1 to F16.

[0187] (F19) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface, a display, and a position sensor, cause the electronic device to perform a method according to any one of F1 to F16.

[0188] (F20) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface, a display, and a position sensor is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method of any one of F1 to F16. In a further aspect, an information processing apparatus is provided for use on an electronic device including a touch-sensitive surface, a display, and a position sensor. The information processing apparatus includes means for performing the method of any one of F1 to F16.

[0189] (F21) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4601, FIG. 46), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4603, FIG. 46), a position sensor unit (e.g., position sensor unit 4607, FIG. 46), and a processing unit (e.g., processing unit 4605, FIG. 46). The processing unit is coupled to the touch-sensitive surface unit, the display unit, and the position sensor unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 46 illustrates display unit 4601 and touch-sensitive surface unit 4603 as being integrated into electronic device 4600, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into one touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a display unit (e.g., display unit 4609, FIG. 46), an extraction unit (e.g., extraction unit 4611, FIG. 46), a judgment unit (e.g., judgment unit 4613, FIG. 46), a storage unit (e.g., storage unit 4615, FIG. 46), a determination unit (e.g., determination unit 4617, FIG. 46), a selection unit (e.g., selection unit 4619, FIG. 46), a receiving unit (e.g., receiving unit 4621, FIG. 46), a providing unit (e.g., providing unit 4623, FIG. 46), and a playback unit (e.g., playback unit 4625, FIG. 46).The processing unit (or one or more components thereof, such as units 4607 to 4625) is configured to automatically and without instructions from the user, when it determines (e.g., by the determining unit 4613) that a user of the electronic device is in a vehicle stopped at the geographical location, and when it determines that the user has left the vehicle at the geographical location, determine (e.g., by the determining unit 4613) whether location information retrieved (e.g., by the retrieving unit 4611) from the location sensor unit (e.g., the location sensor unit 4607) satisfies an accuracy criterion to identify the geographical location (e.g., by the identifying unit 4617), and when it determines (e.g., by the determining unit 4613) that the location information does not satisfy the accuracy criterion, provide (e.g., by the providing unit 4623) a prompt to the user to input information regarding the geographical location, and in response to providing the prompt, receive (e.g., by the receiving unit 4621) information about the geographical location from the user and store (e.g., by the storage unit 4615) the information as vehicle location information.

[0190] (F22) In some embodiments of the electronic device described in F21, the processing unit is further configured to, upon determining that the location information satisfies the accuracy criterion, automatically and without instruction from a user, store the location information as vehicle location information (e.g., by the storage unit 4615).

[0191] (F23) In some embodiments of the electronic device described in F22, the processing unit is further configured to display (e.g., by the display unit 4609 in conjunction with the display unit 4601) a user interface object including vehicle location information in accordance with a determination that the user is heading towards the geographic location.

[0192] (F24) In some embodiments of the electronic device described in F23, the user interface object is a map object that includes an identifier for the user's current location and another identifier for the geographic location.

[0193] (F25) In some embodiments of the electronic device described in F24, the user interface object is displayed on a lock screen of the electronic device (eg, by display unit 4609 in conjunction with display unit 4601).

[0194] (F26) In some embodiments of the electronic device described in F24, the user interface object is displayed (e.g., by display unit 4609 in conjunction with display unit 4601) in response to a swipe gesture that causes the electronic device to enter a search mode.

[0195] (F27) In some embodiments of the electronic device described in F26, determining whether the user is heading towards the geographic location is performed in response to receiving a swipe gesture.

[0196] (F28) In some embodiments of the electronic device described in any one of F21 to F27, the prompt is a voice prompt provided by a virtual assistant available via the electronic device, and receiving information from the user includes receiving (e.g., by the receiving unit 4621) a verbal description from the user identifying the geographic location, and displaying the user interface objects includes displaying (e.g., by the display unit 4609 in conjunction with the display unit 4601) a selectable affordance that, when selected (e.g., by the selection unit 4619), causes the device to play (e.g., by the playback unit 4625) the verbal description.

[0197] (F29) In some embodiments of the electronic device described in any one of F21 to F27, the prompt is displayed on a display of the electronic device (e.g., by a display unit 4609 in conjunction with the display unit 4601), receiving information from the user includes receiving a textual description from the user identifying the geographic location (e.g., by the receiving unit 4621), and displaying the user interface object includes displaying the textual description from the user.

[0198] (F30) In some embodiments of the electronic device described in any one of F21 to F27, determining whether the user is moving towards the geographical location includes determining (e.g., by the determining unit 4613) that the electronic device is moving towards the geographical location using new location information received (e.g., by the receiving unit 4621) from a location sensor unit (e.g., the location sensor unit 4607).

[0199] (F31) In some embodiments of the electronic device described in F30, determining whether the user is heading towards the geographic location includes (i) determining (e.g., by the determining unit 4613) that the electronic device has remained at the different geographic location for more than a threshold time, and (ii) determining (e.g., by the determining unit 4613) that new location information indicates that the electronic device is leaving the different geographic location and heading towards the geographic location.

[0200] (F32) In some embodiments of the electronic device described in any one of F21 to F31, determining that the user is in a vehicle stopped at the geographical location includes: (i) determining that the user is in the vehicle by determining (e.g., by the determining unit 4613) that the electronic device is moving above a threshold speed; and (ii) determining that the vehicle has stopped at the geographical location by one or more of: (a) determining that the electronic device remains at the geographical location for more than a threshold time (e.g., by the determining unit 4613); (b) determining that a communication link between the electronic device and the vehicle has been disconnected (e.g., by the determining unit 4613); and (c) determining that the geographical location corresponds to a location in a parking lot (e.g., by the determining unit 4613).

[0201] (F33) In some embodiments of the electronic device described in F32, determining that the vehicle has stopped at the geographic location includes determining (e.g., by the determination unit 4613) that the electronic device has remained at the geographic location for more than a threshold time.

[0202] (F34) In some embodiments of the electronic device described in F32 or F33, determining that the vehicle has stopped at the geographic location includes determining (e.g., by the determining unit 4613) that a communication link between the electronic device and the vehicle has been disconnected.

[0203] (F35) In some embodiments of the electronic device described in any one of F32 to F34, determining that the vehicle has stopped at the geographic location includes determining (e.g., by the determination unit 4613) that the geographic location corresponds to a location within a parking lot.

[0204] (F36) In some embodiments of the electronic device according to any one of F21 to F35, the accuracy criterion comprises a criterion that is met if the accuracy of the GPS readings associated with the location information exceeds a threshold level of accuracy.

[0205] (G1) In some embodiments, a method is performed in an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a position sensor, a touch-sensitive surface (e.g., touch-sensitive surface 195, FIG. 1D ), and a display (e.g., display 194, FIG. 1D ). The method includes monitoring a geographic location of the electronic device using the position sensor. The method further includes determining, based on the monitored geographic location, that the electronic device is within a threshold distance of a predetermined type of point of interest. The method also includes identifying at least one activity currently popular at the point of interest in accordance with determining that the electronic device is within the threshold distance of the point of interest, retrieving information about the point of interest, and retrieving information about the at least one activity currently popular at the point of interest. The method further includes detecting a first input via the touch-sensitive surface that, when detected, causes the electronic device to transition to a search mode, and in response to detecting the first input, transitioning to the search mode includes presenting, prior to receiving any user input at the search interface, an affordance via the display including (i) information regarding at least one activity and (ii) an indication that the at least one activity has been identified as currently popular in the location of interest.

[0206] (G2) In some embodiments of the method of claim G1, the method includes detecting a second input and, in response to detecting the second input, updating the affordance to include available information about a current activity at a second point of interest different from the point of interest, the point of interest being within a threshold distance of the electronic device.

[0207] (G3) In some embodiments of the method described in G1 or G2, the affordance further includes a selectable category of points of interest, the method further including detecting a selection of a corresponding selectable category, and in response to detecting the selection, updating the affordance to include information regarding additional points of interest located within a second threshold distance of the device.

[0208] (G4) In some embodiments of the method of any one of G1 to G3, the point of interest is an amusement park and the retrieved information includes current wait times for rides at the amusement park.

[0209] (G5) In some embodiments of the method of claim G4, the retrieved information includes information regarding waiting times for rides located within a predefined distance of the electronic device.

[0210] (G6) In some embodiments of the method of any one of G1 to G3, the point of interest is a restaurant and the retrieved information includes information about popular menu items of the restaurant.

[0211] (G7) In some embodiments of the method of claim G6, the retrieved information is retrieved from a social network associated with the user of the electronic device.

[0212] (G8) In some embodiments of the method according to any one of G1 to G3, the point of interest is a movie theater and the retrieved information includes information regarding movie theater showtimes.

[0213] (G9) In some embodiments of the method according to claim G8, the retrieved information is retrieved from a social network associated with the user of the electronic device.

[0214] (G10) In some embodiments of the methods described in any one of G1 to G9, after the electronic device is unlocked, the affordance is available in response to a substantially horizontal swipe on an initial page of a home screen of the electronic device.

[0215] (G11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, a position sensor, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in any one of G1 to G10.

[0216] (G12) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, a position sensor, and means for performing a method according to any one of G1 to G10.

[0217] (G13) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface, a display, and a position sensor, cause the electronic device to perform a method according to any one of G1 to G10.

[0218] (G14) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface, a display, and a position sensor is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method of any one of G1 to G10. In a further aspect, an information processing apparatus is provided for use in an electronic device including a touch-sensitive surface, a display, and a position sensor. The information processing apparatus includes means for performing a method of any one of G1 to G10.

[0219] (G15) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4701, FIG. 47), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4703, FIG. 47), a position sensor unit 4707, and a processing unit (e.g., processing unit 4705, FIG. 47). The processing unit is coupled to the touch-sensitive surface unit, the display unit, and the position sensor unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 47 illustrates display unit 4701 and touch-sensitive surface unit 4703 as being integrated into electronic device 4700, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. In some embodiments, the touch-sensitive surface unit and the display unit are integrated into one touch-sensitive display unit (also referred to herein as a touch-sensitive display). The processing unit includes a detection unit (e.g., detection unit 4709, FIG. 47), a display unit (e.g., display unit 4711, FIG. 47), an extraction unit (e.g., extraction unit 4713, FIG. 47), a judgment unit (e.g., judgment unit 4715, FIG. 47), an identification unit (e.g., identification unit 4717, FIG. 47), an unlocking unit (e.g., unlocking unit 4719, FIG. 47), and a search mode transition unit (e.g., search mode transition unit 4721, FIG. 47).The processing unit (or one or more components thereof, such as units 4709 to 4721), without receiving any instruction from a user of the electronic device, monitors a geographical location of the electronic device using a location sensor unit (e.g., location sensor unit 4707), determines (e.g., by determining unit 4715) that the electronic device is within a threshold distance of a predetermined type of point of interest based on the monitored geographical location, identifies (e.g., by identifying unit 4717) at least one activity that is currently popular at the point of interest according to determining that the electronic device is within the threshold distance of the point of interest, and retrieves information about the at least one activity that is currently popular at the point of interest (e.g., by retrieving unit 4713), including retrieving information about the at least one activity that is currently popular at the point of interest. ), configured to detect (e.g., by detection unit 4709) via a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4703) a first input that, when detected, causes the electronic device to transition to a search mode, and in response to detecting the first input, transition to the search mode (e.g., by search mode transition unit 4721), where transitioning to the search mode includes presenting (e.g., by display unit 4711) via a display unit (e.g., display unit 4701) an affordance including (i) information regarding the at least one activity and (ii) an indication that the at least one activity has been identified as currently popular in the location of interest, prior to receiving any user input at the search interface.

[0220] (G16) In some embodiments of the electronic device described in G15, the processing unit is further configured to detect a second input (e.g., by the detection unit 4709) and, in response to detecting the second input, update the affordance (e.g., by the display unit 4711) to include available information about a current activity at a second point of interest different from the point of interest, where the point of interest is within a threshold distance of the electronic device.

[0221] (G17) In some embodiments of the electronic device described in G15 or G16, the affordance further includes a selectable category of points of interest, and the processing unit is further configured to detect (e.g., by the detection unit 4709) a selection of a corresponding selectable category and, in response to detecting the selection, update (e.g., by the display unit 4711) the affordance to include information about additional points of interest located within a second threshold distance of the device.

[0222] (G18) In some embodiments of the electronic device described in any one of G15 to G17, the point of interest is an amusement park and the retrieved information includes current wait times for rides at the amusement park.

[0223] (G19) In some embodiments of the electronic device described in G18, the retrieved information includes information regarding waiting times for rides located within a predefined distance range of the electronic device.

[0224] (G20) In some embodiments of the electronic device described in any one of G15 to G17, the point of interest is a restaurant and the retrieved information includes information about popular menu items of the restaurant.

[0225] (G21) In some embodiments of the electronic device described in G20, the retrieved information is retrieved from a social network associated with a user of the electronic device.

[0226] (G22) In some embodiments of the electronic device described in any one of G15 to G17, the point of interest is a movie theater and the retrieved information includes information about movie theater showtimes.

[0227] (G23) In some embodiments of the electronic device described in G22, the retrieved information is retrieved from a social network associated with a user of the electronic device.

[0228] (G24) In some embodiments of the electronic device described in any one of G15 to G23, after the electronic device is unlocked (e.g., by the unlocking unit 4719), the affordance is available in response to a substantially horizontal swipe on an initial page of a home screen of the electronic device.

[0229] (H1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (touch screen 112, as shown in FIG. 1C , in some embodiments the touch-sensitive surface and the display are integrated). The method includes receiving at least a portion of a voice communication (e.g., a live call or a prerecorded voicemail of 10 seconds or less), the portion of the voice communication including speech provided by a remote user of a remote device that is different from a user of the electronic device. The method also includes extracting a content item based at least in part on the speech provided by the remote user of the remote device. The method further includes determining whether the content item is currently available on the electronic device. The method includes, in accordance with a determination that the content item is not currently available on the electronic device, (i) identifying an application associated with the content item and (ii) displaying a selectable description of the content item on the display. The method includes storing the content item for presentation in the identified application (e.g., storing the content item so that it is available for presentation by the identified application) in response to detecting a selection of the selectable description. In this manner, a user can remember content items mentioned or spoken in a voice communication without having to remember all of the "spoken" details and later enter those details to create an appropriate content item. Instead, the electronic device can detect and extract the content items based on the speech describing various corresponding content items, and then provide selectable descriptions of the content items from which the user can select and store the corresponding content items on the electronic device.

[0230] (H2) In some embodiments of the method described in H1, the content item is a new event.

[0231] (H3) In some embodiments of the method described in H1, the content item is new event details for an event currently associated with a calendar application on the electronic device.

[0232] (H4) In some embodiments of the method described in H1, the content item is a new contact.

[0233] (H5) In some embodiments of the method described in H1, the content item is new contact information for an existing contact associated with a phone application on the electronic device.

[0234] (H6) In some embodiments of the method according to any one of H1 to H5, the voice communication is a live call.

[0235] (H7) In some embodiments of the method according to any one of H1 to H5, the voice communication is a live FaceTime call.

[0236] (H8) In some embodiments of the method of any one of H1 to H5, the voice communication is a pre-recorded voicemail.

[0237] (H9) In some embodiments of the method of any one of H1 to H8, displaying the selectable description includes displaying, within the user interface, the selectable description including recent calls made using the phone application. In this manner, the user may easily and conveniently access the extracted content items (e.g., extracted during a corresponding voice communication) including the recent calls directly from the user interface.

[0238] (H10) In some embodiments of the method as set forth in H9, the selectable description is displayed with an indication that the content item is associated with an audio communication.

[0239] (H11) In some embodiments of the method as recited in H9, detecting the selection includes receiving the selection while a user interface is displayed that includes the recent calls.

[0240] (H12) In some embodiments of the method according to any one of H1 to H11, the method further comprises, in conjunction with displaying the selectable description of the content item, providing feedback to a user of the electronic device that the content item has been detected (e.g. haptic feedback generated by the electronic device or the presentation of a user interface object on the second device, so that the user does not have to remove the phone from his / her ear during a call). In this way, during a voice communication, a concise indication is provided to the user that the content item has been detected / extracted, after which the user can decide whether to store the content item.

[0241] (H13) In some embodiments of the method according to H12, providing feedback includes transmitting information about the detection of the content item to a different electronic device (e.g., a laptop computer, a television monitor, a smart watch, etc.) in the vicinity of the electronic device. In this way, the user does not need to interrupt the voice communication and can still view details about the detected / extracted content item on the different device.

[0242] (H14) In some embodiments of the method of any one of H1 to H13, the method further includes determining that the voice communication includes information about the first physical location (e.g., an address mentioned during the call or a restaurant name spoken during the call, etc., as described in further detail below). The method also includes detecting the input and, in response to detecting the input, opening an application capable of accepting location data and populating the application with information about the first physical location. In this manner, the electronic device, in addition to detecting and extracting the event and contact information, can extract location information spoken in the voice communication and provide this location information to the user in an appropriate application (e.g., so that the user is not burdened with remembering specific location details spoken in the call, especially new details that may be unfamiliar to the user, but rather the device extracts these location details and displays them for use by the user).

[0243] (H15) In some embodiments of the method described in H14, the application is a map application, and populating the map application with information about the first physical location includes populating a map displayed within the map application with a location identifier corresponding to the first physical location. In this manner, the user can easily use the newly extracted location details to navigate to new destinations, view how far away a particular location is, and easily use other functionality provided by the map application.

[0244] (H16) In some embodiments of the method of any one of H1 to H13, the method further includes determining that the voice communication includes information about the first physical location. The method also includes detecting an input (e.g., a search activation gesture, such as a swipe gesture, described in more detail below) and, in response to detecting the input, populating a search interface with information about the first physical location. In this manner, in addition to (or instead of) providing the location information to the user for use in a particular application, the electronic device can also provide the location information for use in the search interface (e.g., to search for related points of interest or to search for additional details about the first physical location, such as phone numbers, menus, etc.).

[0245] (H17) In some embodiments of the methods described in any one of H1 to H16, extracting the content item includes analyzing a portion of the voice communication for detection of a predetermined type of content, and the analysis is performed while outputting the voice communication via an audio system in communication with the electronic device (e.g., the voice communication is analyzed in real time while the voice communication is being output to a user of the electronic device).

[0246] (H18) In some embodiments of the method described in H17, analyzing the voice communication includes (i) converting speech provided by a remote user of the remote device into text; (ii) applying a natural language processing algorithm to the text to determine whether the text includes one or more predefined keywords; and (iii) determining that the voice communication includes speech describing the content item according to a determination that the text includes the corresponding predefined keywords.

[0247] (H19) In some embodiments of the methods described in any one of H1 to H18, receiving at least a portion of the voice communication includes receiving an indication from a user of the electronic device that the portion of the voice communication should be analyzed.

[0248] (H20) In some embodiments of the method described in H19, the indication corresponds to the selection of a hardware button (e.g., a user selects a hardware button while a voice communication is being output by an audio system to indicate that a predefined number of seconds (e.g., the previous 10, 15, or 20 seconds) of the voice communication should be analyzed). In some embodiments, the button may also be a button presented on a display of the electronic device for user selection (e.g., a button displayed during a voice communication that reads "Tap here to analyze new content of this voice communication").

[0249] (H21) In some embodiments of the method described in H19, the indication corresponds to a command from a user of the electronic device that includes the words "hey Siri." Thus, the user can simply instruct the electronic device to begin analyzing a portion of the voice communication to detect content items (such as information about events, contact information, and physical locations) that were spoken in the voice communication.

[0250] (H22) In some embodiments of the method according to any one of H1 to H21, the method further includes receiving a second portion of the voice communication. The second portion includes speech provided by a remote user of the remote device and speech provided by a user of the electronic device (e.g., the voice communication is a live call and the second portion includes a conversation between the user and the remote user). The method also includes extracting a second content item based at least in part on the speech provided by the remote user of the remote device and the speech provided by the user of the electronic device. In accordance with a determination that the second content item is not currently available on the electronic device, the method includes (i) identifying a second application associated with the second content item and (ii) displaying a second selectable description of the second content item on the display. In response to detecting a selection of the second selectable description, the method includes storing the second content item for presentation in the identified second application.

[0251] (H23) In some embodiments of the method described in H22, a selectable description and a second selectable description are displayed within a user interface that includes recent calls made using a phone application. In this manner, the user is provided with one interface that conveniently includes content items found in several voice communications (e.g., several calls, voicemails, or calls and voicemails).

[0252] (H24) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs when executed by the one or more processors causing the electronic device to perform a method according to any one of H1 to H23.

[0253] (H25) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of H1 to H23.

[0254] (H26) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of H1 to H23.

[0255] (H27) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive surface and a display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of H1 to H23.

[0256] (H28) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of H1 to H23.

[0257] (H29) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4801, FIG. 48), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4803, FIG. 48), and a processing unit (e.g., processing unit 4805, FIG. 48). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 48 illustrates display unit 4801 and touch-sensitive surface unit 4803 as being integrated into electronic device 4800, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an audio communication receiving unit (e.g., audio communication receiving unit 4807, FIG. 48), a content item extracting unit (e.g., content item extracting unit 4809, FIG. 48), an availability determining unit (e.g., availability determining unit 4811, FIG. 48), an application identifying unit (e.g., application identifying unit 4813, FIG. 48), a displaying unit (e.g., displaying unit 4815, FIG. 48), a content item storing unit (e.g., content item storing unit 4817, FIG. 48), a feedback providing unit (e.g., feedback providing unit 4819, FIG. 48), an input detecting unit (e.g., input detecting unit 4821, FIG. 48), an application opening unit (e.g., application opening unit 4823, FIG. 48), a populating unit (e.g., populating unit 4825, FIG. 48), and an audio communication analyzing unit (e.g., audio communication analyzing unit 4827, FIG. 48). The processing unit (or one or more components thereof, such as units 4807 to 4827) is configured to receive (e.g., by the audio communication receiving unit 4807) at least a portion of an audio communication, the portion of the audio communication including speech provided by a remote user of a remote device different from the user of the electronic device.The processing unit is further configured to extract a content item based at least in part on speech provided by a remote user of the remote device (e.g., by the content item extraction unit 4809) and determine whether the content item is currently available on the electronic device (e.g., by the availability determination unit 4811). Following a determination that the content item is not currently available on the electronic device, the processing unit is further configured to (i) identify an application associated with the content item (e.g., by the application identification unit 4813), and (ii) cause a selectable description of the content item to be displayed on the display (e.g., by the display unit 4815 and / or the display unit 4801). In response to detecting a selection of the selectable description (e.g., by the input detection unit 4821 and / or the touch-sensitive surface unit 4803), the processing unit is configured to store (e.g., by the content item storage unit 4817) the content item for presentation in the identified application.

[0258] (H30) In some embodiments of the electronic device described in H29, the processing unit (or one or more components thereof, such as units 4907 to 4927) is further configured to perform a method according to any one of H2 to H23.

[0259] (I1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes receiving at least a portion of a voice communication, the portion of the voice communication (e.g., a live call, a recorded voicemail) including speech provided by a remote user of a remote device that is different from a user of the electronic device. The method also includes determining that the voice communication includes speech identifying the physical location. In response to determining that the voice communication includes speech identifying the physical location, the method includes providing an indication that information regarding the physical location has been detected (e.g., providing haptic feedback to the user, displaying a user interface object with information regarding the physical location, or transmitting information regarding the physical location to a nearby device for display at the nearby device). The method further includes detecting an input via the touch-sensitive surface. The method includes, in response to detecting the input, (i) opening an application that accepts geographic location data, and (ii) populating the application with information about the physical location. In this manner, a user can store information about a physical location mentioned or spoken in a voice communication without having to remember all of the details that were spoken and later enter those details in an appropriate application. Instead, the electronic device can detect and extract information about the physical location based on speech describing the physical location (e.g., a description of a restaurant, driving directions to the physical location, etc.) and then provide an indication that information about the corresponding physical location has been detected.

[0260] (I2) In some embodiments of the method described in I1, the voice communication is a live call.

[0261] (I3) In some embodiments of the method described in I1, the voice communication is a live FaceTime call.

[0262] (I4) In some embodiments of the method described in I1, the voice communication is a pre-recorded voicemail.

[0263] (I5) In some embodiments of the methods of any one of I1 to I4, providing the indication includes displaying within the user interface a selectable description of a physical location that includes recent calls made using a phone application.

[0264] (I6) In some embodiments of the method described in I5, the selectable description indicates that the content item is associated with voice communication.

[0265] (I7) In some embodiments of the method described in I5 or I6, detecting the input includes detecting input on a selectable description while a user interface is displayed that includes the recent calls.

[0266] (I8) In some embodiments of the methods of any one of I1 to I7, providing the indication includes providing tactile feedback to a user of the electronic device.

[0267] (I9) In some embodiments of the methods of any one of I1 to I8, providing the indication includes transmitting information about the physical location to a different electronic device in the vicinity of the electronic device.

[0268] (I10) In some embodiments of the methods of any one of I1 to I9, determining that the voice communication includes speech describing a physical location includes analyzing a portion of the voice communication to detect information regarding the physical location, the analysis being performed while outputting the voice communication via an audio system in communication with the electronic device.

[0269] (I11) In some embodiments of the methods described in any one of I1 to I10, receiving at least a portion of the voice communication includes receiving an instruction from a user of the electronic device that the portion of the voice communication should be analyzed.

[0270] (I12) In some embodiments of the method of I11, the instruction corresponds to selection of a hardware button. In some embodiments, the button may also be a button presented on a display of the electronic device for user selection (e.g., a button that appears during a voice communication saying "tap here to analyze new content of this voice communication").

[0271] (I13) In some embodiments of the method described in I11, the instruction corresponds to a command from a user of the electronic device that includes the words "hey Siri."

[0272] (I14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in any one of I1 to I13.

[0273] (I15) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of any one of I1 to I13.

[0274] (I16) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method as described in any one of I1 to I13.

[0275] (I17) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive surface and a display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of I1 to I13.

[0276] (I18) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of I1 to I13.

[0277] (I19) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4901, FIG. 49), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4903, FIG. 49), and a processing unit (e.g., processing unit 4905, FIG. 49). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 49 illustrates display unit 4901 and touch-sensitive surface unit 4903 as being integrated into electronic device 4900, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an audio communication receiving unit (e.g., audio communication receiving unit 4907, FIG. 49), a content item extraction unit (e.g., content item extraction unit 4909, FIG. 49), an indication providing unit (e.g., indication providing unit 4911, FIG. 49), an input detection unit (e.g., input detection unit 4913, FIG. 49), an application opening unit (e.g., application opening unit 4915, FIG. 49), an application populating unit (e.g., application populating unit 4917, FIG. 49), a feedback providing unit (e.g., feedback providing unit 4919, FIG. 49), and a audio communication analysis unit (e.g., audio communication analysis unit 4921, FIG. 49). The processing unit (or one or more components thereof, such as units 4907 to 4921) is configured to receive at least a portion of an audio communication, the portion of the audio communication including speech provided (e.g., by the audio communication receiving unit 4907) by a remote user of a remote device different from a user of the electronic device. The processing unit is further configured to determine (e.g., by the content item extraction unit 4909) that the audio communication includes speech identifying a physical location.In response to determining that the voice communication includes speech identifying the physical location, the processing unit is configured to provide an indication that information related to the physical location has been detected (e.g., by content item extraction unit 4909). The processing unit is also configured to detect input via the touch-sensitive surface unit (e.g., by input detection unit 4911). In response to detecting the input, the processing unit is configured to (i) open an application that accepts geographic location data (e.g., by application opening unit 4913) and (ii) populate the application with information related to the physical location (e.g., by application populating unit 4915).

[0278] (I20) In some embodiments of the electronic device described in I19, the processing unit (or one or more components thereof, such as units 4907 to 4921) is further configured to perform a method according to any one of I2 to I13.

[0279] (J1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes presenting a text entry field and a conversation record in a message application on the display. The method also includes determining, while the message application is presented on the display, that a next possible input from a user of the electronic device is information related to a physical location. The method further includes analyzing content associated with the text entry field and the conversation record to determine a suggested physical location based at least in part on a portion of the analyzed content. The method further includes presenting, within the message application on the display, a selectable user interface element that identifies the suggested physical location and receiving a selection of the selectable user interface element. The method includes presenting a representation of the suggested physical location in the text entry field in response to receiving the selection. In this manner, a user of an electronic device is conveniently provided with the content they need, without the user having to type anything or search for the content (e.g., a user can enter their current location by simply selecting a selectable user interface element, without having to access a maps application to reveal their exact location, return to a messages application, and make explicit input to send the location information).

[0280] (J2) In some embodiments of the method described in J1, the message application includes a virtual keyboard, and the selectable user interface element is displayed in a suggestions portion adjacent to and above the virtual keyboard.

[0281] (J3) In some embodiments of the method described in J1 or J2, determining that a next possible input from a user of the electronic device is information regarding a physical location includes processing the text input field and content associated with the conversation record to detect that the conversation transcription includes a question regarding the user's current location. In this manner, the user is provided with a suggested physical location that is directly related to a description in the conversation transcription (e.g., in response to a second user's question of "where are you," the user is presented with a user interface object that, when selected, causes the device to transmit information regarding the user's current location to the second user).

[0282] (J4) In some embodiments of the method described in J3, processing the content includes applying a natural language processing algorithm to detect one or more predefined keywords that form a question (e.g., "Where are you" or "What is your address").

[0283] (J5) In some embodiments of the method described in J3 or J4, the question is included in a message received from a second user different from the user.

[0284] (J6) In some embodiments of the method according to any one of J1 to J5, determining that a next possible input from a user of the electronic device is information regarding a physical location includes monitoring typing input received from the user in a text input portion of a messaging application.

[0285] (J7) In some embodiments of the method of any one of J1 to J6, the method further includes ceasing presentation of the selectable user interface element in accordance with a determination that the user is typing and has not selected a selectable user interface element. In this manner, the device does not continue to present selectable user interface objects if it can be determined that the user is not interested in selecting objects.

[0286] (J8) In some embodiments of the method of any one of J1 to J7, the method further includes ceasing presentation of the selectable user interface element in accordance with a determination that the user has provided additional input indicating that the user does not want to select the selectable user interface element. In this manner, the device does not continue to present the selectable user interface object if it can be determined that the user is not interested in selecting the object.

[0287] (J9) In some embodiments of the method according to any one of J1 to J5, the representation of the proposed physical location includes information identifying a current geographic location of the electronic device.

[0288] (J10) In some embodiments of the method according to any one of J1 to J9, the proposed representation of the physical location is an address.

[0289] (J11) In some embodiments of the method according to any one of J1 to J9, the proposed physical location is a map object including an identifier for the proposed physical location.

[0290] (J12) In some embodiments of the method according to any one of J1 to J11, the suggested physical location corresponds to a location recently viewed by the user in an application other than a messaging application.

[0291] (J13) In some embodiments of the method according to any one of J1 to J12, the message application is an email application.

[0292] (J14) In some embodiments of the method according to any one of J1 to J12, the message application is a text message application.

[0293] (J15) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs when executed by the one or more processors causing the electronic device to perform a method as described in any one of J1 to J14.

[0294] (J16) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of J1 to J14.

[0295] (J17) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method as described in any one of J1 to J14.

[0296] (J18) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to a method described in any one of J1 to J14.

[0297] (J19) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of J1 to J14.

[0298] (J20) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5001, FIG. 50), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5003, FIG. 50), and a processing unit (e.g., processing unit 5005, FIG. 50). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 50 illustrates display unit 5001 and touch-sensitive surface unit 5003 as being integrated into electronic device 5000, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5007, FIG. 50), a next input determination unit (e.g., next input determination unit 5009, FIG. 50), a content analysis unit (e.g., content analysis unit 5011, FIG. 50), a selection receiving unit (e.g., selection receiving unit 5013, FIG. 50), a typing input monitoring unit (e.g., typing input monitoring unit 5015, FIG. 50), and a presentation termination unit (e.g., presentation termination unit 5017, FIG. 50). The processing unit (or one or more components thereof such as units 5007 to 5017) is configured to present (e.g., by the presentation unit 5007 and / or the display unit 5001) a text input field and a conversation record in a message application on the display. While the message application is presented on the display, the processing unit is also configured to determine (e.g., by the next input determination unit 5009) that a next possible input from a user of the electronic device is information about a physical location.The processing unit is further configured to analyze content associated with the text input field and the conversation record to determine a suggested physical location based at least in part on a portion of the analyzed content (e.g., by the content analysis unit 5011), present a selectable user interface element within the message application on the display (e.g., by the presentation unit 5007) identifying the suggested physical location, receive a selection of the selectable user interface element (e.g., by the selection receiving unit 5013 and / or the touch-sensitive surface unit 5003), and, in response to receiving the selection, present a representation of the suggested physical location within the text input field (e.g., by the presentation unit 5007).

[0299] (J21) In some embodiments of the electronic device described in J20, the processing unit (or one or more components thereof, such as units 5007 to 5017) is further configured to perform a method described in any one of J2 to J14.

[0300] (K1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes obtaining, while displaying the first application, information identifying a first physical location viewed by a user in the first application (e.g., a restaurant searched for by a user in an application enabling searching for local businesses). The method also includes closing the first application and, after closing the first application, receiving a request from the user to open a second application, the second application being different from the first application. In some embodiments, the request is received without receiving any input in the first application (e.g., the request does not include clicking a link or button in the first application). In response to receiving the request and following a determination that the second application is capable of accepting geographic location information, the method includes presenting the second application, where presenting the second application includes populating the second application with information based at least in part on the information identifying the first physical location. In this manner, a user does not need to manually transfer information between two different applications. Instead, the device intelligently determines that the second application is capable of accepting geographic location information, and then populates the second application directly with information related to the physical location viewed in the first application (e.g., populating a map object in the second application to include an identifier for the physical location).

[0301] (K2) In some embodiments of the method according to K1, receiving the request to open the second application includes detecting an input on an affordance of the second application after the first application is closed. In other words, the request does not correspond to clicking on a link in the first application, but instead explicitly and directly requests that the user open the second application, and the device then determines to populate the second application with information about a previously viewed physical location (previously viewed in a different first application) such that the user can further explore or consider this previously viewed physical location in the second application.

[0302] (K3) In some embodiments of the method described in K2, the affordance of the second application is an icon displayed within a home screen of the electronic device. In some embodiments, the home screen is a system-level component of an operating system and includes icons for invoking applications available on the electronic device.

[0303] (K4) In some embodiments of the method described in K2, detecting the input includes (i) detecting a double tap on a physical home button; (ii) in response to detecting the double tap, displaying an application switching user interface; and (iii) detecting a selection of an affordance from within the application switching user interface.

[0304] (K5) In some embodiments of the methods described in any one of K1 to K4, populating the second application includes displaying a user interface object including information based, at least in part, on information identifying the first physical location.

[0305] (K6) In some embodiments of the method described in K5, the user interface object includes a textual description informing the user that the first physical location was recently viewed in the first application.

[0306] (K7) In some embodiments of the method described in K6, the user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location.

[0307] (K8) In some embodiments of the methods described in K6 or K7, the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard.

[0308] (K9) In some embodiments of the method described in any one of K6 to K8, obtaining information includes obtaining information regarding the second physical location, and displaying the user interface object includes displaying a user interface object having information regarding the second physical location.

[0309] (K10) In some embodiments of the methods described in any one of K1 to K9, determining that the second application is capable of accepting the geographic location information includes one or more of: (i) determining that the second application includes an input receiving field capable of accepting and processing geographic location data; (ii) determining that the second application is capable of displaying the geographic location information on a map; (iii) determining that the second application is capable of using the geographic location information to facilitate route guidance; and (iv) determining that the second application is capable of using the geographic location information to locate and provide transportation services.

[0310] (K11) In some embodiments of the method described in K10, determining that the second application is capable of accepting geographic location information includes determining that the second application includes an input receiving field capable of accepting and processing geographic location data, the input receiving field being a search box that enables searching within a map displayed within the second application.

[0311] (K12) In some embodiments of the method according to any one of K1 to K11, the method further includes, in response to receiving the request, determining whether the second application is associated with the first application (e.g., opened a threshold number of times after opening the first application) based on the user's application usage history.

[0312] (K13) In some embodiments of the method described in K12, the method further includes providing the second application with access to information identifying the first physical location before presenting the second application, where before the access is provided, the second application does not have access to information identifying the first physical location. In this manner, the second application can receive information regarding an action taken by the user in the first application, and the user is then provided with a way to use this information within the second application (e.g., to retrieve further information regarding the first physical location or to use the first physical location for any service, such as a ride-sharing service, available through the second application).

[0313] (K14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs when executed by the one or more processors causing the electronic device to perform a method as described in any one of K1 to K13.

[0314] (K15) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of K1 to K13.

[0315] (K16) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of K1 to K13.

[0316] (K17) In a further aspect, there is provided a graphical user interface on an electronic device having a touch-sensitive surface and a display. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of K1 to K13.

[0317] (K18) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of K1 to K13.

[0318] (K19) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5101, FIG. 51), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5103, FIG. 51), and a processing unit (e.g., processing unit 5105, FIG. 51). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 51 illustrates display unit 5101 and touch-sensitive surface unit 5103 as being integrated into electronic device 5100, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an information acquiring unit (e.g., information acquiring unit 5107, FIG. 51), an application termination unit (e.g., application termination unit 5109, FIG. 51), a request receiving unit (e.g., request receiving unit 5111, FIG. 51), an application performance determining unit (e.g., application performance determining unit 5113, FIG. 51), an application presenting unit (e.g., application presenting unit 5115, FIG. 51), an application populating unit (e.g., application populating unit 5117, FIG. 51), an input detecting unit (e.g., input detecting unit 5119, FIG. 51), an application switching user interface display unit (e.g., application switching user interface display unit 5121, FIG. 51), an application association determining unit (e.g., application association determining unit 5123, FIG. 51), and an access providing unit (e.g., access providing unit 5125, FIG. 51). The processing unit (or one or more components thereof, such as units 5107 to 5125) is configured to obtain (e.g., by the information obtaining unit 5107) information identifying a first physical location viewed by the user in the first application while displaying the first application.The processing unit is also configured to terminate the first application (e.g., by application termination unit 5109) and, after termination of the first application, receive a request from the user to open a second application different from the first application (e.g., by request receiving unit 5111). In response to receiving the request and according to a determination (e.g., a determination processed or performed by application capabilities determining unit 5113) that the second application can accept geographic location information, present the second application (e.g., by application presenting unit 5115), where presenting the second application includes populating the second application with information based at least in part on the information identifying the first physical location (e.g., by application populating unit 5117).

[0319] (K20) In some embodiments of the electronic device described in K19, the processing unit (or one or more components thereof, such as units 5107 to 5125) is further configured to perform a method according to any one of K2 to K13.

[0320] (L1) In some embodiments, a method is performed in an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes obtaining information identifying a first physical location viewed by a user in a first application and detecting a first input. The method includes, in response to detecting the first input, (i) identifying a second application capable of accepting geographic location information, and (ii) presenting, on at least a portion of the display, an affordance having a suggestion to open the second application, different from the first application, the second application having information related to the first physical location. The method also includes detecting the second input in the affordance. In response to detecting the second input in the affordance, (i) opening the second application, and (ii) populating the second application to include information based, at least in part, on the information identifying the first physical location.

[0321] Compared to the operation related to K1 above, the operation related to L1 does not receive a specific request from the user to open the second application before providing the user with a proposal to open the second application having information about the first physical location. In this manner, by providing the operations related to both K1 and L1 above (as well as combinations thereof using several processing steps of each of these methods), the electronic device can provide an efficient user experience that allows the user to predictively use location data either before or after opening an application that can accept geographic location information. In addition, in an embodiment of L1 where a determination that the second application can accept geographic location information is made by L1 even before opening the second application, and thus an input corresponds to a request to open an application switching user interface, the application switching user interface simply displays a proposal to open an application (e.g., the second application) having information about the first physical location if it is known that the application can accept location data.

[0322] (L2) In some embodiments of the method described in L1, the first input corresponds to a request to open an application switching user interface (e.g., the first input is a double tap on a physical home button of the electronic device).

[0323] (L3) In some embodiments of the method described in L2, the affordance is presented within an application switching user interface.

[0324] (L4) In some embodiments of the method described in L3, presenting the affordance includes, in conjunction with presenting the affordance, presenting a representation of an application running on the electronic device (e.g., a snapshot of application content of the application) within an application switching user interface, and presenting the affordance within an area of ​​the display below the representation of the application.

[0325] (L5) In some embodiments of the method described in L1, the first input corresponds to a request to open a home screen of the electronic device (e.g., the first input is a single tap on a physical home button of the electronic device).

[0326] (L6) In some embodiments of the method described in L5, the affordance is presented on a portion of the home screen.

[0327] (L7) In some embodiments of the method of any one of L1 to L6, the suggestion includes a type-specific textual description associated with the second application.

[0328] (L8) In some embodiments of the methods described in any one of L1 to L7, populating the second application includes displaying a user interface object including information based, at least in part, on information identifying the first physical location.

[0329] (L9) In some embodiments of the method described in L8, the user interface object includes a textual description informing the user that the first physical location was recently viewed in the first application.

[0330] (L10) In some embodiments of the method described in L9, the user interface object is a map displayed within a second application, and populating the second application includes populating the map to include an identifier of the first physical location.

[0331] (L11) In some embodiments of the method described in L9 or L10, the second application is presented with a virtual keyboard and the user interface object is displayed above the virtual keyboard.

[0332] (L12) In some embodiments of the methods described in any one of L1 to L11, identifying the second application capable of accepting the geographic location information includes one or more of: (i) determining that the second application includes an input receiving field capable of accepting and processing geographic location data; (ii) determining that the second application is capable of displaying the geographic location information on a map; (iii) determining that the second application is capable of using the geographic location information to facilitate route guidance; and (iv) determining that the second application is capable of using the geographic location information to locate and provide transportation services.

[0333] (L13) In some embodiments of the method described in L12, identifying that the second application is capable of accepting geographic location information includes determining that the second application includes an input receiving field capable of accepting and processing geographic location data, the input receiving field being a search box that enables searching within a map displayed within the second application.

[0334] (L14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, cause the electronic device to perform a method as described in any one of L1 to L13.

[0335] (L15) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing the method of any one of L1 to L13.

[0336] (L16) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of L1 to L13.

[0337] (L17) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method described in any one of L1 to L13.

[0338] (L18) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of L1 to L13.

[0339] (L19) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5201, FIG. 52), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5203, FIG. 52), and a processing unit (e.g., processing unit 5205, FIG. 52). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. IE (e.g., computing devices A through D). For ease of explanation, FIG. 52 illustrates display unit 5201 and touch-sensitive surface unit 5203 as being integrated into electronic device 5200, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an information acquiring unit (e.g., information acquiring unit 5207, FIG. 52), an input detection unit (e.g., input detection unit 5209, FIG. 52), an application identifying unit (e.g., application identifying unit 5211, FIG. 52), an affordance presenting unit (e.g., affordance presenting unit 5213, FIG. 52), an application opening unit (e.g., application opening unit 5215, FIG. 52), an application populating unit (e.g., application populating unit 5217, FIG. 52), an application switching user interface presenting unit (e.g., application switching user interface presenting unit 5219, FIG. 52), and an application performance determining unit (e.g., application performance determining unit 5221, FIG. 52). The processing unit (or one or more components thereof, such as units 5207 to 5221) is configured to obtain information identifying a first physical location viewed by a user in a first application (e.g., by the information obtaining unit 5207) and detect a first input (e.g., by the input detection unit 5209).In response to detecting the first input, the processing unit is configured to (i) identify (e.g., by the application identifying unit 5209) a second application capable of accepting geographic location information, and (ii) present (e.g., by the affordance presenting unit 5213) on at least a portion of the display an affordance having a proposal to open a second application that is different from the first application and has information about the first physical location. The processing unit is also configured to detect (e.g., by the input detecting unit 5209) the second input in the affordance. In response to detecting the second input in the affordance, the processing unit is configured to (i) open (e.g., by the application opening unit 5215) and (ii) populate (e.g., by the application populating unit 5217) the second application to include information based, at least in part, on the information identifying the first physical location.

[0340] (L20) In some embodiments of the electronic device described in L19, the processing unit (or one or more components thereof, such as units 5207 to 5221) is further configured to perform a method according to any one of L2 to L13.

[0341] (M1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes obtaining information identifying a first physical location viewed by a user in a first application executing on the electronic device. The method also includes determining that the user has entered a vehicle. In response to determining that the user has entered the vehicle, providing a prompt to the user to use the first physical location as a route guidance destination. The method includes receiving an instruction from the user in response to providing the prompt to use the first physical location as a route guidance destination. The method further includes facilitating route guidance to the first physical location. In this manner, the user is conveniently provided with routing suggestions for the destination based on the previously viewed physical location in an application on the electronic device.

[0342] (M2) In some embodiments of the method of M1, the method further includes detecting that the electronic device has received a message, the message including detecting that the message includes information identifying a second physical location, and in response to the detection, providing a new prompt to the user to use the second physical location as a new destination for route directions. In this manner, the user can dynamically add a waypoint or add a new destination for route directions based on information included in the message (e.g., text, email, voicemail, etc.).

[0343] (M3) In some embodiments of the method described in M2, the method further includes facilitating route guidance to the second physical location in response to receiving an instruction from the user to use the second physical location as a new destination.

[0344] (M4) In some embodiments of the method according to M2 or M3, detecting that the message includes information identifying the second physical location includes performing the detection while a virtual assistant available on the electronic device is reading the message to the user via an audio system in communication with the electronic device. In this way, when the user is listening to the message read by the audio system (e.g., via a personal assistant available via the electronic device), the electronic device detects the information identifying the second physical location and uses the detected information to suggest using the second physical location as a new destination. Thus, the user does not need to divert his or her attention from the road while driving, and yet can dynamically adjust route guidance settings and destinations.

[0345] (M5) In some embodiments of the methods described in any one of M2 to M4, determining that the user has entered the vehicle includes detecting that the electronic device has established a communications link with the vehicle.

[0346] (M6) In some embodiments of the method of any one of M2 to M5, facilitating route guidance includes providing route guidance via a display of the electronic device.

[0347] (M7) In some embodiments of the method of any one of M2 through M5, facilitating route guidance includes transmitting, to the vehicle, information identifying the first physical location.

[0348] (M8) In some embodiments of the methods described in any one of M2 to M7, facilitating route guidance includes providing route guidance through an audio system in communication with the electronic device (e.g., through a car speaker or through a speaker built into the device itself).

[0349] (M9) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs when executed by the one or more processors causing the electronic device to perform a method as described in any one of M1 to M8.

[0350] (M10) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of M1 to M8.

[0351] (M11) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of M1 to M8.

[0352] (M12) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of M1 to M8.

[0353] (M13) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of M1 to M8.

[0354] (M14) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5301, FIG. 53), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5303, FIG. 53), and a processing unit (e.g., processing unit 5305, FIG. 53). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. IE (e.g., computing devices A through D). For ease of explanation, FIG. 53 illustrates display unit 5301 and touch-sensitive surface unit 5303 as being integrated into electronic device 5300, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes an information obtaining unit (e.g., information obtaining unit 5307, FIG. 53), a vehicle entry determining unit (e.g., vehicle entry determining unit 5309, FIG. 53), a prompt providing unit (e.g., prompt providing unit 5311, FIG. 53), an instruction receiving unit (e.g., instruction receiving unit 5313, FIG. 53), a route guidance facilitating unit (e.g., route guidance facilitating unit 5315, FIG. 53), and a message detecting unit (e.g., message detecting unit 5317, FIG. 53). The processing unit (or one or more components thereof such as units 5307 to 5317) is configured to obtain (e.g., by the information obtaining unit 5307) information identifying a first physical location viewed by a user in a first application running on the electronic device. The processing unit is also configured to determine (e.g., by the vehicle entry determining unit 5309) that the user has entered a vehicle. In response to determining that the user has entered the vehicle, the processing unit is configured to provide (e.g., by the prompt providing unit 5311) a prompt to the user to use the first physical location as a route guidance destination. In response to providing the prompt, receive (e.g., by the command receiving unit 5313) an instruction from the user to use the first physical location as a route guidance destination.The processing unit is further configured to facilitate route guidance to the first physical location (eg, by route guidance facilitation unit 5307).

[0355] (M15) In some embodiments of the electronic device described in M14, the processing unit (or one or more components thereof, such as units 5307 to 5317) is further configured to perform a method according to any one of M2 to M8.

[0356] (N1) In some embodiments, a method is performed in an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured in accordance with any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes presenting content in a first application. The method also includes receiving a request from a user to open a second application, different from the first application, the second application including an input receiving field. The method includes presenting an input receiving field in the second application in response to receiving the request. The method includes providing a selectable user interface object for enabling a user to paste at least a portion of the content into the input receiving field before receiving any user input in the input receiving field. The method includes pasting the portion of the content into the input receiving field in response to detecting a selection of the selectable user interface object. In this manner, the user is provided with a proactive paste action in the second application-based content previously viewed with the first action (e.g., this allows the user to paste content into the second application without having to reopen the first application, perform an explicit copy action, reopen the second application, and then explicitly request to paste the copied content into the second application).

[0357] (N2) According to some embodiments of the method described in N1, the method includes identifying an input-receiving field as a field capable of accepting a portion of content prior to providing a selectable user interface object.

[0358] (N3) According to some embodiments of the method as described in N2, identifying the input-receiving field as a field capable of accepting the portion of content is performed in response to detecting a selection of the input-receiving field.

[0359] (N4) According to some embodiments of the method according to any one of N1 to N3, the portion of the content corresponds to an image.

[0360] (N5) According to some embodiments of the method according to any one of N1 to N3, the part of the content corresponds to text content.

[0361] (N6) According to some embodiments of the method according to any one of N1 to N3, the part of the content corresponds to text content and an image.

[0362] (N7) According to some embodiments of the method according to any one of N1 to N6, the first application is a web browsing application and the second application is a messaging application.

[0363] (N8) According to some embodiments of the method according to any one of N1 to N6, the first application is a photo browsing application and the second application is a message application.

[0364] (N9) According to some embodiments of the method according to any one of N1 to N8, the method includes receiving a request to copy at least a portion of the content before receiving the request to open the second application.

[0365] (N10) According to some embodiments of the methods according to any one of N1 to N9, the selectable user interface object is displayed with an indication that the piece of content was recently viewed in the first application, thus providing the user with a clear indication as to why the paste suggestion was made.

[0366] (N11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs, when executed by the one or more processors, causing the electronic device to perform a method as described in any one of N1 to N10.

[0367] (N12) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of N1 to N10.

[0368] (N13) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method according to any one of N1 to N10.

[0369] (N14) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of N1 to N10.

[0370] (N15) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus including means for performing a method according to any one of N1 to N10.

[0371] (N16) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5401, FIG. 54), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5403, FIG. 54), and a processing unit (e.g., processing unit 5405, FIG. 54). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 54 illustrates display unit 5401 and touch-sensitive surface unit 5403 as being integrated into electronic device 5400, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5407, FIG. 54), a request receiving unit (e.g., request receiving unit 5409, FIG. 54), a user interface object providing unit (e.g., user interface object providing unit 5411, FIG. 54), a proactive pasting unit (e.g., proactive pasting unit 5413, FIG. 54), and a performance determination unit (e.g., performance determination unit 5415, FIG. 54).The processing unit (or one or more components thereof, such as units 5407 to 5415) is configured to present content in a first application (e.g., by the presentation unit 5407 and / or the display unit 5401), receive a request from a user to open a second application different from the first application (e.g., by the request receiving unit and / or the touch-sensitive surface unit 5403), the second application including an input receiving field, and in response to receiving the request, present the input receiving field in the second application (e.g., by the presentation unit 5407 and / or the display unit 5401), provide a selectable user interface object for enabling a user to paste at least a portion of the content into the input receiving field before receiving any user input in the input receiving field (e.g., by the user interface object providing unit 5411 and / or the display unit 5401), and in response to detecting selection of the selectable user interface object, paste a portion of the content into the input receiving field (e.g., by the proactive pasting unit 5413).

[0372] (N17) In some embodiments of the electronic device described in N16, the processing unit (or one or more components thereof, such as units 5407 to 5415) is further configured to perform a method according to any one of N1 to N10.

[0373] (O1) In some embodiments, a method is performed on an electronic device (e.g., portable multifunction device 100 ( FIG. 1A ) configured according to any one of computing devices A-D ( FIG. 1E )) having a touch-sensitive surface and a display (in some embodiments, touch screen 112, the touch-sensitive surface and the display are integrated as shown in FIG. 1C ). The method includes presenting textual content associated with an application on the display. The method also includes determining that a portion of the textual content is related to (i) a location, (ii) a contact, or (iii) an event. If the method determines that the portion of the textual content is related to a location, the method includes obtaining location information from a location sensor on the electronic device and preparing the obtained location information for display as a predicted content item. If the method determines that the portion of the textual content is related to a contact, the method includes performing a search on the electronic device for contact information associated with the portion of the textual content and preparing information associated with the at least one contact retrieved by the search for display as a predicted content item. Upon determining that the portion of the textual content is related to an event, the method includes performing a new search on the electronic device for event information related to the portion of the textual content, and preparing information based at least in part on the at least one event retrieved from the new search for display as a predicted content item. The method further includes displaying an affordance within the application that includes the predicted content item, detecting a selection of the affordance via the touch-sensitive surface, and, in response to detecting the selection, displaying information associated with the predicted content item on the display adjacent to the textual content. In this manner, a user is conveniently provided with predicted content items that can be used to complete a sentence (or, for example, answer a question posed by another user in a messaging application) without the user having to type anything or search information available on the electronic device to find the desired information.For example, the electronic device may provide telephone numbers, current location, availability for scheduling new events, and details related to current events, all without any explicit request or extra effort by the user, thus saving time while still allowing the user to be efficiently provided with desired information.

[0374] (O2) According to some embodiments of the method as described in O1, the portion of the textual content corresponds to textual content most recently presented within the application.

[0375] (O3) According to some embodiments of the method described in O1 or O2, the application is a message application and part of the text content is a question received within the message application from a remote user of a remote device different from the electronic device.

[0376] (O4) According to some embodiments of the method according to O1 or O2, the part of the textual content is an input provided by a user of the electronic device in an input receiving field in the application.

[0377] (O5) According to some embodiments of the method described in O1, the portion of the textual content is identified in response to a user input selecting a user interface object that includes the portion of the textual content.

[0378] (O6) According to some embodiments of the method described in O5, the application is a messaging application and the user interface object is a message bubble in a conversation displayed within the messaging application.

[0379] (O7) According to some embodiments of the method according to O5 or O6, the method further includes detecting a selection of the second user interface object, and in response to detecting the selection, (i) ceasing to display the affordance with the predicted content item and determining that the textual content associated with the second user interface object is related to a location, a contact, or an event, and displaying the new predicted content item within the application according to the determination. In this manner, the user can easily return to a message conversation and select a previously received message, and still be provided with the appropriate predicted content item.

[0380] (O8) According to some embodiments of the methods according to any one of O1 to O7, the affordance is displayed within an input receiving field adjacent to a virtual keyboard within the application.

[0381] (O9) According to some embodiments of the method described in O8, the input receiving field is a field that displays typing input received on a virtual keyboard.

[0382] (O10) According to some embodiments of the methods described in any one of O1 to O9, determining includes parsing the text content as it is received by the application to detect stored patterns known to be associated with contacts, events and / or locations.

[0383] (O11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory that stores one or more programs, the one or more programs when executed by the one or more processors causing the electronic device to perform a method as described in any one of O1 to O10.

[0384] (O12) In yet another aspect, an electronic device is provided that includes a touch-sensitive surface, a display, and means for performing a method according to any one of O1 to O10.

[0385] (O13) In yet another aspect, a non-transitory computer-readable storage medium is provided that stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform a method as described in any one of O1 to O10.

[0386] (O14) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface comprises a user interface displayed according to a method according to any one of O1 to O10.

[0387] (O15) In a further aspect, there is provided an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing a method according to any one of O1 to O10.

[0388] (O16) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5501, FIG. 55), a touch-sensitive surface unit (e.g., touch-sensitive surface unit 5503, FIG. 55), and a processing unit (e.g., processing unit 5505, FIG. 55). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 55 illustrates display unit 5501 and touch-sensitive surface unit 5503 as being integrated into electronic device 5500, although in some embodiments one or both of these units communicate with the electronic device even though the units remain physically separate from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5507, FIG. 55), a determination unit (e.g., determination unit 5509, FIG. 55), an acquisition unit (e.g., acquisition unit 5511, FIG. 55), a search implementation unit (e.g., search implementation unit 5513, FIG. 55), an information preparation unit (e.g., information preparation unit 5515, FIG. 55), an affordance display unit (e.g., affordance display unit 5517, FIG. 55), and a detection unit (e.g., detection unit 5519, FIG. 55).The processing unit (or one or more components thereof, such as units 5507 to 5519) presents textual content associated with the application on a display (e.g., by the presentation unit 5507 and / or the display unit 5501); determines (e.g., by the determining unit 5509) that a portion of the textual content is related to (i) a location, (ii) a contact, or (iii) an event; if it determines that the portion of the textual content is related to a location, obtains location information from a location sensor on the electronic device (e.g., by the obtaining unit 5511) and prepares the obtained location information for display as a predicted content item (e.g., by the information preparing unit 5515); if it determines that the portion of the textual content is related to a contact, performs a search on the electronic device for contact information related to the portion of the textual content (e.g., by the search performing unit 5513) and prepares information associated with the at least one contact retrieved from the search for display as a predicted content item. the electronic device (e.g., by the information preparing unit 5515), upon determining that a portion of the textual content is related to an event, performing a new search on the electronic device for event information related to the portion of the textual content (e.g., by the search performing unit 5513), preparing (e.g., by the information preparing unit 5515) information based at least in part on the at least one event retrieved by the new search for display as a predicted content item, displaying (e.g., by the affordance display unit 5517 and / or the display unit 5501) an affordance including the predicted content item within the application, detecting (e.g., by the detection unit 5519) a selection of the affordance via the touch-sensitive surface, and in response to detecting the selection, displaying (e.g., by the presentation unit 5507 and / or the display unit 5501) information associated with the predicted content item on the display adjacent to the textual content.

[0389] (O17) In some embodiments of the electronic device described in O16, the processing unit (or one or more components thereof, such as units 5507 to 5519) is further configured to perform a method according to any one of O1 to O10.

[0390] As described above (and in more detail below), one aspect of the present technology is the collection and use of data available from various sources (e.g., based on speech provided during voice communications) to improve the delivery to users of content that may be of interest to them. The present disclosure contemplates that in some examples, this collected data may include personal information data that uniquely identifies a particular person or that may be used to contact or locate a particular person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, home addresses, or any other identifiable information.

[0391] This disclosure recognizes that the use of such personal information data may be used to the benefit of the user in the present technology. For example, personal information data may be used to deliver targeted content that is more appealing to the user. Thus, use of such personal information data allows for intentional control of the content that is delivered. Additionally, other uses of personal information data that benefit the user are contemplated by this disclosure.

[0392] The present disclosure further contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other uses of such personal information data will comply with well-defined privacy policies and / or privacy practices. In particular, such entities should implement and always use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for keeping personal information data strictly confidential. For example, personal information from users should be collected for the entity's lawful and legitimate uses and should not be shared or sold except for those lawful uses. Furthermore, such collection should only be performed after informing and obtaining consent from the user. Furthermore, such entities may take all necessary measures to protect and secure access to such personal information data and ensure that others having access to that personal information data comply with their privacy policies and procedures. Furthermore, such entities may subject themselves to third-party assessments to attest to their compliance with widely accepted privacy policies and practices.

[0393] Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which a user selectively blocks use of or access to personal information data. That is, the present disclosure contemplates that hardware and / or software elements may be provided to prevent or block access to such personal information data. For example, when monitoring voice communications or actions performed by a user within an application, the present technology may be configured to allow a user to select "opt-in" or "opt-out" of participating in the collection of personal information data during registration for the service. In another example, a user may choose not to provide location information to a given content distribution service. In yet another example, a user may choose not to provide precise location information but allow transfer of location zone information.

[0394] Thus, while this disclosure broadly encompasses the use of personal information data to implement one or more of the various disclosed embodiments, this disclosure also contemplates that the various embodiments can also be implemented without requiring access to such personal information data. That is, various embodiments of the technology are not rendered inoperable by the absence of all or a portion of such personal information data. For example, content can be selected and delivered to a user by inferring preferences based on non-personal information or minimal personal information, such as content requested by a device associated with the user, other non-personal information available to the content delivery service, or publicly available information.

[0395] It should be noted that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described herein are not exhaustive, and many additional features and advantages will be apparent to those skilled in the art, particularly in view of the drawings, the specification, and the claims. Furthermore, it should be noted that the language used in this specification has been selected solely for ease of reading and explanation, and not to define or limit the subject matter of the present invention. [Brief description of the drawings]

[0396] For a better understanding of the various described embodiments, please refer to the Detailed Description section below in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout the drawings, in which:

[0397] [Figure 1A] 1 is a high-level block diagram of a computing device with a touch-sensitive display in accordance with some embodiments.

[0398] [Figure 1B] 1 is a block diagram of example components for event processing according to some embodiments.

[0399] [Figure 1C] 1 is an overview of a portable multifunction device having a touch-sensitive display in accordance with some embodiments.

[0400] [Figure 1D] 1 is a schematic used to illustrate a computing device with a touch-sensitive surface that is separate from a display, in accordance with some embodiments. [Figure 1E]

[0401] [Diagram 2] 1 is an overview of a touch screen used to present a user interface for a menu of applications according to some embodiments.

[0402] [Figure 3A] FIG. 2 is a block diagram illustrating a data structure for storing application usage data according to some embodiments. [Figure 3B] FIG. 2 is a block diagram illustrating a data structure for storing application usage data according to some embodiments.

[0403] [Figure 4A] FIG. 1 is a block diagram illustrating a data structure for storing trigger conditions according to some embodiments. [Figure 4B] FIG. 1 is a block diagram illustrating a data structure for storing trigger conditions according to some embodiments.

[0404] [Diagram 5] 1 is a block diagram illustrating an example of a system for establishing a trigger condition, according to some embodiments.

[0405] [Figure 6A] 1 is a flow chart illustrating a method for proactively identifying and surfacing (eg, surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments. [Figure 6B]1 is a flow chart illustrating a method for proactively identifying and surfacing (eg, surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments.

[0406] [Figure 7A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 7B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments.

[0407] [Figure 8A] 1 is a flow chart illustrating a method for proactively identifying and surfacing (eg, surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments. [Figure 8B] 1 is a flow chart illustrating a method for proactively identifying and surfacing (eg, surfacing for user selection) relevant content on an electronic device with a touch-sensitive display according to some embodiments.

[0408] [Figure 9A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 9B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 9C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments. [Figure 9D]1 is an overview of a touch-sensitive display used to present a user interface for proactively identifying and surfacing related content according to some embodiments.

[0409] [Figure 10A] 1 is a flow chart illustrating a method for proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display according to some embodiments. [Figure 10B] 1 is a flow chart illustrating a method for proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display according to some embodiments. [Figure 10C] 1 is a flow chart illustrating a method for proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display according to some embodiments.

[0410] [Figure 11A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11E] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11F] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11G] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11H] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11I] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments. [Figure 11J] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting search queries based on content currently displayed on the touch-sensitive display, according to some embodiments.

[0411] [Figure 12] 1 is a flow chart illustrating a method for entering a search mode based on heuristics according to some embodiments.

[0412] [Figure 13A]1 is an overview of a touch-sensitive display used to present a user interface for entering a search mode based on heuristics according to some embodiments. [Figure 13B] 1 is an overview of a touch-sensitive display used to present a user interface for entering a search mode based on heuristics according to some embodiments.

[0413] [Figure 14] 1 is a flow chart illustrating a method for proactively providing a vehicle location on an electronic device with a touch-sensitive display according to some embodiments.

[0414] [Figure 15A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing vehicle position according to some embodiments. [Figure 15B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing vehicle position according to some embodiments.

[0415] [Figure 16A] 1 is a flow chart illustrating a method for proactively providing nearby point of interest (POI) information in response to a search query according to some embodiments. [Figure 16B] 1 is a flow chart illustrating a method for proactively providing nearby point of interest (POI) information in response to a search query according to some embodiments.

[0416] [Figure 17A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby point of interest (POI) information in response to a search query, according to some embodiments. [Figure 17B]1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby point of interest (POI) information in response to a search query, according to some embodiments. [Figure 17C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby point of interest (POI) information in response to a search query, according to some embodiments. [Figure 17D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby point of interest (POI) information in response to a search query, according to some embodiments. [Figure 17E] 1 is an overview of a touch-sensitive display used to present a user interface for proactively providing nearby point of interest (POI) information in response to a search query, according to some embodiments.

[0417] [Figure 18A] 1 is a flow chart illustrating a method for extracting a content item from an audio communication and interacting with the extracted content item according to some embodiments. [Figure 18B] 1 is a flow chart illustrating a method for extracting a content item from an audio communication and interacting with the extracted content item according to some embodiments.

[0418] [Figure 19A] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications according to some embodiments. [Figure 19B] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications according to some embodiments. [Figure 19C]1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications according to some embodiments. [Figure 19D] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications according to some embodiments. [Figure 19E] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications according to some embodiments. [Figure 19F] 1 is an overview of a touch-sensitive display used to present a user interface for displaying and interacting with content items extracted from audio communications according to some embodiments.

[0419] [Figure 20] 1 is a flow chart illustrating a method of determining that an audio communication includes speech that identifies a physical location and populating an application with information about the physical location, according to some embodiments.

[0420] [Figure 21A] 1 is an overview of a touch-sensitive display used to present a user interface for determining that a voice communication includes speech identifying a physical location and populating an application with information about that physical location, according to some embodiments. [Figure 21B] 1 is an overview of a touch-sensitive display used to present a user interface for determining that a voice communication includes speech identifying a physical location and populating an application with information about that physical location, according to some embodiments.

[0421] [Figure 22A]1 is a flow chart illustrating a method for proactively suggesting physical locations for use in a messaging application according to some embodiments. [Figure 22B] 1 is a flow chart illustrating a method for proactively suggesting physical locations for use in a messaging application according to some embodiments.

[0422] [Figure 22C] 1 is a flow chart illustrating a method for proactively suggesting information about locations, events, or contacts according to some embodiments.

[0423] [Figure 23A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23E]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23F] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23G] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23H] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23I] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23J] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23K]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23L] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23M] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23N] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. [Figure 23O] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments.

[0424] [Figure 24A] 1 is a flow chart illustrating a method for proactively populating an application with information previously viewed by a user in a different application according to some embodiments. [Figure 24B] 1 is a flow chart illustrating a method for proactively populating an application with information previously viewed by a user in a different application according to some embodiments.

[0425] [Figure 25A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25D] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25E] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25F] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25G] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Fig. 25H] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25I] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. [Figure 25J] 1 is an overview of a touch-sensitive display used to present a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments.

[0426] [Figure 26A] 1 is a flow chart illustrating a method of proactively suggesting information previously viewed by a user in a first application for use in a second application according to some embodiments. [Figure 26B] 1 is a flow chart illustrating a method of proactively suggesting information previously viewed by a user in a first application for use in a second application according to some embodiments.

[0427] [Figure 27] 1 is a flow chart illustrating a method for proactively suggesting physical locations for use as destinations for vehicular routing according to some embodiments.

[0428] [Figure 28] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting physical locations for use as destinations for vehicular route guidance, according to some embodiments.

[0429] [Figure 29] 1 is a flow chart illustrating a method for proactively suggesting paste actions according to some embodiments.

[0430] [Figure 30A] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments. [Figure 30B] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments. [Figure 30C] 1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments. [Figure 30D]1 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting paste actions according to some embodiments.

[0431] [Figure 31-1] 1 illustrates a mobile device configured to perform dynamic tuning of the mobile device, according to some embodiments.

[0432] [Figure 31-2] 1 illustrates an example process for invoking a heuristic process according to some embodiments.

[0433] [Figure 31-3] 1 illustrates a process for adjusting settings on a mobile device using a heuristic process according to some embodiments.

[0434] [Figure 31-4] 1 illustrates an example system for performing background fetch updates for an application according to some embodiments.

[0435] [Figure 31-5] 1 illustrates peer prediction for determining the likelihood of a user invocation for an application on a mobile device 100 according to some embodiments.

[0436] [Figure 31-6] FIG. 2 is a flow diagram of an example process for predictively launching applications and performing background updates according to some embodiments.

[0437] [Figure 31-7] FIG. 1 is a flow diagram of an example process for determining when to launch an application on a mobile device according to some embodiments.

[0438] [Figure 31-8]FIG. 11 is a flow diagram illustrating state transitions for populating a trend table according to some embodiments.

[0439] [Figure 31-9] 1 is a block diagram illustrating a system for providing push notifications to a mobile device according to some embodiments.

[0440] [Figure 31-10] FIG. 13 is a flow diagram of an example process for performing a non-waking push at a push notification server according to some embodiments.

[0441] [Figure 31-11] FIG. 1 is a flow diagram of an example process for performing a background update of an application in response to a low priority push notification, according to some embodiments.

[0442] [Figure 31-12] FIG. 1 is a flow diagram of an example process for performing a background update of an application in response to a high priority push notification, according to some embodiments.

[0443] [Figure 31-13] FIG. 1 is a block diagram of an example system for performing background downloading and / or uploading of data on a mobile device according to some embodiments.

[0444] [Figure 31-14] FIG. 11 is a flow diagram of an example process for performing background downloads and uploads according to some embodiments.

[0445] [Figure 31-15] 1 illustrates an example graphical user interface (GUI) for enabling and / or disabling background updates for applications on a mobile device according to some embodiments.

[0446] [Figure 31-16] 1 illustrates an example system for sharing data between peer devices according to some embodiments.

[0447] [Figure 31-17] 1 illustrates an example process for sharing data between peer devices according to some embodiments.

[0448] [Figure 32-1] FIG. 1 is a block diagram of an embodiment of a system for returning search results based on an entered query prefix, according to some embodiments.

[0449] [Figure 32-2] 1 is a flow diagram of an embodiment of a process for determining query completion and related results based on an entered query prefix according to some embodiments.

[0450] [Figure 32-3] FIG. 2 is a block diagram of an embodiment of an aggregator and multiple search domains according to some embodiments.

[0451] [Figure 32-4] FIG. 2 is a diagram of an embodiment for a query completion search domain according to some embodiments.

[0452] [Figure 32-5] FIG. 2 is a diagram of one embodiment of a map search domain. [Figure 32-6] 1 is a flow diagram of an embodiment of a process for determining query completions from multiple search domains according to some embodiments.

[0453] [Figure 32-7] 1 is a flowchart of an embodiment of a process for determining relevant results across multiple search domains from determined query completions according to some embodiments.

[0454] [Figure 32-8]FIG. 2 is a block diagram of an embodiment of a system for incorporating user feedback into a feedback search index according to some embodiments.

[0455] [Figure 32-9] 1 is a flowchart of an embodiment of a process for incorporating user feedback into a citation search index according to some embodiments.

[0456] [Figure 32-10] 1 is a flow diagram of an embodiment of a process for collecting user feedback during a user search session according to some embodiments.

[0457] [Figure 32-11] 11 is a flow diagram of an embodiment of a process for incorporating user feedback into a feedback index according to some embodiments.

[0458] [Figure 32-12] 11 is a flow diagram of an embodiment of a process for updating a results cache using user feedback according to some embodiments.

[0459] [Figure 32-13] FIG. 2 is a block diagram of an embodiment of a federator for performing multi-domain searches using characterized query completion, according to some embodiments.

[0460] [Figure 32-14] 1 is a flowchart of an embodiment of a process for determining relevant results using a vocabulary service according to some embodiments.

[0461] [Figure 32-15] 1 is a flow diagram of an embodiment of a process for characterizing query completion according to some embodiments.

[0462] [Figure 32-16]FIG. 2 is a block diagram of an embodiment of a completion module that determines query completions from multiple search domains according to some embodiments.

[0463] [Figure 32-17] FIG. 2 is a block diagram of an embodiment of a results module that determines relevant results across multiple search domains from a determined query completion according to some embodiments.

[0464] [Figure 32-18] 1 is a block diagram of an embodiment of a feedback collection module for collecting user feedback during a user search session according to some embodiments.

[0465] [Figure 32-19] FIG. 13 is a block diagram of an embodiment of a feedback processing module that incorporates user feedback into a feedback index according to some embodiments.

[0466] [Figure 32-20] 4 is a block diagram of an embodiment of a query results update module that uses user feedback to update a results cache according to some embodiments.

[0467] [Figure 32-21] FIG. 13 is a block diagram of an embodiment of a feedback processing module that incorporates user feedback into a feedback index according to some embodiments.

[0468] [Figure 32-22] 4 is a block diagram of an embodiment of a query results update module that uses user feedback to update a results cache according to some embodiments.

[0469] [Figure 33-1] 1 illustrates, in block diagram form, a local search subsystem and a remote search subsystem on a computing device as known in the prior art.

[0470] [Figure 33-2] 1 illustrates, in block diagram form, a local search subsystem with local learning capabilities that may be used to improve results returned from a remote search application on a computing device, according to some embodiments.

[0471] [Figure 33-3] 1 illustrates, in block diagram form, a method for locally learning query features using local search queries, local results, and local feedback based on the local results, according to some embodiments.

[0472] [Figure 33-4] 1 illustrates, in block diagram form, a method for locally learning query features using search results returned from both local and remote search queries and local feedback on both local and remote search query results, according to some embodiments.

[0473] [Figure 33-5] 1 illustrates, in block diagram form, a method for locally learning query features sent by a remote service to a local device in response to a query passed to the remote service, according to some embodiments.

[0474] [Figure 33-6] 1 illustrates, in block diagram form, a method for receiving or determining, locally training, and using new features according to some embodiments.

[0475] [Figure 33-7] 1 illustrates an exemplary embodiment of a software stack that can be used in some embodiments of the present invention, according to some embodiments.

[0476] [Figure 34-5A] FIG. 1 illustrates a block diagram of an example data architecture for suggested contacts according to some embodiments.

[0477] [Figure 34-5B] FIG. 2 illustrates a block diagram of an example data architecture for suggested calendar events according to some embodiments.

[0478] [Figure 34-6A] 1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6A illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13. [Figure 34-6B] 1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6B illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13. [Figure 34-6C] 1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6C illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13. [Fig. 34-6D]1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6D illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13. [Figure 34-6E] 1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6E illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13. [Fig. 34-6F] 1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6F illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13. [Figure 34-6G]1A-1B, 2, and 3 provide an illustration of an example device for performing the techniques for suggesting contact and event information described in this section. FIG. 34-6G illustrates an example user interface for suggesting contact and event information. The user interfaces in these figures are also used to illustrate the processes described below, including the processes illustrated in FIG. 34-7A-FIG. 34-13.

[0479] [Figure 34-7A] 1 illustrates a flow diagram of an example process for generating suggested contacts according to some embodiments. [Figure 34-7B] 1 illustrates a flow diagram of an example process for generating suggested contacts according to some embodiments.

[0480] [Figure 34-8A] 1 illustrates a flow diagram of an exemplary process for updating an existing contact with suggested items of contact information according to some embodiments. [Fig. 34-8B] 1 illustrates a flow diagram of an exemplary process for updating an existing contact with suggested items of contact information according to some embodiments.

[0481] [Figure 34-9A] 1 illustrates a flow diagram of an exemplary process for displaying contacts with suggested contact information according to some embodiments. [Figure 34-9B] 1 illustrates a flow diagram of an exemplary process for displaying contacts with suggested contact information according to some embodiments.

[0482] [Figure 34-10] 1 illustrates a flow diagram of an exemplary process for displaying suggested contact information along with a message according to some embodiments.

[0483] [Figure 34-11A]1 illustrates a flow diagram of an exemplary process for generating suggested calendar events according to some embodiments. [Fig. 34-11B] 1 illustrates a flow diagram of an exemplary process for generating suggested calendar events according to some embodiments.

[0484] [Figure 34-12] 1 illustrates a flow diagram of an example process for displaying suggested event information along with a message according to some embodiments.

[0485] [Figure 34-13] 1 illustrates a flow diagram of an example process for displaying multiple suggested contact or event information along with a message according to some embodiments.

[0486] [Figure 34-14] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments.

[0487] [Figure 34-15] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments.

[0488] [Figure 35-1] 3 is a flowchart of a method 35_100 for suggesting applications based on a detected event according to some embodiments.

[0489] [Figure 35-2] 2 illustrates a segmentation process 35_200 according to some embodiments.

[0490] [Figure 35-3] 3 illustrates a decision tree 35_300 that may be generated by some embodiments.

[0491] [Figure 35-4]3 is a flowchart of a method 35_400 for suggesting applications to a user of a computing device based on an event according to some embodiments.

[0492] [Figure 35-5A] 1 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments. [Figure 35-5B] 1 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments. [Figure 35-5C] 1 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments. [Fig. 35-5D] 1 illustrates a plot of an example binomial distribution for various exact and inexact numerical values, according to some embodiments.

[0493] [Figure 35-6A] 1 illustrates a parent model and sub-models resulting from segmentation according to some embodiments; [Figure 35-6B] 1 illustrates a parent model and sub-models resulting from segmentation according to some embodiments;

[0494] [Figure 35-7] 3 illustrates an example architecture 35_700 for providing a user with a user interface for interacting with one or more applications according to some embodiments.

[0495] [Figure 36-1] 1 is a flow chart of a method for identifying an application based on a trigger event according to some embodiments.

[0496] [Figure 36-2] 1 illustrates a block diagram of a system for determining a trigger event according to some embodiments.

[0497] [Figure 36-3]1 illustrates a block diagram of a system for identifying applications for a user based on a trigger event according to some embodiments.

[0498] [Figure 36-4] FIG. 1 illustrates a block diagram of a system for identifying applications using multiple predictive models according to some embodiments.

[0499] [Figure 36-5] 1 is a flow chart of a method for identifying an application based on a trigger event on a device according to some embodiments.

[0500] [Figure 36-6] FIG. 2 is a simplified diagram of a device having a user interface for a music application according to some embodiments.

[0501] [Figure 36-7A] 1 is a flow chart of a method for removing an identified application from a user interface according to some embodiments. [Figure 36-7B] 1 is a flow chart of a method for removing an identified application from a user interface according to some embodiments.

[0502] [Figure 37-1] 1 is a flowchart of a method 100 for suggesting destinations based on a detected event according to some embodiments.

[0503] [Figure 37-2] 1 illustrates a block diagram of a system for determining a trigger event according to some embodiments.

[0504] [Figure 37-3] 1 illustrates a block diagram of a system for identifying a destination based on a trigger event according to some embodiments.

[0505] [Figure 37-4] 1 illustrates an example of suggested destinations in a user interface for an email application according to some embodiments.

[0506] [Figure 37-5] 1 illustrates an example of suggesting destinations in a user interface for a search application according to some embodiments.

[0507] [Figure 37-6] 1 is a flowchart of a method 37-600 for suggesting recipients to a user of a computing device based on an event, according to some embodiments.

[0508] [Figure 37-7] 1 illustrates an example data flow for suggesting a destination according to some embodiments.

[0509] [Figure 37-8] 1 illustrates a block diagram of an interaction module according to some embodiments.

[0510] [Figure 37-9] 3 illustrates an example architecture 37-900 for providing a user with a user interface for suggesting destinations according to some embodiments.

[0511] [Figure 38-1] FIG. 1 illustrates a block diagram of different components of a mobile computing device configured to implement various techniques described herein, according to some embodiments.

[0512] [Figure 38-2] 38-2 illustrates a method performed by the application prediction engine of FIG. 38-1 according to some embodiments.

[0513] [Figure 38-3] 2 illustrates a method performed by the search application of FIG. 1 according to some embodiments.

[0514] [Figure 38-4] 38-1 illustrates a conceptual diagram of an example user interface of the search application of FIG. 38-1 according to some embodiments.

[0515] [Figure 39-1] FIG. 1 illustrates a block diagram of different components of a mobile computing device configured to implement various techniques described herein, according to some embodiments.

[0516] [Figure 39-2] 39-1 (or FIG. 1A) according to some embodiments, illustrates a more detailed block diagram of certain components of the mobile computing device shown in FIG. 39-1 (or FIG. 1A).

[0517] [Figure 39-3A] 1 illustrates a method for high-level initialization and operation of a prediction engine according to some embodiments.

[0518] [Figure 39-3B] 1 illustrates a method for synchronously providing predictions in a prediction engine according to some embodiments.

[0519] [Figure 39-3C] 1 illustrates a method for asynchronously providing predictions in a prediction engine according to some embodiments.

[0520] [Figure 39-4A] 1 illustrates a method for a consumer application to request synchronously receiving predictions according to some embodiments.

[0521] [Figure 39-4B] 1 illustrates a method for a consumer application to register to receive predictions asynchronously, according to some embodiments.

[0522] [Figure 39-5A]1 illustrates a method for managing prediction engine registrations at a prediction engine center according to some embodiments.

[0523] [Figure 39-5B] 1 illustrates a method for synchronously providing predictions to consumer applications at a prediction engine center according to some embodiments.

[0524] [Figure 39-5C] 1 illustrates a method for asynchronously providing predictions to consumer applications at a prediction engine center according to some embodiments.

[0525] [Figure 40-1] 1 is a block diagram of an example system for monitoring, predicting, and notifying a context client of changes in a current context of a computing device according to some embodiments.

[0526] [Figure 40-2A] 1 illustrates an example of context items that may constitute a current context according to some embodiments.

[0527] [Figure 40-2B] 13 illustrates an example of a new context item being added to a current context according to some embodiments.

[0528] [Figure 40-3] 1 illustrates an example callback predicate database according to some embodiments.

[0529] [Figure 40-4] 1 is a graph illustrating example state changes over time associated with a context item according to some embodiments;

[0530] [Figure 40-5] 1 is a graph illustrating an example event stream associated with a context item according to some embodiments;

[0531] [Figure 40-6] 1 illustrates an example historical event stream database according to some embodiments.

[0532] [Figure 40-7] 1 is a block diagram of an example system for providing context callback notifications to a requesting client according to some embodiments.

[0533] [Figure 40-8A] FIG. 2 is a block diagram of an example system illustrating reinitiating a terminated request client according to some embodiments.

[0534] [Figure 40-8B] FIG. 2 is a block diagram of an example system illustrating reinitiating a terminated request client according to some embodiments.

[0535] [Figure 40-9A] FIG. 1 is a block diagram of an example system illustrating restarting a terminated context daemon according to some embodiments.

[0536] [Figure 40-9B] FIG. 1 is a block diagram of an example system illustrating restarting a terminated context daemon according to some embodiments.

[0537] [Figure 40-10A] FIG. 2 is a block diagram of an example system illustrating restarting terminated context daemons and request clients according to some embodiments.

[0538] [Figure 40-10B] FIG. 2 is a block diagram of an example system illustrating restarting terminated context daemons and request clients according to some embodiments.

[0539] [Figure 40-11]FIG. 1 is a block diagram of an example system configured to restart a client and / or a context daemon based on device state information received by an activation daemon, according to some embodiments.

[0540] [Figure 40-12A] FIG. 2 is a block diagram of an example system illustrating restarting a context daemon using a launch daemon, according to some embodiments.

[0541] [Figure 40-12B] FIG. 2 is a block diagram of an example system illustrating restarting a context daemon using a launch daemon, according to some embodiments.

[0542] [Figure 40-13A] FIG. 2 is a block diagram of an example system illustrating restarting a requesting client using an activation daemon, according to some embodiments.

[0543] [Figure 40-13B] FIG. 2 is a block diagram of an example system illustrating resuming a requesting client using an activation daemon, according to some embodiments.

[0544] [Figure 40-14] 1 is a graph illustrating an example of slot-wise averaging to predict future events, according to some embodiments;

[0545] [Figure 40-15] 1 illustrates an example graph showing slot weighting according to some embodiments.

[0546] [Figure 40-16A] 1 is a graph illustrating an example method for predicting future context according to some embodiments;

[0547] [Figure 40-16B]1 is a graph illustrating an example method for converting slot-wise probabilities into a probability curve according to some embodiments.

[0548] [Figure 40-17] 1 illustrates an example event stream including predicted future events according to some embodiments.

[0549] [Figure 40-18] FIG. 1 is a flow diagram of an example process for notifying a client of a context change on a computing device according to some embodiments.

[0550] [Figure 40-19] FIG. 11 is a flow diagram of an example process for restarting a context daemon to service a callback request according to some embodiments.

[0551] [Figure 40-20] FIG. 11 is a flow diagram of an example process for restarting a callback client to receive a callback notification according to some embodiments.

[0552] [Figure 40-21] FIG. 1 is a flow diagram of an example process for predicting future events based on past context information according to some embodiments.

[0553] [Figure 40-22] FIG. 11 is a flow diagram of an example process for servicing a sleep context callback request according to some embodiments.

[0554] [Figure 41-1] FIG. 1 is a block diagram of one embodiment of a system for indexing application state for use in a local device search index.

[0555] [Figure 41-2]FIG. 1 is a block diagram of one embodiment of a system for searching application state using an on-device application state search index.

[0556] [Figure 41-3] 1 is a block diagram of an embodiment of a user interface displaying application state query results among other query results.

[0557] [Figure 41-4A] FIG. 1 is a flow diagram of one embodiment of a process for indexing application state received from multiple different applications on a device.

[0558] [Figure 41-4B] FIG. 2 is a flow diagram of one embodiment of a process for determining query results for a query using an application state index.

[0559] [Figure 41-5] FIG. 1 is a flow diagram of one embodiment of a process for receiving and presenting application state as part of a query result.

[0560] [Figure 41-6] FIG. 1 is a block diagram of one embodiment of a system for indexing application state for use in a remote search index.

[0561] [Figure 41-7] FIG. 1 is a block diagram of one embodiment of a system for searching application state using a remote application state search index.

[0562] [Figure 41-8] FIG. 13 is a flow diagram of one embodiment of a process for adding application state to an application state index.

[0563] [Figure 41-9]FIG. 11 is a flow diagram of one embodiment of a process for exporting application state to an application state indexing service.

[0564] [Figure 41-10] 1 is a flow diagram of one embodiment of a process for performing a query search using an application state index.

[0565] [Figure 41-11] FIG. 1 is a flow diagram of one embodiment of a process for receiving and presenting application state as part of a query result.

[0566] [Figure 41-12] FIG. 1 is a block diagram of one embodiment of a system for indexing application state views for use in a remote search index.

[0567] [Figure 41-13] FIG. 2 is a block diagram of one embodiment of an application view.

[0568] [Figure 41-14] 1 is a flow diagram of one embodiment of a process for generating an application state view using application state.

[0569] [Figure 41-15] 1 is a flow diagram of one embodiment of a process for receiving and presenting application state, including an application state view, as part of a query result.

[0570] [Diagram 42] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Diagram 43] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Diagram 44] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Diagram 45]FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 46] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 47] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 48] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 49] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 50] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 51] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 52] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Diagram 53] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 54] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. [Figure 55] FIG. 2 is a functional block diagram of an electronic device in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0571] As discussed above and in more detail below, there is a growing demand for electronic devices with faster and more efficient methods and interfaces for quickly accessing applications and desired functions within those applications. In particular, there is a growing demand for devices that assist users to provide proactive assistance by eliminating repetitive tasks and identifying and providing relevant information before the user explicitly requests it. Furthermore, there is a growing demand for quickly accessing applications and desired functions within those applications at specific time periods (e.g., accessing a calendar application after waking up each morning) and at specific locations (e.g., accessing a music application at the gym). Disclosed herein are new methods and interfaces that address those demands and provide users with a way to quickly access applications and functions within those applications at those specific locations and time periods. Such methods and interfaces optionally complement or replace existing methods for accessing applications. Such methods and interfaces reduce the cognitive burden on users and create a more efficient human-machine interface. In the case of battery-operated devices, such methods and interfaces conserve power and extend the time between charging the battery. Moreover, such methods and interfaces help extend the life of a touch-sensitive display by reducing the amount of input required (e.g., instead of continually and mindlessly tapping a touch-sensitive display to identify desired information, the methods and interfaces disclosed herein proactively provide information without requiring user input).

[0572] In the following, Figures 1A-1E and 2 provide a description of an example device. Figures 10 and 11 provide functional block diagrams of an example electronic device. Figures 3A-3B and 4A-4B are block diagrams of example data structures used to proactively identify and surface related content (these data structures are used in the methods described in connection with Figures 6A-6B and in the methods described with reference to Figures 8A-8B). Figure 5 is a block diagram illustrating an example system for establishing trigger conditions used to proactively identify and surface related content (these example systems are used in the methods described in connection with Figures 6A-6B and in the methods described with reference to Figures 8A-8B). Figures 6A-6B are a flow chart illustrating a method for proactively identifying and surfacing related content. Figures 7A-7B are schematics of a touch-sensitive display used to illustrate example user interfaces and gestures for proactively identifying and surfacing related content. Figures 8A-8B are a flow chart illustrating a method for proactively identifying and surfacing related content. Figures 9A-9D are schematics of touch-sensitive displays used to illustrate additional user interfaces for proactively identifying and surfacing related content. Figures 3A-3B, 4A-4B, 5, and 7A-7B are used to illustrate the method and / or process of Figures 6A-6B. Figures 3A-3B, 4A-4B, 5, and 9A-9D are used to illustrate the method and / or process of Figures 8A-8B.

[0573] Figures 10A-10C are flow charts illustrating a method of proactively suggesting search queries based on currently displayed content on an electronic device with a touch-sensitive display. Figures 11A-11J are schematics of a touch-sensitive display used to illustrate a user interface for proactively suggesting search queries based on currently displayed content on the touch-sensitive display. Figures 11A-11J are used to illustrate the methods and / or processes of Figures 10A-10C.

[0574] Figure 12 is a flow chart illustrating a method of transitioning to a search mode based on heuristics. Figures 13A-13B are schematics of a touch-sensitive display used to illustrate a user interface for transitioning to a search mode based on heuristics. Figures 13A-13B are used to illustrate the method and / or process of Figure 12.

[0575] Figure 14 is a flow chart illustrating a method for proactively providing a vehicle location on an electronic device with a touch-sensitive display, according to some embodiments. Figures 15A-15B are schematics of a touch-sensitive display used to present a user interface for proactively providing a vehicle location, according to some embodiments. Figures 15A-15B are used to illustrate the method and / or process of Figure 14.

[0576] 16A-16B are flow charts illustrating a method for proactively providing nearby points of interest (POI) information for a search query, according to some embodiments. FIGs. 17A-17E are schematics of a touch-sensitive display used to present a user interface for proactively providing nearby points of interest (POI) information for a search query, according to some embodiments. FIGs. 16A-16B are used to illustrate the methods and / or processes of FIGs. 17A-17E.

[0577] Figures 18A-18B are flow charts illustrating a method of extracting a content item from an audio communication and interacting with the extracted content item, according to some embodiments. Figures 19A-19F are schematics of a touch-sensitive display used to illustrate a user interface for displaying and interacting with content items extracted from an audio communication, according to some embodiments. Figures 19A-19F are used to illustrate the methods and / or processes of Figures 18A-18B.

[0578] Figure 20 is a flow chart illustrating a method for determining that an audio communication includes speech identifying a physical location and populating an application with information about the physical location, according to some embodiments. Figures 21A-21B are schematics of a touch-sensitive display used to show a user interface for determining that an audio communication includes speech identifying a physical location and populating an application with information about the physical location, according to some embodiments. Figures 19A-19F and 21A-21B are used to illustrate the method and / or process of Figure 20.

[0579] Figures 22A-22B are flow charts illustrating a method for proactively suggesting physical locations for use in a messaging application, according to some embodiments. Figures 23A-23O are schematics of a touch-sensitive display used to present a user interface for proactively suggesting information about locations, events, or contacts (e.g., for easy selection by a user and inclusion within a messaging application), according to some embodiments. Figures 23A-23O are used to illustrate the methods and / or processes of Figures 22A-22B.

[0580] Figure 22C is a flow chart illustrating a method for proactively suggesting information related to locations, events, or contacts according to some embodiments. Figures 23A-23O are used to illustrate the method and / or process of Figure 22C.

[0581] Figures 24A-24B are flow charts illustrating a method for proactively populating an application with information previously viewed by a user in a different application, according to some embodiments. Figures 25A-25J are schematics of a touch-sensitive display used to show a user interface for proactively populating an application with information previously viewed by a user in a different application (e.g., populating a ride-sharing application with information about locations viewed by a user in a reviews application), according to some embodiments. Figures 25A-25J are used to illustrate the methods and / or processes of Figures 24A-24B.

[0582] Figures 26A-26B are flow charts illustrating a method for proactively suggesting information previously viewed by a user in a first application for use in a second application, according to some embodiments. Figures 25A-25J are used to illustrate the method and / or process of Figures 26A-26B.

[0583] Fig. 27 is a flow chart illustrating a method for proactively suggesting physical locations for use as destinations for vehicular route guidance according to some embodiments. Fig. 28 is an overview of a touch-sensitive display used to present a user interface for proactively suggesting physical locations for use as destinations for vehicular route guidance according to some embodiments. Fig. 28 is used to illustrate the method and / or process of Fig. 27.

[0584] 29 is a flow chart illustrating a method for proactively suggesting paste actions according to some embodiments. FIGs. 30A-30D are schematics of a touch-sensitive display used to show a user interface for proactively suggesting paste actions according to some embodiments. FIGs. 30A-30D are used to show the method and / or process of FIG.

[0585] Sections 1-11 in the "Additional Description of the Invention" section provide additional details that complement the description provided in connection with Figures 1A-30D.

[0586] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments being described. However, it will be apparent to those skilled in the art that the various embodiments being described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0587] In this specification, terms such as first, second, etc. are used in some examples to describe various elements, but it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first contact can be referred to as a second contact, and similarly, a second contact can be referred to as a first contact, without departing from the scope of the various embodiments described. Although a first contact and a second contact are both contacts, they are not the same contact.

[0588] The terms used in the description of the various embodiments described herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used in the various described embodiments and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is also to be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0589] As used herein, the term "if" is optionally interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are optionally interpreted to mean "upon determining," "in response to determining," "upon detecting (the stated condition or event)," or "in response to detecting (the stated condition or event)," depending on the context.

[0590] The disclosure herein refers interchangeably to detection of a touch input on, at, across, on, or substantially within a particular user interface element or a particular portion of a touch-sensitive display. As used herein, a touch input detected "at" a particular user interface element may also be a touch input detected "on," "across," "on," or "substantially within" the same user interface element, depending on the context. In some embodiments, and as described in more detail below, the desired sensitivity level for detecting touch input is configured by a user of the electronic device (e.g., the user may determine (and configure and operate the electronic device) that touch input should be detected only if it is entirely within a user interface element).

[0591] Described herein are embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices. In some embodiments, the device is a portable communication device, such as a mobile telephone, that also includes other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, IPHONE®, IPOD TOUCH®, and IPAD® devices from APPLE Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch-sensitive displays and / or touchpads), are also optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch-sensitive displays and / or touchpads).

[0592] In the following description, an electronic device is described that includes a display and a touch-sensitive surface, however, it should be understood that the electronic device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0593] The device typically supports a variety of applications, such as one or more of drawing applications, presentation applications, word processing applications, website creation applications, disc authoring applications, spreadsheet applications, gaming applications, telephony applications, video conferencing applications, email applications, instant messaging applications, health / fitness applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0594] Various applications executing on the device optionally use at least one common physical user-interface device, such as a touch-sensitive surface. One or more features of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or changed for each application and / or within the corresponding application. In this way, the common physical architecture (such as the touch-sensitive surface) of the device optionally supports a variety of applications with user interfaces that are intuitive and transparent to the user.

[0595] Attention is now directed to an embodiment of a portable electronic device with a touch-sensitive display. FIG. 1A is a block diagram illustrating a portable multifunction device 100 (also referred to herein interchangeably as electronic device 100 or device 100) with a touch-sensitive display 112, according to some embodiments. Touch-sensitive display 112 may conveniently be referred to as a "touch screen" and may also be known or referred to as a touch-sensitive display system. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), controller 120, one or more processing units (CPUs) 122, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more light sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of a contact on device 100 (e.g., a touch-sensitive surface such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating a tactile output on device 100 (e.g., generating a tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or a touchpad of device 100). These components optionally communicate via one or more communication buses or signal lines 103.

[0596] As used herein and in the claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or a surrogate (proxy) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four distinct values, and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using a variety of methods and a variety of sensors, or combinations of sensors. For example, one or more force sensors disposed under or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some implementations, the force measurements of multiple force sensors are combined (e.g., a weighted average) to determine an estimate of the contact force. Similarly, a pressure-sensing tip of a stylus is optionally used to determine the pressure of a stylus on the touch-sensitive surface. Alternatively, the size and / or change in a contact area detected on the touch-sensitive surface, the capacitance and / or change in the capacitance of the touch-sensitive surface proximate the contact, and / or the resistance and / or change in the capacitance of the touch-sensitive surface proximate the contact, are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measure of the force or pressure of the contact is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measure). In some implementations, the surrogate measure of the force or pressure of the contact is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).

[0597] As used herein and in the claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of gravity of the device that is to be detected by a user with the user's sense of touch. For example, in a situation where a device or a component of the device is in contact with a touch-sensitive user's surface (e.g., the fingers, palm, or other part of the user's hand), the tactile output generated by the physical displacement is interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical property of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "down-click" or "up-click" of a physical actuator button. In some cases, a user will feel a tactile sensation such as a "down-click" or "up-click" even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's action. As another example, movement of a touch-sensitive surface is optionally interpreted or sensed by a user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. Although such user interpretation of touch depends on the user's personal sensory perception, there are many sensory perceptions of touch that are common to a majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an "up click," "down click," "roughness"), unless otherwise noted, the generated tactile output corresponds to a physical displacement of the device, or a component of the device, that produces the described sensory perception for a typical (or average) user.

[0598] It should be understood that device 100 is but one example of a portable multifunction device, and that device 100 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of components. The various components shown in FIG. 1A are implemented in hardware, software, or a combination of hardware and software, including one or more signal processing circuits, and / or application specific integrated circuits.

[0599] Memory 102 optionally includes high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, or other random access solid-state memory devices), and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory 102 optionally includes one or more storage devices located remotely from processor(s) 122. Access to memory 102 by other components of device 100, such as CPU 122 and peripherals interface 118, is optionally controlled by controller 120.

[0600] A peripherals interface 118 can be used to couple input and output peripherals of the device to the CPU 122 and memory 102. The one or more processors 122 operate or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions for the device 100 and to process data.

[0601] In some embodiments, peripherals interface 118, CPU 122, and controller 120 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0602] The RF (Radio Frequency) circuitry 108 transmits and receives RF signals, also referred to as electromagnetic signals. The RF circuitry 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuitry 108 optionally includes well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, etc. The RF circuitry 108 optionally communicates via wireless communication with the Internet, also referred to as the World Wide Web (WWW), an intranet, and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs), and other devices. The wireless communication optionally uses any of a number of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM)®, Enhanced Data GSM Environment (EDGE)®, High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPADA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth®, and / or Wireless Fidelity (Wi-Fi)® (e.g., IEEE 802.1 la, IEEE 802.1 lb, IEEE 802.1 lg, and / or IEEE 802.1 ln).

[0603] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripherals interface 118, converts the audio data to an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into sound waves audible to humans. The audio circuit 110 also receives an electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data, and transmits the audio data to the peripherals interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripherals interface 118. In some embodiments, the audio circuit 110 further comprises a headset jack. The headset jack provides an interface between audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset that has both an output (e.g., one- or binaural headphones) and an input (e.g., a microphone).

[0604] The I / O subsystem 106 connects input / output peripherals of the device 100, such as the touch screen 112 and other input control devices 116, to a peripheral interface 118. The I / O subsystem 106 optionally includes one or more input controllers 160 for a display controller 156, a light sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and other input or control devices. The one or more input controllers 160 receive electrical signals from and send electrical signals to the other input or control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) 160 are optionally coupled to any (or none) of a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more buttons optionally include up / down buttons for adjusting the volume of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button.

[0605] The touch-sensitive display 112 provides an input and output interface between the device and a user. The display controller 156 receives and / or sends electrical signals to the touch screen 112. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

[0606] Touchscreen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic and / or tactile contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction sets in memory 102) detects contacts (and any movement or breaking of contact) on touchscreen 112 and translates the detected contacts into interactions with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the point of contact between touchscreen 112 and the user corresponds to the area under the user's finger.

[0607] Touch screen 112 optionally uses LCD (liquid crystal display), LPD (light emitting polymer display) or LED (light emitting diode) or OLED (organic light emitting diode) technology, although in other embodiments other display technologies are used. Touch screen 112 and display controller 156 optionally detect contact and any movement or interruption of contact using any of a number of now known or later developed touch sensing technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch screen 112. In one example embodiment, projected mutual capacitance sensing technology is used, such as that found in the IPHONE®, IPOD TOUCH®, and IPAD® from APPLE Inc. of Cupertino, California.

[0608] Touchscreen 112 optionally has a video resolution of greater than 400 dpi. In some embodiments, touchscreen 112 has a video resolution of at least 600 dpi. In other embodiments, touchscreen 112 has a video resolution of at least 1000 dpi. A user optionally contacts touchscreen 112 using any suitable object, such as a stylus, finger, or the like. In some embodiments, the user interface is designed to function primarily with finger-based contacts and gestures. In some embodiments, the device translates finger-based input into precise pointer / cursor position or commands to perform the user's desired action.

[0609] In some embodiments, in addition to a touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating certain functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touchscreen 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0610] Device 100 also includes a power system 162 for providing power to the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power within a portable device.

[0611] Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor coupled to light sensor controller 158 in I / O subsystem 106. Light sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. Light sensor 164 receives light from the environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), light sensor 164 optionally captures still images or video. In some embodiments, the light sensor is located on the back of device 100, opposite touch screen 112 on the front of the device, to enable the touch-sensitive display as a viewfinder for still image and / or video capture. In some embodiments, another light sensor is located on the front of the device to optionally obtain an image of the user for video conferencing, while the user views other video conferencing participants on the touch-sensitive display.

[0612] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 in I / O subsystem 106. Contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor 165 receives contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is located on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite touch screen 112 located on the front of device 100.

[0613] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I / O subsystem 106. In some embodiments, when the multifunction device is placed near a user's ear (e.g., when the user is making a phone call), the proximity sensor is turned off and touch screen 112 is disabled.

[0614] Device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows a tactile output generator coupled to haptic feedback controller 161 in I / O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile output on the device). Contact intensity sensor 165 receives haptic feedback generation instructions from haptic feedback module 133 and generates a tactile output on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is disposed on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112) and generates a tactile output, optionally by moving the touch-sensitive surface vertically (e.g., in / out of the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is disposed on the back of device 100, opposite touch-sensitive display 112, which is disposed on the front of device 100.

[0615] Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows accelerometer 168 coupled to peripherals interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on the touch-sensitive display in portrait or landscape view based on analysis of data received from the one or more accelerometers. In addition to accelerometer(s) 168, device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the location and orientation (e.g., portrait or landscape) of device 100.

[0616] In some embodiments, the software components stored in the memory 102 include an operating system 126, a proactive module 163 (optionally including one or more of an application usage data table 335, a trigger condition table 402, a trigger establishment module 163-1, and / or a usage data collection module 163-2), a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Additionally, in some embodiments, as shown in FIG. 1A, the memory 102 stores a device / global internal state 157. The device / global internal state 157 includes one or more of the following: active application state, indicating which application, if any, is currently active; display state, indicating what applications, views, or other information are occupying various regions of touch-sensitive display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and position information regarding the device's position and / or attitude (e.g., the device's orientation).

[0617] Operating system 126 (e.g., an embedded operating system such as Darwin®, RTXC®, LINUX®, UNIX®, OS X®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0618] The communications module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) are adapted to couple to other devices directly or indirectly through a network (e.g., Internet, wireless LAN, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors the same as, similar to, and / or compatible with the 30-pin connectors used on some embodiments of IPOD devices from APPLE Inc. In other embodiments, the external ports are multi-pin (e.g., 8-pin) connectors the same as, similar to, and / or compatible with the 8-pin connectors used on LIGHTNING connectors from APPLE Inc.

[0619] Contact / motion module 130, optionally in conjunction with display controller 156, detects contact with touch screen 112 and other touch-sensing devices (e.g., a touchpad or physical click wheel). Contact / motion module 130 includes various software components for performing various operations related to detecting contact, such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger up event or an interruption of the contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by the series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions are, optionally, applied to a single contact (e.g., one finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, contact / motion module 130 and display controller 156 detect contacts on a touchpad.

[0620] In some embodiments, contact / motion module 130 uses a set of one or more intensity thresholds for determining whether an action has been performed by a user (e.g., for determining whether a user has selected or "clicked" on an affordance). In some embodiments, at least a subset of the intensity thresholds are determined according to software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of a particular physical actuator and may be adjusted without modifying the physical hardware of device 100). For example, the mouse "click" threshold of a trackpad or touch-sensitive display may be set to any of a wide range of pre-defined thresholds without modifying the hardware of the trackpad or touch-sensitive display. Additionally, in some implementations, a user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once via a system-level click "strength" parameter).

[0621] Contact / motion module 130 optionally detects gesture input by a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timing, and / or intensity of detected contact). Thus, a gesture is optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger down event (e.g., at the location of an icon) followed by detecting a finger lift off event at the same location (or substantially the same location) as the finger down event. As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger down event followed by one or more finger drag events followed by, in some embodiments, detecting a finger lift off event.

[0622] Graphics module 132 includes various known software components for rendering and displaying graphics on touchscreen 112 or other display, including components for modifying the visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

[0623] In some embodiments, graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. Graphics module 132 receives one or more codes specifying the graphics to be displayed, including, as appropriate, coordinate data and other graphic characteristic data, from an application or the like, and then generates screen image data for output to display controller 156.

[0624] The tactile feedback module 133 includes various software components for generating instructions used by the tactile output generator(s) 167 to generate tactile outputs at one or more locations on the device 100 in response to user interaction with the device 100.

[0625] Text input module 134 is optionally a component of graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts module 137, email client module 140, IM module 141, browser module 147, and any other application requiring text input).

[0626] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to a phone 138 application for use in location-based dialing, to a camera 143 application as image / video metadata, and to applications that provide location-based services such as a weather widget, a local yellow pages widget, and a maps / navigation widget).

[0627] Applications (“apps”) 136 optionally include the following modules (or sets of instructions), or a subset or superset thereof: ● a contacts module 137 (sometimes called an address book or contact list); ● Telephone module 138, ● Videoconferencing module 139, ● an email client module 140; ● Instant messaging (IM) module 141; ● Health module 142, A camera module 143 for still and / or video images, ● Image management module 144, ● Browser module 147, ● Calendar module 148, A widget module 149, optionally including one or more of a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widgets 149-6; ● Search module 151, ● a video and music player module 152, optionally consisting of a video player module and a music player module; ● Memo module 153, Map module 154, and / or ●Online video module 155.

[0628] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, website creation applications, disk authoring applications, spreadsheet applications, JAVA® enabled applications, encryption, digital rights management, voice recognition, a widget creator module 149-6 for creating user created widgets, and voice duplication.

[0629] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, contacts module 137 is optionally used to manage an address book or contact list (e.g., stored in memory 102 or in contacts module 137 in memory 370), including adding a name(s) to the address book, deleting a name(s) from the address book, associating phone number(s), email address(es), address(es), or other information with a name, associating a picture with a name, categorizing and sorting names, providing phone numbers or email addresses to initiate and / or facilitate communications via telephone module 138, videoconferencing module 139, email module 140, or IM module 141, and the like.

[0630] Telephone module 138, in conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, is optionally used to enter a series of characters corresponding to a telephone number, access one or more telephone numbers in address book 137, modify an entered telephone number, dial a corresponding telephone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As mentioned above, wireless communication optionally uses any of a number of communication standards, protocols, and technologies.

[0631] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, light sensor 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, videoconferencing module 139 contains executable instructions for initiating, conducting, and terminating a videoconference between a user and one or more other participants in accordance with the user's instructions.

[0632] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 contains executable instructions for composing, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to compose and send emails with still or video images captured by camera module 143.

[0633] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant message module 141 includes executable instructions for inputting character strings corresponding to an instant message, modifying previously input characters, sending the corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and displaying the received instant messages. In some embodiments, the sent and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0634] In conjunction with the RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and video and music player module 146, the health module 142 includes executable instructions for creating workouts (e.g., in terms of time, distance, and / or calorie burn goals), communicating with workout sensors (such as a watch or sports device such as a pedometer), receiving workout sensor data, calibrating sensors used to monitor workouts, selecting and playing music for workouts, and displaying, storing, and transmitting workout data.

[0635] Camera module 143, in conjunction with touch screen 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, and image management module 144, contains executable instructions to capture still images or videos (including video streams) and store them in memory 102, change characteristics of the still images or videos, or delete still images or videos from memory 102.

[0636] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 contains executable instructions for arranging, modifying (e.g., editing) or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still and / or video images.

[0637] The browser module 147, in conjunction with the RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, contains executable instructions for browsing the Internet according to user instructions, including retrieving, linking, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0638] Calendar module 148, in conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, contains executable instructions for creating, displaying, modifying, and storing calendars and data associated with calendars (e.g., calendar items, to-do lists, etc.) in accordance with a user's instructions.

[0639] In conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is a mini-application that is optionally downloaded and used by a user (e.g., weather widget 149-1, stocks widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or created by a user (e.g., user-created widget 149-6). In some embodiments, a widget includes an HTML (HyperText Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, a widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0640] A widget creator module (not shown), in conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, is optionally used by a user to create a widget (e.g., turn a user-specified portion of a web page into a widget).

[0641] In conjunction with the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching for text, music, sound, images, videos, and / or other files in the memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with a user's instructions. In some embodiments, the search module 151 further includes executable instructions for displaying a search entry portion and a prediction portion (e.g., a search entry portion 920 and a prediction portion 930, as described in more detail below in conjunction with FIG. 9B and FIG. 6A-9C). In some embodiments, the search module 151, in conjunction with the proactive module 163, also populates the prediction portion with affordances for suggested or predicted people, actions within an application, applications, nearby places, and / or news articles prior to receiving user input in the search entry portion (as described in more detail below in conjunction with FIG. 3A-9C).

[0642] In conjunction with touchscreen 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 contains executable instructions that enable a user to download and play pre-recorded music or other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing videos (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an IPOD® from APPLE Inc.

[0643] In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, notes module 153 contains executable instructions for creating and managing notes, to-do lists, and the like, in accordance with a user's instructions.

[0644] In conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 can optionally be used to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data about businesses and other points of interest at or near a particular location, and other location-based data) in accordance with a user's instructions.

[0645] In conjunction with touchscreen 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, online video module 155 contains instructions that enable a user to access, browse, receive (e.g., by streaming and / or downloading), and play (e.g., on the touchscreen or on an external display connected via external port 124) online videos in one or more file formats, such as H.264, and send and otherwise manage emails containing links to particular online videos. In some embodiments, instant messaging module 141 is used to send links to particular online videos, rather than email client module 140.

[0646] 1A , portable multifunction device 100 also includes a proactive module 163 for proactively identifying and surfacing relevant content (e.g., surfacing user interface objects that correspond to actions in an application (e.g., a UI object for playing a playlist in a music app) on the lock screen or in a search interface). Proactive module 163 optionally includes the following modules (or sets of instructions), or subsets or supersets: ● Application Usage Table 335 ● Trigger condition table 402 - Trigger establishment module 163-1 ● Usage data collection module 163-2 ● Proactive Suggestion Module 163-3 ●(Voice communication) content extraction module 163-4

[0647] The application usage table 335 and usage data collection module 163-2, in conjunction with the applications 136, GPS module 135, operating system 126, I / O subsystem 106, RF circuitry 108, external port 124, proximity sensor 166, audio circuitry 110, accelerometer 168, speaker 111, microphone 113, and peripherals interface 118, receive and / or store application usage data (e.g., from the components of device 100 identified in FIG. 1A above). In some embodiments, application usage is reported to the usage data collection module 163-2 and subsequently stored in the application usage table 335. In some embodiments, the application usage data includes all (or most importantly, associated, or predictive) contextual usage information corresponding to a user's use of a particular application 136. In some embodiments, each particular application stores usage data while a user interacts with that application, which usage data is then reported to application usage data table 335 for storage (e.g., in FIG. 3B, usage data 193 for a particular application 136-1 includes all sensor readings, in-application actions performed, device binding information, etc., which usage data 193 is sent to application usage table 335 for storage as records in the table). For example, while a user interacts with browser module 147, application usage data is received by application usage table 335, which receives and stores all contextual usage information including the current GPS coordinates of device 100 (e.g., as determined by GPS module 135), motion data (e.g., as determined by accelerometer 168), ambient light data (e.g., as determined by light sensor 164), and in-application actions performed by the user within browser module 147 (e.g., URLs visited, amount of time viewed each page visited), among other stored sensor data and other contextual usage information. Additional information regarding application usage table 335 is provided below in conjunction with FIGS. 3A-3B.As described below in connection with FIG. 5, in some embodiments, application usage data is stored remot...

Claims

1. 1. A method comprising: In an electronic device having a display and a touch-sensitive surface, presenting on the display a text entry field associated with a first application and text content associated with the first application; determining whether at least a portion of the textual content is associated with an information type of a set of one or more information types; In response to determining that at least a portion of the textual content is associated with a first information type of the set of one or more information types, obtaining first information of the first information type; preparing the obtained first information for display as a predictive content item; displaying an affordance within the first application that includes the predictive content item; Detecting an input via the touch-sensitive surface; In response to detecting the input, displaying a representation of the predictive content item in accordance with a determination that the input includes a selection of an affordance; and ceasing display of the affordance in accordance with a determination that the input includes an input that is not a selection of an affordance; and The method includes:

2. The method of claim 1 , wherein the first information type is a location, and obtaining the first information includes obtaining a proposed physical location.

3. The method of claim 2 , wherein obtaining the proposed physical location includes obtaining current location information from a location sensor of the electronic device.

4. 4. The method of claim 2 or 3, wherein obtaining the suggested physical location comprises analyzing textual content and determining the suggested physical location based at least in part on the analyzed textual content.

5. The method of claim 4 , wherein determining the suggested physical location is further based on recently viewed location information in a second application.

6. The method of claim 2 , wherein the representation of the predicted content item is an address of the proposed physical location.

7. The method of claim 2 , wherein the representation of the predicted content item is a map object that includes an identifier for the proposed physical location.

8. 8. The method of claim 1, wherein the first information type is contacts, and obtaining the first information includes performing a search on the electronic device for contact information associated with the portion of the textual content.

9. 9. The method of claim 1, wherein the first information type is an event, and obtaining the first information includes performing a new search on the electronic device for event information associated with the portion of the textual content.

10. Determining whether at least a portion of the textual content is associated with an information type of the set of one or more information types includes: performing natural language processing on the portion of the textual content; determining that the portion of the textual content includes a question regarding a current location of a user of the electronic device; 10. The method of claim 1 , comprising:

11. Determining whether at least a portion of the textual content is associated with an information type of the set of one or more information types includes:

11. The method of claim 1, further comprising analyzing textual content received by the first application to detect stored patterns known to be associated with an information type from the set of one or more information types.

12. The method of claim 1 , wherein the affordance is displayed adjacent to the text input field.

13. The method of claim 1 , wherein determining whether at least a portion of the textual content is associated with an information type of a set of one or more information types is performed automatically.

14. The method of claim 13 , wherein determining whether at least a portion of the textual content is associated with an information type of a set of one or more information types is based on the textual content.

15. 15. The method of claim 1, further comprising not obtaining a second information of a second information type of the set of one or more information types in accordance with a determination that the portion of the textual content is related to the first information type of the set of one or more information types.

16. The method of claim 1 , wherein the set of one or more information types includes a location information type, a contact information type, and an event information type.

17. A method according to any one of claims 1 to 16, wherein the input that is not a selection of the affordance includes typing input.

18. A computer program for causing an electronic device to carry out the method of any one of claims 1 to 17.

19. 1. An electronic device comprising: A memory storing the computer program according to claim 18; one or more processors capable of executing the computer programs stored in the memory; 11. An electronic device comprising: a first input / output connector for connecting a first input / output port to a touch-sensitive display;

20. configured to communicate with the touch-sensitive display; 18. An electronic device comprising means for carrying out the method according to any one of claims 1 to 17.

Images