System and method for proactively identifying and surfacing relevant content on touch-sensitive device

Abstract

To provide an intuitive way to extract content items from voice communications and present them to a user on an electronic device with a touch-sensitive display.SOLUTION: In response to determining that voice communication includes speech that identifies a physical location, an electronic device provides (2011) an indication that information about the physical location has been detected. Providing the indication includes (2013) displaying a selectable description of the physical location within a user interface that includes recent calls made using a telephone application, or within a user interface that is associated with the voice communication, or within both such user interfaces.SELECTED DRAWING: Figure 20

Description

Technical Field

[0001] Embodiments disclosed herein generally relate to an electronic device having a touch-sensitive display, and more particularly to systems and methods for proactively identifying relevant content and surfacing it on an electronic device that communicates with a display and a touch-sensitive surface.

Background Art

[0002] Handheld electronic devices having touch-sensitive displays are widespread. Users of these widespread handheld electronic devices are currently installing a large number of applications on their devices and using these applications to perform their daily activities more efficiently. However, to access these applications, the user typically has to unlock their device, locate the desired application (e.g., by looking around the home screen to find the location of the icon associated with the desired application or by searching for the desired application within a search interface), and then also locate the location of the desired function within the desired application. Thus, instead of being able to simply and immediately execute the desired application (e.g., by a single touch input) and / or execute the desired function, the user often spends a significant amount of time locating the desired application and the location of the desired functions within these applications.

[0003] Furthermore, the installation of a large number of applications bombards the user with a continuous stream of information, so that not all of it can be viewed immediately. For this reason, users often desire to revisit specific information they noticed earlier or use specific information later. However, often the user is unable to locate the location of the specific information or cannot remember how to locate the location of the specific information.

[0004] For this reason, it is desirable to provide an intuitive and easy-to-use system and method for proactively identifying relevant content (e.g., specific information) and surfacing it on an electronic device that communicates with a display and a touch-sensitive surface. SUMMARY OF THE INVENTION

[0005] Accordingly, there is a need for an electronic device having a faster and more efficient method and interface for quickly accessing applications and desired functions within these applications. Further, there is a need for an electronic device that assists the user in managing the continuous stream of information the user receives daily by proactively identifying and providing relevant information (e.g., contacts, nearby locations, applications, news articles, addresses, and other information available on the device) before this information is explicitly requested by the user. Such methods and interfaces optionally complement or replace conventional methods for accessing applications. Such methods and interfaces result in a more efficient human-machine interface by reducing the input required for the user to identify the location of the desired information. In the case of battery-powered devices, such methods and interfaces save power and extend the battery charging intervals (e.g., by reducing the number of touch inputs required to perform various functions). Further, the lifespan of the touch-sensitive display is extended by reducing the number of touch inputs required by such methods and interfaces (e.g., instead of continuously and aimlessly tapping the touch-sensitive display to identify the location of the desired information, the methods and interfaces disclosed herein proactively provide this information without requiring user input).

[0006] The above-described deficiencies and other problems associated with user interfaces for electronic devices having a touch-sensing surface are addressed by the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensing display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or sets of instructions stored in the memory for performing a plurality of functions. In some embodiments, the user interacts with the GUI primarily by means of a stylus and / or finger contact and gestures on the touch-sensing surface. In some embodiments, the functions optionally include image editing, drawing, presenting, word processing, website creation, disk authoring, spreadsheet creation, game play, making a phone call, video conferencing, sending an email, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, and / or digital video playback. The executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0007] (A1)In some embodiments, a method is performed in an electronic device having a touch-sensing display (touch screen 112, FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E). The method includes executing an application on the electronic device in response to an instruction from a user of the electronic device. The method further includes collecting usage data while the application is being executed. The usage data includes at least one or more actions (or types of actions) performed by the user within the application. The method also includes (i) automatically and without human intervention obtaining at least one trigger condition based on the collected usage data, and (ii) associating the at least one trigger condition with a particular action among the one or more actions performed by the user within the application. The method includes providing an indication to the user that a particular action associated with the trigger condition is available when it is determined that the at least one trigger condition is met.

[0008] (A2)In some embodiments of the method described in A1, obtaining at least one trigger condition includes sending the usage data to one or more servers located remotely from the electronic device and receiving at least one trigger condition from the one or more servers.

[0009] (A3)In some embodiments of the method described in A1 or A2, providing an indication includes displaying a user interface object corresponding to a particular action associated with the trigger condition on a lock screen of the touch-sensing display.

[0010] (A4)In some embodiments of the method described in A3, the user interface object includes an explanation of the particular action associated with the trigger condition.

[0011] In some embodiments of the method described in (A5) A4, the user interface object further includes an icon associated with the application.

[0012] In some embodiments of the method described in any one of (A6) A3 to A5, the method further includes detecting a first gesture in the user interface object. In response to detecting the first gesture, (i) an application is displayed on the touch-sensitive display, and (ii) the method includes performing a specific action associated with a trigger condition while the application is being displayed.

[0013] In some embodiments of the method described in (A7) A6, the first gesture is a swipe gesture on the user interface object.

[0014] In some embodiments of the method described in any one of (A8) A3 to A5, the method further includes detecting a second gesture in the user interface object. In response to detecting the second gesture and while the lock screen is being continuously displayed on the touch-sensitive display, a specific action associated with a trigger condition is performed.

[0015] In some embodiments of the method described in (A9) A8, the second gesture is a single tap in a default area of the user interface object.

[0016] In some embodiments of the method described in any one of (A10) A3 to A9, the user interface object is displayed at a default center of the lock screen.

[0017] In some embodiments of the method described in (A11) A1, providing an indication to the user that a specific action associated with a trigger condition is available includes performing the specific action.

[0018] (A12) In some embodiments of the method described in A3, the user interface object is an icon associated with an application, and the user interface object is substantially displayed at a corner of the lock screen of the touch-sensitive display.

[0019] (A13) In some embodiments of the method described in any one of A1 to A12, the method further includes receiving, from the user, an instruction to unlock the electronic device. The method includes displaying, in response to receiving the instruction, the home screen of the electronic device on the touch-sensitive display. The method also includes providing, on the home screen, an indication to the user that a specific action associated with a trigger condition is available.

[0020] (A14) In some embodiments of the method described in A13, the home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion displayed adjacent to the first portion for launching a second set of applications available on the electronic device. The second portion is displayed on all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion.

[0021] (A15) In some embodiments of the method described in A14, the second set of applications is different from the first set of applications and smaller than the first set of applications.

[0022] (A16) In some embodiments of the method described in any one of A1 to A15, determining that at least one trigger condition is satisfied includes determining that the electronic device is coupled to a second device different from the electronic device.

[0023] (A17)In some embodiments of the method according to any one of A1 to A16, determining that at least one trigger condition is met includes determining that the electronic device has arrived at an address corresponding to the home or workplace associated with the user.

[0024] (A18)In some embodiments of the method according to A17, determining that the electronic device has arrived at an address corresponding to the home or workplace associated with the user includes monitoring motion data from the accelerometer of the electronic device and determining, based on the monitored motion data, that the electronic device has not moved for a threshold amount of time.

[0025] (A19)In some embodiments of the method according to any one of A1 to A18, the usage data further includes verbal instructions provided by the user to the virtual assistant application while the application is running. At least one trigger condition is further based on the verbal instructions provided to the virtual assistant application.

[0026] (A20)In some embodiments of the method according to A19, the verbal instructions include a request to create a reminder corresponding to the current state of the application, and the current state corresponds to the state of the application when the verbal instructions are provided.

[0027] (A21)In some embodiments of the method according to A20, the state of the application when the verbal instructions are provided is selected from the group consisting of the page displayed within the application when the verbal instructions are provided, the content being played within the application when the verbal instructions are provided, the notification displayed within the application when the verbal instructions are provided, and the active portion of the page displayed within the application when the verbal instructions are provided.

[0028] (A22)In some embodiments of the method according to A20, the verbal instructions include the word "this" regarding the current state of the application.

[0029] (A23)In another aspect, the method is executed on one or more electronic devices (e.g., the portable multifunctional device 100 (FIG. 5) and one or more servers 502 (FIG. 5)). The method includes executing an application on a first electronic device among the one or more electronic devices in response to an instruction from a user of the first electronic device. The method includes automatically collecting usage data without human intervention while the application is being executed, the usage data including at least one or more actions (or types of actions) performed by the user within the application. The method further includes automatically providing at least one trigger condition without human intervention based on the collected usage data. The method further includes associating the at least one trigger condition with a specific action among the one or more actions performed by the user within the application. The method includes providing an indication to the user that a specific action associated with the trigger condition is available when it is determined that the at least one trigger condition is met.

[0030] (A24)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method according to any one of A1 to A22.

[0031] (A25)In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for executing the method according to any one of A1 to A22.

[0032] In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform the method described in any one of A1 to A22.

[0033] (A27) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of A1 to A22.

[0034] (A28)In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 4201, FIG. 42), a touch sensing surface unit (e.g., touch sensing surface unit 4203, FIG. 42), and a processing unit (e.g., processing unit 4205, FIG. 42). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 42 shows display unit 4201 and touch sensing surface unit 4203 as being integrated into electronic device 4200, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit is coupled to the touch sensing surface unit and the display unit. In some embodiments, the touch sensing surface unit and the display unit are integrated into a single touch sensing display unit (also referred to herein as a touch display). The processing unit includes an execution unit (e.g., execution unit 4207, FIG. 42), a collection unit (e.g., collection unit 4209, FIG. 42), an acquisition unit (e.g., acquisition unit 4211, FIG. 42), an association unit (e.g., association unit 4213, FIG. 42), a provision unit (e.g., provision unit 4215, FIG. 42), a transmission unit (e.g., transmission unit 4217, FIG. 42), a reception unit (e.g., reception unit 4219, FIG. 42), a display unit (e.g., display unit 4221, FIG. 42), a detection unit (e.g., detection unit 4223, FIG. 42), an implementation unit (e.g., implementation unit 4225, FIG. 42), a determination unit (e.g., determination unit 4227, FIG. 42), and a monitoring unit (e.g., monitoring unit 4229, FIG. 42).The processing unit (or one or more components thereof such as units 4207 to 4229) executes an application on the electronic device in response to an instruction from a user of the electronic device (e.g., by execution unit 4207), and while executing the application, collects usage data (e.g., by collection unit 4209), the usage data at least includes one or more actions executed by the user within the application, collects the usage data, and based on the collected usage data, automatically and without human intervention, obtains at least one trigger condition (e.g., by obtaining unit 4211), associates at least one trigger condition with a specific action among one or more actions executed by the user within the application (e.g., by association unit 4213), and when it is determined that at least one trigger condition is satisfied, provides an indication to the user that the specific action associated with the trigger condition is available (e.g., by providing unit 4215).

[0035] (A29)In some embodiments of the electronic device described in A28, obtaining at least one trigger condition includes transmitting the usage data to one or more servers located remotely from the electronic device (e.g., by transmission unit 4217) and receiving at least one trigger condition from the one or more servers (e.g., by receiving unit 4219).

[0036] (A30)In some embodiments of the electronic device described in A28 or A29, providing the indication includes displaying a user interface object corresponding to the specific action associated with the trigger condition on the lock screen of the touch-sensitive display unit (e.g., by display unit 4217 and / or display unit 4201).

[0037] In some embodiments of the electronic device described in A30, the user interface object includes an explanation of a specific action associated with a trigger condition.

[0038] In some embodiments of the electronic device described in A31, the user interface object further includes an icon associated with an application.

[0039] In some embodiments of the electronic device described in any one of A30 to A32, the processing unit is further configured to detect a first gesture in the user interface object (e.g., by the detection unit 4223 and / or the touch sensing surface unit 4203). In response to detecting the first gesture, (i) an application is displayed on the touch sensing display unit (e.g., by the display unit 4217 and / or the display unit 4201), and (ii) while the application is being displayed, a specific action associated with the trigger condition is executed (e.g., by the execution unit 4225).

[0040] In some embodiments of the electronic device described in A33, the first gesture is a swipe gesture on the user interface object.

[0041] In some embodiments of the electronic device described in any one of A30 to A33, the processing unit is further configured to detect a second gesture in the user interface object (e.g., by the detection unit 4223 and / or the touch sensing surface unit 4203). In response to detecting the second gesture and while the lock screen is being continuously displayed on the touch sensing display unit, the processing unit is configured to execute a specific action associated with the trigger condition (e.g., by the execution unit 4225).

[0042] (A36)In some embodiments of the electronic device described in A35, the second gesture is a single tap in a default area of the user interface object.

[0043] (A37)In some embodiments of the electronic device described in any one of A30 to A36, the user interface object is displayed in a default central part of the lock screen.

[0044] (A38)In some embodiments of the electronic device described in A28, providing an indication to the user that a specific action associated with a trigger condition is available includes performing the specific action (e.g., by the execution unit 4225).

[0045] (A39)In some embodiments of the electronic device described in A30, the user interface object is an icon associated with an application, and the user interface object is substantially displayed at a corner of the lock screen of the touch-sensing display unit.

[0046] (A40)In some embodiments of the electronic device described in any one of A28 to A39, the processing unit is further configured to receive a command from the user to unlock the electronic device (e.g., by the receiving unit 4219). In response to receiving the command, the processing unit is configured to display the home screen of the electronic device on the touch-sensing display unit (e.g., by the display unit 4217 and / or the display unit 4201). The processing unit is also configured to provide an indication to the user that a specific action associated with a trigger condition is available on the home screen (e.g., by the providing unit 4215).

[0047] (A41)In some embodiments of the electronic device described in A40, the home screen includes: (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device; and (ii) a second portion, adjacent to the first portion, for launching a second set of applications available on the electronic device. The second portion is displayed on all user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion (e.g., by display unit 4217 and / or display unit 4201).

[0048] (A42)In some embodiments of the electronic device described in A41, the second set of applications is different from the first set of applications and smaller than the first set of applications.

[0049] (A43)In some embodiments of the electronic device described in any one of A28 to A42, determining that at least one trigger condition is satisfied includes determining that the electronic device is coupled to a second device different from the electronic device (e.g., by determination unit 4227).

[0050] (A44)In some embodiments of the electronic device described in any one of A28 to A43, determining that at least one trigger condition is satisfied includes determining that the electronic device has arrived at an address corresponding to a home or workplace associated with the user (e.g., by determination unit 4227).

[0051] (A45)In some embodiments of the electronic device described in A44, determining that the electronic device has arrived at an address corresponding to a home or workplace associated with the user includes monitoring motion data from the accelerometer of the electronic device (e.g., by the monitoring unit 4229) and determining, based on the monitored motion data, that the electronic device has not moved for a threshold amount of time (e.g., by the determination unit 4227).

[0052] (A46)In some embodiments of the electronic device described in any one of A28 to A45, the usage data further includes verbal instructions provided by the user to the virtual assistant application while the application is being continuously executed. At least one trigger condition is further based on the verbal instructions provided to the virtual assistant application.

[0053] (A47)In some embodiments of the electronic device described in A46, the verbal instruction includes a request to create a reminder corresponding to the current state of the application, and the current state corresponds to the state of the application when the verbal instruction is provided.

[0054] (A48)In some embodiments of the electronic device described in A47, the state of the application when the verbal instruction is provided is selected from the group consisting of the page displayed within the application when the verbal instruction is provided, the content being played within the application when the verbal instruction is provided, the notification displayed within the application when the verbal instruction is provided, and the active portion of the page displayed within the application when the verbal instruction is provided.

[0055] (A49)In some embodiments of the electronic device described in A46, the verbal instruction includes the word "this" regarding the current state of the application.

[0056] (B1)In some embodiments, a method is performed in an electronic device having a touch-sensing display (touch screen 112, FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E). The method includes obtaining at least one trigger condition based on usage data associated with a user of the electronic device, the usage data including one or more actions (or types of actions) performed by the user within an application while the application is being executed on the electronic device. The method also includes associating the at least one trigger condition with a particular action among the one or more actions performed by the user within the application. The method includes providing an indication to the user that a particular action associated with the trigger condition is available when it is determined that the at least one trigger condition is met.

[0057] (B2)In some embodiments of the method according to B1, the method further includes the method described in any one of A2 to A22.

[0058] (B3)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensing display, one or more processors, and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to perform the method described in B1 or B2.

[0059] (B4)In yet another aspect, an electronic device is provided, the electronic device including a touch-sensing display and means for performing the method described in B1 or B2.

[0060] (B5) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform the method described in B1 or B2.

[0061] (B6) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided. In some embodiments, the graphical user interface includes a user interface that is displayed according to the method described in B1 or B2.

[0062] In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4201, FIG. 42), a touch sensing surface unit (e.g., touch sensing surface unit 4203, FIG. 42), and a processing unit (e.g., processing unit 4205, FIG. 42). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A to D). For ease of explanation, FIG. 42 shows the display unit 4201 and the touch sensing surface unit 4203 as being integrated into the electronic device 4200, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes an execution unit (e.g., execution unit 4207, FIG. 42), a collection unit (e.g., collection unit 4209, FIG. 42), an acquisition unit (e.g., acquisition unit 4211, FIG. 42), an association unit (e.g., association unit 4213, FIG. 42), a provision unit (e.g., provision unit 4215, FIG. 42), a transmission unit (e.g., transmission unit 4217, FIG. 42), a reception unit (e.g., reception unit 4219, FIG. 42), a display unit (e.g., display unit 4221, FIG. 42), a detection unit (e.g., detection unit 4223, FIG. 42), an implementation unit (e.g., implementation unit 4225, FIG. 42), a determination unit (e.g., determination unit 4227, FIG. 42), and a monitoring unit (e.g., monitoring unit 4229, FIG. 42).The processing unit (or one or more components thereof such as units 4207 to 4229) obtains at least one trigger condition based on usage data associated with a user of the electronic device, where the usage data includes one or more actions performed by the user within an application while the application is being executed on the electronic device (e.g., by obtaining unit 4211), associates the at least one trigger condition with a specific action among the one or more actions performed by the user within the application (e.g., by associating unit 4213), and when it is determined that the at least one trigger condition is satisfied, provides an indication to the user that the specific action associated with the trigger condition is available (e.g., by providing unit 4215).

[0063] (B8)In some embodiments of the electronic device according to B7, obtaining at least one trigger condition includes transmitting usage data to one or more servers located remotely from the electronic device (e.g., by transmitting unit 4217) and receiving at least one trigger condition from the one or more servers (e.g., by receiving unit 4219).

[0064] (B9)In some embodiments of the electronic device according to B7 or B8, providing an indication includes displaying a user interface object corresponding to a specific action associated with the trigger condition on a lock screen of a touch-sensitive display (e.g., by display unit 4217 and / or display unit 4201).

[0065] (B10)In some embodiments of the electronic device according to B9, the user interface object includes an explanation of the specific action associated with the trigger condition.

[0066] In some embodiments of the electronic device described in B10 (B11), the user interface object further includes an icon associated with the application.

[0067] In some embodiments of the electronic device described in any one of B9 to B11 (B12), the processing unit is further configured to detect a first gesture in the user interface object (e.g., by the detection unit 4223 and / or the touch sensing surface unit 4203). In response to detecting the first gesture, (i) an application is displayed on the touch sensing display (e.g., by the display unit 4217 and / or the display unit 4201), and (ii) while the application is being displayed, a specific action associated with the trigger condition is executed (e.g., by the execution unit 4225).

[0068] In some embodiments of the electronic device described in B12 (B13), the first gesture is a swipe gesture on the user interface object.

[0069] In some embodiments of the electronic device described in any one of B9 to B12 (B14), the processing unit is further configured to detect a second gesture in the user interface object (e.g., by the detection unit 4223 and / or the touch sensing surface unit 4203). In response to detecting the second gesture and while the lock screen is continuously displayed on the touch sensing display, the processing unit is configured to execute a specific action associated with the trigger condition (e.g., by the execution unit 4225).

[0070] In some embodiments of the electronic device described in B14 (B15), the second gesture is a single tap in a default area of the user interface object.

[0071] In some embodiments of the electronic device described in any one of B9 to B15 (B16), the user interface object is displayed at a default central portion of the lock screen.

[0072] In some embodiments of the electronic device described in B7 (B17), providing an indication to the user that a particular action associated with a trigger condition is available includes performing the particular action (e.g., by implementation unit 4225).

[0073] In some embodiments of the electronic device described in B9 (B18), the user interface object is an icon associated with an application, and the user interface object is displayed substantially at a corner of the lock screen of the touch-sensitive display.

[0074] In some embodiments of the electronic device described in any one of B7 to B18 (B19), the processing unit is further configured to receive (e.g., by receiving unit 4219) a command from the user to unlock the electronic device. In response to receiving the command, the processing unit is configured to display (e.g., by display unit 4217 and / or display unit 4201) the home screen of the electronic device on the touch-sensitive display. The processing unit is also configured to provide (e.g., by providing unit 4215) an indication to the user that a particular action associated with a trigger condition is available on the home screen.

[0075] In some embodiments of the electronic device described in (B20)B19, the home screen includes: (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device; and (ii) a second portion, adjacent to the first portion, for launching a second set of applications available on the electronic device. The second portion is displayed on all the user interface pages included in the first portion, and providing an indication on the home screen includes displaying the indication on the second portion (e.g., by the display unit 4217 and / or the display unit 4201).

[0076] (B21)In some embodiments of the electronic device described in B20, the second set of applications is different from the first set of applications and is smaller than the first set of applications.

[0077] (B22)In some embodiments of the electronic device described in any one of B7 to B21, determining that at least one trigger condition is satisfied includes determining that the electronic device is coupled to a second device different from the electronic device (e.g., by the determination unit 4227).

[0078] (B23)In some embodiments of the electronic device described in any one of B7 to B22, determining that at least one trigger condition is satisfied includes determining that the electronic device has arrived at an address corresponding to the home or workplace associated with the user (e.g., by the determination unit 4227).

[0079] In some embodiments of the electronic device described in (B24) B23, determining that the electronic device has arrived at an address corresponding to a home or workplace associated with the user includes monitoring motion data from an accelerometer of the electronic device (e.g., by monitoring unit 4229) and determining, based on the monitored motion data, that the electronic device has not moved for a threshold amount of time (e.g., by determination unit 4227).

[0080] In some embodiments of the electronic device described in any one of (B25) B7 to B24, the usage data further includes verbal commands provided by the user to the virtual assistant application while the application is being continuously executed. At least one trigger condition is further based on verbal commands provided to the virtual assistant application.

[0081] In some embodiments of the electronic device described in (B26) B25, the verbal command includes a request to create a reminder corresponding to the current state of the application, and the current state corresponds to the state of the application when the verbal command was provided.

[0082] In some embodiments of the electronic device described in (B27) B26, the state of the application when the verbal command was provided is selected from the group consisting of the page displayed within the application when the verbal command was provided, the content being played within the application when the verbal command was provided, the notification displayed within the application when the verbal command was provided, and the active portion of the page displayed within the application when the verbal command was provided.

[0083] In some embodiments of the electronic device described in (B28) B26, the verbal command includes the word "this" regarding the current state of the application.

[0084] (C1)In some embodiments, the method is performed on an electronic device having a touch-sensitive display (touch screen 112, FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E). The method includes detecting, on the touch-sensitive display, an activation gesture of a search by a user of the electronic device. The method includes, in response to detecting the activation gesture of the search, causing a search interface to be displayed on the touch-sensitive display, the search interface including (i) a search entry portion and (ii) a prediction portion that is displayed before any user input is received in the search entry portion. The prediction portion is populated with one or more of: (a) at least one affordance for contacting a person among a plurality of previously contacted persons, the person being automatically selected, at least in part, based on the current time from among the plurality of previously contacted persons; and (b) at least one affordance for performing a predicted action within an application among a plurality of applications available on the electronic device, the predicted action being automatically selected, at least in part, based on an application usage history associated with the user of the electronic device.

[0085] (C2)In some embodiments of the method of C1, the person is further selected, at least in part, based on location data corresponding to the electronic device.

[0086] (C3)In some embodiments of the method of C1 or C2, the application usage history and contact information for the person are retrieved from the memory of the electronic device.

[0087] (C4)In some embodiments of the method of C1 or C2, the application usage history and contact information for the person are retrieved from a server located remotely from the electronic device.

[0088] (C5)In some embodiments of the method described in any one of C1 to C4, the prediction part is further populated with at least one affordance for executing the predicted application, and the predicted application is automatically selected at least in part based on the application usage history.

[0089] (C6)In some embodiments of the method described in any one of C1 to C5, the prediction part is further populated with at least one affordance of the predicted location (or nearby location) category, and the predicted location category is automatically selected at least in part based on one or more of the current time and the location data corresponding to the electronic device.

[0090] (C7)In some embodiments of the method described in any one of C1 to C6, the method further includes detecting a user input for scrolling the prediction part. The method includes scrolling the prediction part according to the user input in response to detecting the user input for scrolling the prediction part. The method includes causing at least one affordance of the predicted news article to be displayed in the prediction part in response to the scrolling (for example, the predicted news article is expected to be of interest to the user).

[0091] (C8)In some embodiments of the method described in C7, the predicted news article is automatically selected at least in part based on the location data corresponding to the electronic device.

[0092] (C9)In some embodiments of the method described in any one of C1 to C8, the method further includes detecting a selection of at least one affordance for executing the predicted action within the application. The method includes displaying the application on the touch-sensitive display and executing the predicted action within the application in response to detecting the selection.

[0093] (C10)In some embodiments of the method described in C3 or C4, the method further includes detecting a selection of at least one affordance for contacting a person. The method includes contacting the person using contact information for the person in response to detecting the selection.

[0094] (C11)In some embodiments of the method described in C5, the method further includes detecting a selection of at least one affordance for executing a predicted application. The method includes displaying the predicted application on a touch-sensitive display in response to detecting the selection.

[0095] (C12)In some embodiments of the method described in C6, the method further includes detecting a selection of at least one affordance of a predicted location category. The method further includes, in response to detecting the selection, (i) receiving data corresponding to at least one nearby location, and (ii) displaying the received data corresponding to at least one nearby location on a touch-sensitive display.

[0096] (C13)In some embodiments of the method described in C7, the method further includes detecting a selection of at least one affordance of a predicted news article. The method includes displaying the predicted news article on a touch-sensitive display in response to detecting the selection.

[0097] (C14)In some embodiments of the method described in any one of C1 to C13, the search activation gesture is available from at least two different user interfaces, and a first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of a series of home screen pages on a touch-sensitive display.

[0098] (C15)In some embodiments of the method described in C14, when the corresponding home screen page is the first home screen page among a series of home screen pages, the search activation gesture includes one of the following, namely, (i) a gesture that moves substantially downward with respect to the user of the electronic device, or (ii) a continuous gesture that moves with respect to the user in a substantially left-to-right direction and substantially perpendicular downward.

[0099] (C16)In some embodiments of the method described in C15, when the corresponding home screen page is the second home screen page among a series of home screen pages, the search activation gesture includes a continuous gesture that moves substantially downward with respect to the user of the electronic device.

[0100] (C17)In some embodiments of the method described in C14, the second user interface among at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display.

[0101] (C18)In some embodiments of the method described in C17, the search activation gesture includes contact at a default search activation portion of the application switching interface on the touch-sensitive display.

[0102] (C19)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive display, one or more processors, and a memory storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of C1 to C18.

[0103] (C20)In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive display and means for executing the method described in any one of C1 to C18.

[0104] (C21) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch-sensitive display, cause the electronic device to perform the method described in any one of C1 to C18.

[0105] (C22) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of C1 to C18.

[0106] (C23) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4301, FIG. 43), a touch sensing surface unit (e.g., touch sensing surface unit 4303, FIG. 43), and a processing unit (e.g., processing unit 4305, FIG. 43). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A to D). For ease of explanation, FIG. 43 shows the display unit 4301 and the touch sensing surface unit 4303 as being integrated into the electronic device 4300, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes a display unit (e.g., display unit 4309, FIG. 43), a detection unit (e.g., detection unit 4307, FIG. 43), an extraction unit (e.g., extraction unit 4311, FIG. 43), a populating unit (e.g., populating unit 4313, FIG. 43), a scrolling unit (e.g., scrolling unit 4315, FIG. 43), a visualization unit (e.g., visualization unit 4317, FIG. 43), a selection unit (e.g., selection unit 4319, FIG. 43), a communication unit (e.g., communication unit 4321, FIG. 43), a receiving unit (e.g., receiving unit 4323, FIG. 43), and an execution unit (e.g., execution unit 4325, FIG. 43). The processing unit (or one or more of its components such as units 4307 to 4225) is configured to detect a search activation gesture of a user of the electronic device on the touch sensing display (e.g., by detection unit 4307 and / or touch sensing surface unit 4303), and in response to detecting the search activation gesture, display a search interface on the touch sensing display (e.g., by display unit 4309 and / or display unit 4301), and the search interface includes (i) a search entry portion and (ii) a prediction portion that is displayed before receiving any user input in the search entry portion.The prediction part is populated with one or more of: (a) at least one affordance for contacting a person among a plurality of previously contacted persons, where the person is automatically selected, at least in part, based on the current time from among the plurality of previously contacted persons (e.g., by selection unit 4319), the at least one affordance; and (b) at least one affordance for performing a predicted action within an application among a plurality of applications available on the electronic device, where the predicted action is automatically selected, at least in part, based on the application usage history associated with the user of the electronic device (e.g., by selection unit 4319), the at least one affordance.

[0107] (C24)In some embodiments of the electronic device according to C23, the person is further selected, at least in part, based on location data corresponding to the electronic device (e.g., by selection unit 4319).

[0108] (C25)In some embodiments of the electronic device according to C23 or C24, the application usage history and contact information for the person are retrieved from the memory of the electronic device (e.g., by retrieval unit 4311).

[0109] (C26)In some embodiments of the electronic device according to C23 or C24, the application usage history and contact information for the person are retrieved from a server located remotely from the electronic device (e.g., by retrieval unit 4311).

[0110] (C27)In some embodiments of the electronic device according to any one of C23 to C26, the prediction part is further populated with at least one affordance for performing a predicted application (e.g., by populating unit 4313), where the predicted application is automatically selected, at least in part, based on the application usage history (e.g., by selection unit 4319).

[0111] (C28)In some embodiments of the electronic device according to any one of C23 to C27, the prediction part is further populated with at least one affordance of the predicted location category (e.g., by the populating unit 4313), and the predicted location category is automatically selected (e.g., by the selection unit 4319) based at least in part on the current time and one or more of the position data corresponding to the electronic device.

[0112] (C29)In some embodiments of the electronic device according to any one of C23 to C28, the processing unit is further configured to detect a user input for scrolling the prediction part (e.g., by the detection unit 4307 and / or the touch sensing surface unit 4303). In response to detecting a user input for scrolling the prediction part, the processing unit is configured to scroll the prediction part according to the user input (e.g., by the scrolling unit 4319). In response to the scrolling, the processing unit is configured to visualize at least one affordance of the predicted news article in the prediction part (e.g., by the visualization unit 4317) (e.g., the predicted news article is expected to be of interest to the user).

[0113] (C30)In some embodiments of the electronic device according to C7, the predicted news article is automatically selected (e.g., by the selection unit 4319) based at least in part on the position data corresponding to the electronic device.

[0114] In some embodiments of the electronic device described in any one of C23 to C30, the processing unit is further configured to detect a selection of at least one affordance for executing a predicted action within an application (e.g., by the detection unit 4307 and / or the touch sensing surface unit 4303). In response to detecting the selection, the processing unit is configured to display the application on a touch sensing display (e.g., by the display unit 4309) and execute the predicted action within the application (e.g., by the execution unit 4325).

[0115] (C32) In some embodiments of the electronic device described in C25 or C26, the processing unit is further configured to detect a selection of at least one affordance for contacting a person (e.g., by the detection unit 4307 and / or the touch sensing surface unit 4303). In response to detecting the selection, the processing unit is configured to contact the person using contact information for the person (e.g., by the contact unit 4321).

[0116] (C33) In some embodiments of the electronic device described in C27, the processing unit is further configured to detect a selection of at least one affordance for executing a predicted application (e.g., by the detection unit 4307 and / or the touch sensing surface unit 4303). In response to detecting the selection, the processing unit is configured to display the predicted application on a touch sensing display (e.g., by the display unit 4307).

[0117] (C34)In some embodiments of the electronic device described in C28, the processing unit is further configured to detect the selection of at least one affordance of a predicted location category (e.g., by detection unit 4307 and / or touch sensing surface unit 4303). In response to detecting the selection, the processing unit is configured to (i) receive data corresponding to at least one nearby location (e.g., by receiving unit 4323), and (ii) display the received data corresponding to at least one nearby location on a touch sensing display (e.g., display unit 4301) (e.g., by display unit 4307).

[0118] (C35)In some embodiments of the electronic device described in C29, the processing unit is further configured to detect the selection of at least one affordance of a predicted news article (e.g., by detection unit 4307 and / or touch sensing surface unit 4303). In response to detecting the selection, the processing unit is configured to display the predicted news article on a touch sensing display (e.g., display unit 4301) (e.g., by display unit 4307).

[0119] (C36)In some embodiments of the electronic device described in any one of C23 to C35, the search activation gesture is available from at least two different user interfaces, and the first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of a series of home screen pages on a touch sensing display.

[0120] In some embodiments of the electronic device described in C36, when the corresponding home screen page is the first home screen page among a series of home screen pages, the search activation gesture includes one of the following, namely, (i) a gesture that moves substantially downward with respect to the user of the electronic device, or (ii) a continuous gesture that moves with respect to the user in a substantially left-to-right direction and substantially perpendicular downward.

[0121] In some embodiments of the electronic device described in C38, when the corresponding home screen page is the second home screen page among a series of home screen pages, the search activation gesture includes a continuous gesture that moves substantially downward with respect to the user of the electronic device.

[0122] In some embodiments of the electronic device described in C39, the second user interface among at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display.

[0123] In some embodiments of the electronic device described in C40, the search activation gesture includes contact at a predefined search activation portion of the application switching interface on the touch-sensitive display.

[0124] Accordingly, an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface provides a faster and more efficient method and interface for proactively accessing applications and proactively executing functions within the applications, thereby enhancing the effectiveness, efficiency, and user satisfaction in such devices. Such methods and interfaces may complement or replace conventional methods for accessing applications and related functions thereof.

[0125] (D1)In some embodiments, a method is performed on an electronic device (e.g., a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E)) having a touch sensing surface (e.g., touch sensing surface 195, FIG. 1D) and a display (e.g., display 194, FIG. 1D). The method includes displaying, on the display, content associated with an application being executed on the electronic device. The method further includes detecting, via the touch sensing surface, a swipe gesture that, when detected, causes the electronic device to transition to a search mode different from the application. The method also includes transitioning to the search mode in response to detecting the swipe gesture, the search mode including a search interface displayed on the display. In conjunction with transitioning to the search mode, the method includes determining at least one proposed search query based at least in part on information associated with the content. The method includes populating the displayed search interface with at least one proposed search query before receiving any user input in the search interface. In this way, instead of having to remember information and re-enter it into the search interface, the device provides the user with relevant suggestions based on the application content the user is viewing, and the user only needs to select one of the suggestions without having to type anything.

[0126] (D2)In some embodiments of the method described in D1, detecting the swipe gesture includes detecting a swipe gesture on at least a portion of the currently displayed content.

[0127] (D3)In some embodiments of the method described in D1 or D2, the method further includes detecting an input corresponding to a request to view the home screen of the electronic device before detecting a swipe gesture, and in response to detecting the input, stopping displaying content associated with an application and displaying a corresponding page of the home screen of the electronic device. In some embodiments, the corresponding page is the initial page among a series of home screen pages, and the swipe gesture is detected while the initial page of the home screen is being displayed on the display.

[0128] (D4)In some embodiments of the method described in any one of D1 to D3, the search interface is displayed semi-transparently overlapping the application.

[0129] (D5)In some embodiments of the method described in any one of D1 to D4, the method further includes determining at least one proposed search query based at least in part on text content, according to a determination that the content includes text content.

[0130] (D6)In some embodiments of the method described in D5, determining at least one proposed search query based at least in part on text content includes analyzing the text content to detect one or more predefined keywords used to determine at least one proposed search query.

[0131] (D7)In some embodiments of the method described in any one of D1 to D6, determining at least one proposed search query includes determining a plurality of proposed search queries, and populating the search interface includes populating the search interface with a plurality of proposed search queries.

[0132] (D8) In some embodiments of the method according to any one of D1 to D7, the method includes detecting a new swipe gesture on currently displayed new content via a touch sensing surface, and in response to detecting the new swipe gesture, transitioning to a search mode, where transitioning to the search mode includes displaying a search interface on the display, and in addition to transitioning to the search mode and in accordance with the determination that the new content does not include text content, populating the search interface with search queries proposed based on a set of selected past search queries by a user of the electronic device.

[0133] (D9) In some embodiments of the method according to D8, based on location information provided by a second application different from the application, the search interface is displayed together with a target location.

[0134] (D10) In some embodiments of the method according to any one of D8 or D9, the search interface further includes one or more proposed applications.

[0135] (D11) In some embodiments of the method according to any one of D8 to D10, the set of past search queries is selected at least in part based on the frequency of recent search queries.

[0136] (D12) In some embodiments of the method according to any one of D1 to D11, the method further includes, in addition to transitioning to the search mode, obtaining information associated with the content by using one or more accessibility functions available on the electronic device.

[0137] In some embodiments of the method described in (D13) D12, using one or more accessibility features includes (i) applying a natural language processing algorithm to text content currently displayed within an application, and (ii) determining one or more keywords for explaining the content using data obtained from the natural language processing algorithm, wherein at least one proposed search query is determined based on the one or more keywords, thereby generating information associated with the content using one or more accessibility features.

[0138] In some embodiments of the method described in (D14) D13, determining one or more keywords for explaining the content also includes (i) extracting metadata corresponding to non-text content currently displayed within the application, and (ii) using the extracted metadata, in addition to data obtained from the natural language processing algorithm, to determine the one or more keywords.

[0139] In some embodiments of the method described in (D15) D1 to D14, the search interface further includes one or more trend queries.

[0140] In some embodiments of the method described in (D16) D15, the search interface further includes one or more applications that a user of the electronic device is expected to be interested in.

[0141] In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to perform the method described in any one of D1 to D16.

[0142] (D18) In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, and means for performing the method described in any one of D1 to D16.

[0143] (D19) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface and a display, cause the electronic device to perform the method described in any one of D1 to D16.

[0144] (D20) In a further aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of D1 to D16. In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for performing the method described in any one of D1 to D16.

[0145] (D21) In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 4401, FIG. 44), a touch sensing surface unit (e.g., touch sensing surface unit 4403, FIG. 44), and a processing unit (e.g., processing unit 4405, FIG. 44). The processing unit is coupled to the touch sensing surface unit and the display unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 44 shows display unit 4401 and touch sensing surface unit 4403 as being integrated into electronic device 4400, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. In some embodiments, the touch sensing surface unit and the display unit are integrated into a single touch sensing display unit (also referred to herein as a touch sensing display). The processing unit includes a detection unit (e.g., detection unit 4407, FIG. 44), a display unit (e.g., display unit 4409, FIG. 44), an extraction unit (e.g., extraction unit 4411, FIG. 44), a search mode transition unit (e.g., search mode transition unit 4412, FIG. 44), a populating unit (e.g., populating unit 4413, FIG. 44), an acquisition unit (e.g., acquisition unit 4415, FIG. 44), a determination unit (e.g., determination unit 4417, FIG. 44), and a selection unit (e.g., selection unit 4419, FIG. 44).The processing unit (or one or more of its components such as units 1007 to 1029) displays, on a display unit (e.g., display unit 4407), content associated with an application being executed on the electronic device (e.g., by display unit 4407), detects a swipe gesture (e.g., by detection unit 4407) that, when detected via a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4403), causes the electronic device to transition to a search mode different from the application, and in response to detecting the swipe gesture, the search mode transitions (e.g., by search mode transition unit 4412) to a search mode that includes a search interface displayed on the display unit (e.g., display unit 4407), and in conjunction with transitioning to the search mode, determines (e.g., by determination unit 4417) at least one proposed search query based on at least some of the information associated with the content, and populates (e.g., by populating unit 4413) the displayed search interface with at least one proposed search query before receiving any user input in the search interface.

[0146] (D22)In some embodiments of the electronic device described in D21, detecting a swipe gesture includes detecting a swipe gesture on at least a portion of the currently displayed content (e.g., by detection unit 4407).

[0147] (D23) In some embodiments of the electronic device described in D21 or D22, the processing unit detects an input corresponding to a request to view the home screen of the electronic device (e.g., by the detection unit 4407) before detecting a swipe gesture, and in response to the detection of the input (e.g., by the detection unit 4407), stops displaying the content associated with the application and further configures to display the corresponding page of the home screen of the electronic device (e.g., by the display unit 4409), where the corresponding page is the initial page among a series of home screen pages, and the swipe gesture is detected while the initial page of the home screen is being displayed on the display unit (e.g., by the detection unit 4407).

[0148] (D24) In some embodiments of the electronic device described in any one of D21 to D23, the search interface is displayed semi-transparently overlapping the application (e.g., by the display unit 4409 and / or the display unit 4401).

[0149] (D25) In some embodiments of the electronic device described in any one of D21 to D24, the processing unit is further configured to determine at least one proposed search query based at least in part on the text content according to a determination that the content includes text content (e.g., by the determination unit 4417).

[0150] (D26) In some embodiments of the electronic device described in D25, determining at least one proposed search query based at least in part on the text content includes detecting one or more defined keywords used (e.g., by the detection unit 4407) to analyze the text content to determine at least one proposed search query (e.g., by the determination unit 4417).

[0151] (D27) In some embodiments of the electronic device described in any one of D21 to D26, determining at least one proposed search query includes determining a plurality of proposed search queries (e.g., by determination unit 4417), and populating the search interface includes populating the search interface with a plurality of proposed search queries (e.g., by populating unit 4413).

[0152] (D28) In some embodiments of the electronic device described in any one of D21 to D27, the processing unit detects a new swipe gesture on the newly displayed content via a touch sensing surface unit (e.g., by touch sensing surface unit 4403) (e.g., by detection unit 4407), and in response to detecting the new swipe gesture, transitioning to the search mode includes displaying a search interface on the display unit (e.g., by display unit 4409) (e.g., by search mode transition unit 4412), and in addition to transitioning to the search mode, and in accordance with the determination that the new content does not include text content, the search interface is further configured to be populated with proposed search queries based on a set of selected past search queries by the user of the electronic device (e.g., by populating unit 4413).

[0153] (D29) In some embodiments of the electronic device described in D28, a target location is displayed on the search interface based on location information provided by a second application different from the application (e.g., display unit 4409).

[0154] (D30) In some embodiments of the electronic device described in D28 or D29, the search interface further includes one or more proposed applications.

[0155] In some embodiments of the electronic device described in any one of D28 to D30 (D31), a set of past search queries is selected, at least in part, based on the frequency of recent search queries (e.g., by the selection unit 4419).

[0156] In some embodiments of the electronic device described in any one of D21 to D31 (D32), the processing unit is further configured to obtain information associated with the content by using one or more accessibility functions available on the electronic device in conjunction with transitioning to a search mode (e.g., by the acquisition unit 4415).

[0157] In some embodiments of the electronic device described in D32 (D33), using one or more accessibility functions includes (i) applying a natural language processing algorithm to text content currently displayed within an application, and (ii) determining one or more keywords that describe the content using data obtained from the natural language processing algorithm (e.g., by the acquisition unit 4415) (e.g., by the determination unit 4417), wherein at least one proposed search query is determined based on the one or more keywords (e.g., by the determination unit 4417), thereby generating information associated with the content using one or more accessibility functions.

[0158] In some embodiments of the electronic device described in D33 (D34), determining one or more keywords that describe the content also includes (i) retrieving metadata corresponding to non-text content currently displayed within the application (e.g., by the retrieval unit 4411), and (ii) determining the one or more keywords using the retrieved metadata in addition to data obtained from the natural language processing algorithm (e.g., by the determination unit 4417).

[0159] In some embodiments of the electronic device according to any one of D21 to D34 (D35), the search interface further includes one or more trend queries.

[0160] In some embodiments of the electronic device according to D35 (D36), the search interface further includes one or more applications that are expected to be of interest to the user of the electronic device.

[0161] In some embodiments, a method is performed in an electronic device (e.g., a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E)) having a touch sensing surface (e.g., touch sensing surface 195, FIG. 1D) and a display (e.g., display 194, FIG. 1D). The method includes detecting a swipe gesture on the user interface via the touch sensing surface, the swipe gesture causing the electronic device to transition to a search mode when detected. The method further includes transitioning to the search mode in response to detecting the swipe gesture, and transitioning to the search mode includes populating a search interface different from the user interface with a first content item before receiving any user input within the search interface. In some embodiments, populating the search interface with the first content item according to a determination that the user interface includes content associated with an application different from a home screen including selectable icons for invoking applications includes populating the search interface with at least one proposed search query based at least in part on content associated with the application, and populating the search interface with the first content item according to a determination that the user interface is associated with a page of the home screen includes populating the search interface with an affordance including a selectable description of at least one target location within a threshold distance of the current location of the electronic device.

[0162] (E2) In some embodiments of the method described in E1, populating the search interface with affordances includes displaying a search entry portion of the search interface on a touch-sensitive surface, and the method further includes detecting an input in the search entry portion and, in response to detecting the input in the search entry portion, stopping displaying the affordances and displaying at least one proposed search query within the search interface.

[0163] (E3) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to perform the method described in E1 or E2.

[0164] (E4) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method described in E1 or E2.

[0165] (E5) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform the method described in E1 or E2.

[0166] (E6) In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in E1 or E2. In a further aspect, an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display is provided. The information processing apparatus includes means for performing the method described in E1 or E2.

[0167] (E7)In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 4501, FIG. 45), a touch sensing surface unit (e.g., touch sensing surface unit 4503, FIG. 45), and a processing unit (e.g., processing unit 4505, FIG. 45). The processing unit is coupled to the touch sensing surface unit and the display unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 45 shows the display unit 4501 and the touch sensing surface unit 4503 as being integrated into the electronic device 4500, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. In some embodiments, the touch sensing surface unit and the display unit are integrated into a single touch sensing display unit (also referred to herein as a touch sensing display). The processing unit includes a detection unit (e.g., detection unit 4507, FIG. 45), a display unit (e.g., display unit 4509, FIG. 45), a populating unit (e.g., populating unit 4511, FIG. 45), and a search mode transition unit (e.g., search mode transition unit 4513, FIG. 45). The processing unit (or one or more of its components such as units 4507 through 4513) is configured to detect, via the touch sensing surface unit (e.g., touch sensing surface unit 4503), a swipe gesture on the user interface, where the swipe gesture causes the electronic device to transition to a search mode when the swipe gesture is detected (e.g., by detection unit 4507), and in response to detecting the swipe gesture, transition to the search mode (e.g., by search mode transition unit 4513), where transitioning to the search mode includes populating a search interface different from the user interface with a first content item (e.g., by populating unit 4511) before receiving any user input within the search interface.In some embodiments, populating the search interface with a first content item in accordance with a determination that the user interface includes content associated with an application different from a selectable icon for invoking the application includes (e.g., by populating unit 4511) populating the search interface with at least one proposed search query based at least in part on content associated with the application, and populating the search interface with a first content item in accordance with a determination that the user interface is associated with a page of the home screen includes (e.g., by populating unit 4511) populating the search interface with an affordance that includes a selectable description of at least one target location within a threshold distance of the current location of the electronic device.

[0168] (E8) In some embodiments of the electronic device according to E7, populating the search interface with an affordance includes (e.g., by display unit 4507 and / or display unit 4501) displaying a search entry portion of the search interface, the processing unit detecting an input in the search entry portion (e.g., by detection unit 4507), and in response to detecting the input in the search entry portion, stopping displaying the affordance (e.g., by display unit 4507 and / or display unit 4501) and further configured to display at least one proposed search query within the search interface (e.g., by display unit 4507 and / or display unit 4501).

[0169] (F1)In some embodiments, a method is performed in an electronic device (e.g., a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E)) having a position sensor, a touch sensing surface (e.g., touch sensing surface 195, FIG. 1D), and a display (e.g., display 194, FIG. 1D). The method includes automatically and without an instruction from a user, determining that a user of the electronic device is in a vehicle stopped at a geographical location; and when it is determined that the user has left the vehicle at the geographical location, determining whether position information retrieved from the position sensor to identify the geographical location meets an accuracy criterion. The method further includes, when it is determined that the position information does not meet the accuracy criterion, providing a prompt to the user to input information regarding the geographical location. The method also includes receiving, in response to the prompt, information about the geographical location from the user and storing this information as vehicle position information.

[0170] (F2)In some embodiments of the method described in F1, the method further includes, when it is determined that the position information meets the accuracy criterion, automatically and without an instruction from a user, storing this position information as vehicle position information.

[0171] (F3)In some embodiments of the method described in claim F2, the method further includes displaying a user interface object including the vehicle position information in accordance with a determination that the user is heading towards the geographical location.

[0172] (F4)In some embodiments of the method described in claim F3, the user interface object is a map object including an identifier regarding the user's current position and another identifier regarding the geographical location.

[0173] (F5)In some embodiments of the method described in claim F4, the user interface object is displayed on a lock screen of the electronic device.

[0174] (F6) In some embodiments of the method according to claim F4, the user interface object is displayed in response to a swipe gesture that causes the electronic device to enter a search mode.

[0175] (F7) In some embodiments of the method according to claim F6, determining whether the user is heading towards a geographical location is performed in response to receiving the swipe gesture.

[0176] (F8) In some embodiments of the method according to any one of F1 to F7, the prompt is an audio prompt provided by a virtual assistant available via the electronic device, receiving information from the user includes receiving an oral description from the user to identify the geographical location, and displaying the user interface object includes displaying a selectable affordance that, when selected, causes the device to play the oral description.

[0177] (F9) In some embodiments of the method according to any one of F1 to F7, the prompt is displayed on the display of the electronic device, receiving information from the user includes receiving a text description from the user to identify the geographical location, and displaying the user interface object includes displaying the text description from the user.

[0178] (F10) In some embodiments of the method according to any one of F1 to F7, determining whether the user is heading towards a geographical location includes using new location information received from a location sensor to determine that the electronic device is moving towards the geographical location.

[0179] In some embodiments of the method described in (F11) F10, determining whether a user is heading towards a geographical location includes: (i) determining that the electronic device has remained at a different geographical location for a threshold time; and (ii) determining that new location information indicates that the electronic device is leaving a different geographical location and heading towards the geographical location.

[0180] (F12) In some embodiments of the method described in any one of F1 to F11, determining that a user is in a vehicle stopped at a geographical location includes: (i) determining that the user is in the vehicle by determining that the electronic device is moving at a speed exceeding a threshold speed; and (ii) determining that the vehicle has stopped at the geographical location by one or more of: (a) determining that the electronic device has remained at the geographical location for a threshold time; (b) determining that the communication link between the electronic device and the vehicle has been disconnected; and (c) determining that the geographical location corresponds to a location within a parking lot.

[0181] (F13) In some embodiments of the method described in claim F12, determining that the vehicle has stopped at the geographical location includes determining that the electronic device has remained at the geographical location for a threshold time.

[0182] (F14) In some embodiments of the method described in F12 or F13, determining that the vehicle has stopped at the geographical location includes determining that the communication link between the electronic device and the vehicle has been disconnected.

[0183] (F15) In some embodiments of the method described in any one of F12 to F14, determining that the vehicle has stopped at the geographical location includes determining that the geographical location corresponds to a location within a parking lot.

[0184] (F16) In some embodiments of the method described in any one of F1 to F15, the accuracy criterion includes a criterion that is satisfied when the accuracy of the GPS reading associated with the location information exceeds an accuracy threshold level.

[0185] (F17)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensing surface, a display, a position sensor, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of F1 to F16.

[0186] (F18)In yet another aspect, an electronic device is provided, the electronic device including a touch-sensing surface, a display, a position sensor, and means for executing the method described in any one of F1 to F16.

[0187] (F19)In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch-sensing surface, a display, and a position sensor, cause the electronic device to execute the method described in any one of F1 to F16.

[0188] (F20)In a further aspect, a graphical user interface on an electronic device having a touch-sensing surface, a display, and a position sensor is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of F1 to F16. In a further aspect, an information processing apparatus for use in an electronic device including a touch-sensing surface, a display, and a position sensor is provided. The information processing apparatus includes means for executing the method described in any one of F1 to F16.

[0189] (F21) In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 4601, FIG. 46), a touch sensing surface unit (e.g., touch sensing surface unit 4603, FIG. 46), a position sensor unit (e.g., position sensor unit 4607, FIG. 46), and a processing unit (e.g., processing unit 4605, FIG. 46). The processing unit is coupled to the touch sensing surface unit, the display unit, and the position sensor unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 46 shows the display unit 4601 and the touch sensing surface unit 4603 as being integrated into the electronic device 4600, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. In some embodiments, the touch sensing surface unit and the display unit are integrated into one touch sensing display unit (also referred to herein as a touch sensing display). The processing unit includes a display unit (e.g., display unit 4609, FIG. 46), an extraction unit (e.g., extraction unit 4611, FIG. 46), a determination unit (e.g., determination unit 4613, FIG. 46), a memory unit (e.g., memory unit 4615, FIG. 46), an identification unit (e.g., identification unit 4617, FIG. 46), a selection unit (e.g., selection unit 4619, FIG. 46), a receiving unit (e.g., receiving unit 4621, FIG. 46), a providing unit (e.g., providing unit 4623, FIG. 46), and a playback unit (e.g., playback unit 4625, FIG. 46).The processing unit (or one or more components thereof such as units 4607 to 4625) automatically determines, without an instruction from the user, that the user of the electronic device is in a stopped vehicle at a geographical location (e.g., by determination unit 4613). When it is determined that the user has left the vehicle at the geographical location, it determines (e.g., by determination unit 4613) whether the location information retrieved (e.g., by retrieval unit 4611) from the position sensor unit (e.g., position sensor unit 4607) to identify the geographical location (e.g., by identification unit 4617) meets the accuracy criteria. When it is determined that the location information does not meet the accuracy criteria (e.g., by determination unit 4613), it provides (e.g., by providing unit 4623) a prompt for the user to input information regarding the geographical location. In response to the provision of the prompt, it receives (e.g., by receiving unit 4621) information about the geographical location from the user and is configured to store (e.g., by storage unit 4615) this information as vehicle location information.

[0190] (F22)In some embodiments of the electronic device described in F21, when the processing unit determines that the location information meets the accuracy criteria, it is further configured to automatically store (e.g., by storage unit 4615) this location information as vehicle location information without an instruction from the user.

[0191] (F23)In some embodiments of the electronic device described in F22, the processing unit is further configured to display (e.g., by display unit 4609 in conjunction with display unit 4601) a user interface object including the vehicle location information in accordance with the determination that the user is heading towards the geographical location.

[0192] (F24)In some embodiments of the electronic device described in F23, the user interface object is a map object including an identifier regarding the user's current location and another identifier regarding the geographical location.

[0193] (F25)In some embodiments of the electronic device described in F24, the user interface object is displayed on the lock screen of the electronic device (e.g., by display unit 4609 in conjunction with display unit 4601).

[0194] (F26)In some embodiments of the electronic device described in F24, the user interface object is displayed in response to a swipe gesture that causes the electronic device to transition to a search mode (e.g., by display unit 4609 in conjunction with display unit 4601).

[0195] (F27)In some embodiments of the electronic device described in F26, determining whether the user is heading towards a geographical location is performed in response to receiving a swipe gesture.

[0196] (F28)In some embodiments of the electronic device described in any one of F21 to F27, the prompt is an audio prompt provided by a virtual assistant available via the electronic device, receiving information from the user includes receiving an oral description from the user to identify a geographical location (e.g., by receiving unit 4621), and displaying the user interface object includes displaying, when selected (e.g., by selection unit 4619), a selectable affordance that causes the device to reproduce the oral description (e.g., by playback unit 4625) (e.g., by display unit 4609 in conjunction with display unit 4601).

[0197] In some embodiments of the electronic device according to any one of F21 to F27, the prompt is displayed on the display of the electronic device (e.g., by the display unit 4609 in conjunction with the display unit 4601), and receiving information from the user includes receiving a textual description identifying a geographical location from the user (e.g., by the receiving unit 4621), and displaying a user interface object includes displaying a textual description from the user.

[0198] In some embodiments of the electronic device according to any one of F21 to F27, determining whether the user is heading towards a geographical location includes using new location information received from a location sensor unit (e.g., the location sensor unit 4607) (e.g., by the receiving unit 4621) to determine that the electronic device is moving towards the geographical location (e.g., by the determination unit 4613).

[0199] In some embodiments of the electronic device according to F30, determining whether the user is heading towards a geographical location includes (i) determining that the electronic device has remained at a different geographical location for a threshold time (e.g., by the determination unit 4613), and (ii) determining that the new location information indicates that the electronic device is leaving a different geographical location and heading towards the geographical location (e.g., by the determination unit 4613).

[0200] In some embodiments of the electronic device described in any one of F21 to F31 (F32), determining that a user is in a vehicle stopped at a geographical location includes: (i) determining that the user is in the vehicle by determining that the electronic device is moving at a speed exceeding a threshold speed (e.g., by determination unit 4613), and (ii) (a) determining that the electronic device has remained at the geographical location for a time exceeding a threshold time (e.g., by determination unit 4613), (b) determining that the communication link between the electronic device and the vehicle has been disconnected (e.g., by determination unit 4613), and (c) determining that the geographical location corresponds to a location within a parking lot (e.g., by determination unit 4613), and determining that the vehicle has stopped at the geographical location by one or more of the above.

[0201] (F33) In some embodiments of the electronic device described in F32, determining that the vehicle has stopped at a geographical location includes determining that the electronic device has remained at the geographical location for a time exceeding a threshold time (e.g., by determination unit 4613).

[0202] (F34) In some embodiments of the electronic device described in F32 or F33, determining that the vehicle has stopped at a geographical location includes determining that the communication link between the electronic device and the vehicle has been disconnected (e.g., by determination unit 4613).

[0203] (F35) In some embodiments of the electronic device described in any one of F32 to F34, determining that the vehicle has stopped at a geographical location includes determining that the geographical location corresponds to a location within a parking lot (e.g., by determination unit 4613).

[0204] (F36) In some embodiments of the electronic device described in any one of F21 to F35, the accuracy criterion includes a criterion that is satisfied when the accuracy of the GPS reading associated with the location information exceeds an accuracy threshold level.

[0205] (G1) In some embodiments, a method is performed in an electronic device (e.g., a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E)) having a position sensor, a touch sensing surface (e.g., touch sensing surface 195, FIG. 1D), and a display (e.g., display 194, FIG. 1D). The method includes monitoring the geographical location of the electronic device using the position sensor. The method further includes determining, based on the monitored geographical location, that the electronic device is within a threshold distance of a predefined type of target location. The method also includes identifying at least one currently popular activity at the target location and retrieving information about the target location, including retrieving information about at least one currently popular activity at the target location, in accordance with determining that the electronic device is within the threshold distance of the target location. The method further includes detecting, via the touch sensing surface, a first input that causes the electronic device to transition to a search mode when detected, and in response to detecting the first input, transitioning to the search mode, where transitioning to the search mode includes presenting, via the display, an affordance that includes (i) information about at least one activity and (ii) an indication that at least one activity is identified as currently popular at the target location, before receiving any user input in the search interface.

[0206] (G2) In some embodiments of the method of claim G1, the method includes detecting a second input and, in response to detecting the second input, updating the affordance to include available information about current activities at a second target location different from the target location, where the target location is within the threshold distance range of the electronic device.

[0207] (G3)In some embodiments of the method described in G1 or G2, the affordance further includes selectable categories of target locations, and the method further includes detecting a selection of a corresponding selectable category and updating the affordance to include information about additional target locations located within a second threshold distance range of the device in response to detecting the selection.

[0208] (G4)In some embodiments of the method described in any one of G1 to G3, the target location is an amusement park, and the information retrieved includes the current waiting time of the rides in the amusement park.

[0209] (G5)In some embodiments of the method described in claim G4, the information retrieved includes information about the waiting time of rides located within a predefined distance range of the electronic device.

[0210] (G6)In some embodiments of the method described in any one of G1 to G3, the target location is a restaurant, and the information retrieved includes information about the popular menu items in the restaurant.

[0211] (G7)In some embodiments of the method described in claim G6, the information retrieved is retrieved from a social network associated with the user of the electronic device.

[0212] (G8)In some embodiments of the method described in any one of G1 to G3, the target location is a movie theater, and the information retrieved includes information about the showtimes in the movie theater.

[0213] (G9)In some embodiments of the method described in claim G8, the information retrieved is retrieved from a social network associated with the user of the electronic device.

[0214] (G10)In some embodiments of the method described in any one of G1 to G9, after unlocking the electronic device, the affordance is available in response to a substantially horizontal swipe on the initial page of the home screen of the electronic device.

[0215] (G11) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch sensing surface, a display, a position sensor, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of G1 to G10.

[0216] (G12) In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, a position sensor, and means for executing the method described in any one of G1 to G10.

[0217] (G13) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface, a display, and a position sensor, cause the electronic device to execute the method described in any one of G1 to G10.

[0218] (G14) In a further aspect, a graphical user interface on an electronic device having a touch sensing surface, a display, and a position sensor is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of G1 to G10. In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface, a display, and a position sensor is provided. The information processing apparatus includes means for executing the method described in any one of G1 to G10.

[0219] (G15) In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 4701, FIG. 47), a touch sensing surface unit (e.g., touch sensing surface unit 4703, FIG. 47), a position sensor unit 4707, and a processing unit (e.g., processing unit 4705, FIG. 47). The processing unit is coupled to the touch sensing surface unit, the display unit, and the position sensor unit. In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (i.e., computing devices A through D). For ease of explanation, FIG. 47 shows the display unit 4701 and the touch sensing surface unit 4703 as being integrated into the electronic device 4700, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. In some embodiments, the touch sensing surface unit and the display unit are integrated into one touch sensing display unit (also referred to herein as a touch sensing display). The processing unit includes a detection unit (e.g., detection unit 4709, FIG. 47), a display unit (e.g., display unit 4711, FIG. 47), an extraction unit (e.g., extraction unit 4713, FIG. 47), a determination unit (e.g., determination unit 4715, FIG. 47), an identification unit (e.g., identification unit 4717, FIG. 47), an unlocking unit (e.g., unlocking unit 4719, FIG. 47), and a search mode transition unit (e.g., search mode transition unit 4721, FIG. 47).The processing unit (or one or more components thereof such as units 4709 to 4721) monitors the geographical location of the electronic device using a location sensor unit (e.g., location sensor unit 4707) without receiving any commands from the user of the electronic device, and based on the monitored geographical location, determines that the electronic device is within a threshold distance of a target location of a predefined type (e.g., by determination unit 4715). In accordance with the determination that the electronic device is within the threshold distance of the target location, it identifies at least one currently popular activity at the target location (e.g., by identification unit 4717), retrieves information regarding at least one currently popular activity at the target location, including retrieving information regarding the target location (e.g., by retrieval unit 4713), detects a first input (e.g., by detection unit 4709) that causes the electronic device to transition to a search mode when detected via a touch-sensitive surface unit (e.g., touch-sensitive surface unit 4703), and is configured to transition to the search mode (e.g., by search mode transition unit 4721). Transitioning to the search mode includes presenting an affordance (e.g., by display unit 4711) that includes (i) information regarding at least one activity and (ii) an indication that at least one activity has been identified as currently popular at the target location via a display unit (e.g., display unit 4701) before receiving any user input at the search interface.

[0220] (G16) In some embodiments of the electronic device described in G15, the processing unit further detects a second input (e.g., by detection unit 4709) and, in response to detecting the second input, updates the affordance to include available information regarding the current activity at a second target location different from the target location (e.g., by display unit 4711), where the target location is within the threshold distance range of the electronic device.

[0221] In some embodiments of the electronic device described in (G17) G15 or G16, the affordance further includes selectable categories of target locations, and the processing unit detects the selection of a corresponding selectable category (e.g., by the detection unit 4709), and in response to detecting the selection, updates the affordance to include information about additional target locations located within a second threshold distance range of the device (e.g., by the display unit 4711).

[0222] (G18) In some embodiments of the electronic device described in any one of G15 to G17, the target location is an amusement park, and the information retrieved includes the current waiting time of the rides in the amusement park.

[0223] (G19) In some embodiments of the electronic device described in G18, the information retrieved includes information about the waiting time of rides located within a predefined distance range of the electronic device.

[0224] (G20) In some embodiments of the electronic device described in any one of G15 to G17, the target location is a restaurant, and the information retrieved includes information about the popular menu items in the restaurant.

[0225] (G21) In some embodiments of the electronic device described in G20, the information retrieved is retrieved from a social network associated with the user of the electronic device.

[0226] (G22) In some embodiments of the electronic device described in any one of G15 to G17, the target location is a movie theater, and the information retrieved includes information about the screening times in the movie theater.

[0227] (G23) In some embodiments of the electronic device described in G22, the information retrieved is retrieved from a social network associated with the user of the electronic device.

[0228] In some embodiments of the electronic device according to any one of G15 to G23 in (G24), after unlocking the electronic device (e.g., by the unlocking unit 4719), the affordance is available in response to a substantially horizontal swipe on the initial page of the home screen of the electronic device.

[0229] (H1)In some embodiments, a method is performed in an electronic device (e.g., a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E)) having a touch - sensing surface and a display (a touch screen 112, which in some embodiments is integrated with the touch - sensing surface, as shown in FIG. 1C)). The method includes receiving at least a portion of a voice communication (e.g., a live call or recorded voicemail of 10 seconds or less), where a portion of the voice communication includes speech provided by a remote user of a remote device different from the user of the electronic device. The method also includes extracting content items, at least in part, based on speech provided by a remote user of a remote device. The method further includes determining whether the content items are currently available on the electronic device. In accordance with a determination that the content items are not currently available on the electronic device, the method includes (i) identifying an application associated with the content items and (ii) displaying on the display a selectable description of the content items. In response to detecting a selection of the selectable description, the method includes storing the content items for presentation by the identified application (e.g., storing the content items so that they are available for presentation by the identified application). In this way, the user can store content items mentioned or spoken about in a voice communication, without having to remember all of the "details" spoken and later enter these details to create the appropriate content items. Instead, the electronic device detects and extracts content items based on speech describing various corresponding content items, then provides selectable descriptions of the content items that the user can select, and can store the corresponding content items in the electronic device.

[0230] (H2)In some embodiments of the method described in H1, the content items are new events.

[0231] (H3) In some embodiments of the method described in H1, the content item is new event details of an event currently associated with a calendar application on an electronic device.

[0232] (H4) In some embodiments of the method described in H1, the content item is a new contact.

[0233] (H5) In some embodiments of the method described in H1, the content item is new contact information of an existing contact associated with a phone application on an electronic device.

[0234] (H6) In some embodiments of the method described in any one of H1 to H5, the voice communication is a live call.

[0235] (H7) In some embodiments of the method described in any one of H1 to H5, the voice communication is a live FaceTime call.

[0236] (H8) In some embodiments of the method described in any one of H1 to H5, the voice communication is a recorded voicemail.

[0237] (H9) In some embodiments of the method described in any one of H1 to H8, displaying a selectable explanation includes displaying a selectable explanation within a user interface that includes recent calls made using a phone application. In this way, the user can easily and conveniently access content items (e.g., those extracted during the corresponding voice communication) directly from the user interface, including recent calls.

[0238] (H10) In some embodiments of the method described in H9, the selectable explanation is displayed with an indication that the content item is associated with a voice communication.

[0239] In some embodiments of the method described in (H11)H9, detecting a selection includes receiving the selection while a user interface including a recent call is being displayed.

[0240] (H12)In some embodiments of the method described in any one of H1 to H11, the method further includes providing feedback to the user of the electronic device that a content item has been detected, in conjunction with displaying a selectable description of the content item (for example, haptic feedback generated by the electronic device or presentation of a user interface object on a second device. Thereby, the user does not need to move the phone away from their ear during the call). In this way, a concise indication that a content item has been detected / extracted is provided to the user during a voice communication, and then the user can decide whether to remember the content item.

[0241] (H13)In some embodiments of the method described in H12, providing feedback includes transmitting information regarding the detection of the content item to a different electronic device in the vicinity of the electronic device (for example, a laptop computer, a television monitor, a smartwatch, etc.). In this way, the user does not need to interrupt the voice communication, yet can view details regarding the detected / extracted content item on a different device.

[0242] (H14)In some embodiments of the method described in any one of H1 to H13, the method further includes determining that a voice communication includes information regarding a first physical location (e.g., an address mentioned during a call or a restaurant name spoken during a call, etc. Further details will be described below). The method also includes detecting an input and opening an application capable of receiving location data in response to detecting the input, and populating the application with the information regarding the first physical location. In this way, in addition to detecting and extracting events and contact information, the electronic device can extract location information spoken in a voice communication and provide this location information to the user in an appropriate application (e.g., for this purpose, without burdening the user with the task of remembering specific location details spoken during a call, especially new details that the user may not be familiar with, and the device extracts these location details and displays them for the user to use).

[0243] (H15)In some embodiments of the method described in H14, the application is a map application, and populating the map application with the information regarding the first physical location includes populating the map displayed within the map application with a location identifier corresponding to the first physical location. In this way, the user can easily use the newly extracted location details, move to a new destination, view how far a particular location is, and easily use other functions provided by the map application.

[0244] (H16)In some embodiments of the method according to any one of H1 to H13, the method further includes determining that the voice communication includes information regarding a first physical location. The method also includes detecting an input (e.g., a search activation gesture such as a swipe gesture, described in detail below), and populating a search interface with information regarding the first physical location in response to detecting the input. In this way, in addition to (or instead of) providing location information to the user for use in a particular application, the electronic device can also provide the location information for use in a search interface (e.g., to search for relevant locations or to search for additional details regarding the first physical location such as phone numbers, menus, etc.).

[0245] (H17)In some embodiments of the method according to any one of H1 to H16, extracting a content item includes analyzing a portion of the voice communication for detection of a predefined type of content, and the analysis is performed while the voice communication is output via an audio system communicating with the electronic device (e.g., the voice communication is analyzed in real time while the voice communication is being output to the user of the electronic device).

[0246] (H18)In some embodiments of the method according to H17, analyzing the voice communication includes (i) converting speech provided by a remote user of a remote device to text, (ii) applying a natural language processing algorithm to the text to determine whether the text includes one or more predefined keywords, and (iii) determining that the voice communication includes speech describing a content item according to the determination that the text includes the corresponding predefined keyword(s).

[0247] In some embodiments of the method according to any one of (H1) to (H18), receiving at least a portion of the voice communication includes receiving from a user of the electronic device an indication that a portion of the voice communication should be analyzed.

[0248] In some embodiments of the method according to (H19), the indication corresponds to the selection of a hardware button (e.g., the user selects a hardware button while the voice communication is being output by the audio system, indicating that a voice communication for a predetermined number of seconds (e.g., the previous 10, 15, or 20 seconds) should be analyzed). In some embodiments, the button may also be a button presented on the display of the electronic device for user selection (e.g., a button displayed during the voice communication that says "tap here to analyze new content of this voice communication").

[0249] In some embodiments of the method according to (H19), the indication corresponds to a command from the user of the electronic device that includes the phrase "hey Siri". Thus, the user can easily instruct the electronic device to start analyzing a portion of the voice communication and detect content items (such as events, contact information, and information regarding physical location) spoken in the voice communication.

[0250] (H22) In some embodiments of the method according to any one of H1 to H21, the method further includes receiving a second portion of the voice communication. The second portion includes speech provided by a remote user of the remote device and speech provided by a user of the electronic device (e.g., the voice communication is a live call and the second portion includes a conversation between the user and the remote user). The method also includes extracting a second content item, at least in part, based on speech provided by a remote user of the remote device and speech provided by a user of the electronic device. The method includes, in accordance with a determination that the second content item is not currently available on the electronic device, (i) identifying a second application associated with the second content item, and (ii) displaying a second selectable description of the second content item on the display. The method includes storing the second content item for presentation by the identified second application in response to detecting a selection of the second selectable description.

[0251] (H23) In some embodiments of the method according to H22, selectable descriptions and a second selectable description, including a most recent call made using the phone application, are displayed within the user interface. In this way, a user is provided with one interface that conveniently includes content items detected in several voice communications (e.g., several calls, voicemails, or calls and voicemails).

[0252] (H24) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to perform the method according to any one of H1 to H23.

[0253] (H25)In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, and means for performing the method according to any one of H1 to H23.

[0254] (H26)In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface and a display, cause the electronic device to perform the method according to any one of H1 to H23.

[0255] (H27)In a further aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method according to any one of H1 to H23.

[0256] (H28)In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for performing the method according to any one of H1 to H23.

[0257] In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 4801, FIG. 48), a touch sensing surface unit (e.g., touch sensing surface unit 4803, FIG. 48), and a processing unit (e.g., processing unit 4805, FIG. 48). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 48 shows display unit 4801 and touch sensing surface unit 4803 as being integrated into electronic device 4800, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes an audio communication receiving unit (e.g., audio communication receiving unit 4807, FIG. 48), a content item extraction unit (e.g., content item extraction unit 4809, FIG. 48), an availability determination unit (e.g., availability determination unit 4811, FIG. 48), an application identification unit (e.g., application identification unit 4813, FIG. 48), a display unit (e.g., display unit 4815, FIG. 48), a content item storage unit (e.g., content item storage unit 4817, FIG. 48), a feedback providing unit (e.g., feedback providing unit 4819, FIG. 48), an input detection unit (e.g., input detection unit 4821, FIG. 48), an application opening unit (e.g., application opening unit 4823, FIG. 48), a populating unit (e.g., populating unit 4825, FIG. 48), and an audio communication analysis unit (e.g., audio communication analysis unit 4827, FIG. 48). The processing unit (or one or more of its components such as units 4807 through 4827) is configured to receive at least a portion of an audio communication (e.g., by audio communication receiving unit 4807), and a portion of the audio communication includes speech provided by a remote user of a remote device different from the user of the electronic device.The processing unit is further configured to extract at least a part of the content item based on speech provided by a remote user of a remote device (e.g., by the content item extraction unit 4809), and to determine whether the content item is currently available on the electronic device (e.g., by the availability determination unit 4811). In accordance with the determination that the content item is not currently available on the electronic device, the processing unit is further configured to (i) identify the application associated with the content item (e.g., by the application identification unit 4813), and (ii) cause a selectable description of the content item to be displayed on the display (e.g., by the display unit 4815 and / or the display unit 4801). In response to detection of selection of the selectable description (e.g., by the input detection unit 4821 and / or the touch sensing surface unit 4803), the processing unit is configured to store the content item for presentation by the identified application (e.g., by the content item storage unit 4817).

[0258] (H30)In some embodiments of the electronic device described in H29, the processing unit (or one or more of its components such as units 4907 to 4927) is further configured to execute the method described in any one of H2 to H23.

[0259] (I1) In some embodiments, a method is performed in an electronic device having a touch-sensing surface and a display (in some embodiments, a touch screen 112, where the touch-sensing surface and the display are integrated as shown in FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E). The method includes receiving at least a portion of a voice communication, where a portion of the voice communication (e.g., a live call, a recorded voicemail) includes speech provided by a remote user of a remote device different from the user of the electronic device. The method also includes determining that the voice communication includes speech identifying a physical location. In response to determining that the voice communication includes speech identifying a physical location, the method includes providing an indication that information regarding the physical location has been detected (e.g., providing haptic feedback to the user, displaying a user interface object along with information regarding the physical location, or transmitting information regarding the physical location to a nearby device for display thereon). The method further includes detecting an input via the touch-sensing surface. In response to detecting the input, the method includes (i) opening an application that receives geographical location data and (ii) populating the application with information regarding the physical location. In this way, the user can remember information regarding the physical location mentioned or spoken about in the voice communication, remember all of the details spoken, and need not later enter these details into the appropriate application. Instead, the electronic device can detect and extract information regarding the physical location based on speech that describes the physical location (e.g., a description of a restaurant, driving directions to the physical location, etc.) and then provide an indication that the corresponding information regarding the physical location has been detected.

[0260] (I2) In some embodiments of the method described in I1, the voice communication is a live call.

[0261] (I3)In some embodiments of the method described in I1, the voice communication is a live FaceTime call.

[0262] (I4)In some embodiments of the method described in I1, the voice communication is a recorded voicemail.

[0263] (I5)In some embodiments of the method described in any one of I1 to I4, providing the indication includes displaying a selectable description of the physical location, including the most recent call made using the phone application, within the user interface.

[0264] (I6)In some embodiments of the method described in I5, the selectable description indicates that the content item is associated with a voice communication.

[0265] (I7)In some embodiments of the method described in I5 or I6, detecting the input includes detecting an input on the selectable description while the user interface including the most recent call is being displayed.

[0266] (I8)In some embodiments of the method described in any one of I1 to I7, providing the indication includes providing haptic feedback to the user of the electronic device.

[0267] (I9)In some embodiments of the method described in any one of I1 to I8, providing the indication includes transmitting information regarding the physical location to a different electronic device in the vicinity of the electronic device.

[0268] (I10)In some embodiments of the method described in any one of I1 to I9, determining that the voice communication includes speech that describes a physical location includes analyzing a portion of the voice communication to detect information regarding the physical location, and the analysis is performed while the voice communication is being output via an audio system that is communicating with the electronic device.

[0269] (I11) In some embodiments of the method described in any one of I1 to I10, receiving at least a portion of a voice communication includes receiving from a user of the electronic device an instruction that a portion of the voice communication should be analyzed.

[0270] (I12) In some embodiments of the method described in I11, the instruction corresponds to the selection of a hardware button. In some embodiments, the button may also be a button presented on the display of the electronic device for user selection (e.g., a button that says "Tap here to analyze new content of this voice communication" displayed during a voice communication).

[0271] (I13) In some embodiments of the method described in I11, the instruction corresponds to a command from the user of the electronic device that includes the phrase "Hey Siri".

[0272] (I14) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to perform the method described in any one of I1 to I13.

[0273] (I15) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensitive surface, a display, and means for performing the method described in any one of I1 to I13.

[0274] (I16) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to perform the method described in any one of I1 to I13.

[0275] (I17) In yet another aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of I1 to I13.

[0276] (I18) In yet another aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for executing the method described in any one of I1 to I13.

[0277] (I19) In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 4901, FIG. 49), a touch sensing surface unit (e.g., touch sensing surface unit 4903, FIG. 49), and a processing unit (e.g., processing unit 4905, FIG. 49). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 49 shows display unit 4901 and touch sensing surface unit 4903 as being integrated into electronic device 4900, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes an audio communication receiving unit (e.g., audio communication receiving unit 4907, FIG. 49), a content item extraction unit (e.g., content item extraction unit 4909, FIG. 49), an indication providing unit (e.g., indication providing unit 4911, FIG. 49), an input detection unit (e.g., input detection unit 4913, FIG. 49), an application opening unit (e.g., application opening unit 4915, FIG. 49), an application populating unit (e.g., application populating unit 4917, FIG. 49), a feedback providing unit (e.g., feedback providing unit 4919, FIG. 49), and an audio communication analysis unit (e.g., audio communication analysis unit 4921, FIG. 49). The processing unit (or one or more of its components such as units 4907 through 4921) is configured to receive at least a portion of an audio communication, and a portion of the audio communication includes speech provided by a remote user of a remote device different from the user of the electronic device (e.g., by audio communication receiving unit 4907). The processing unit is further configured to determine (e.g., by content item extraction unit 4909) that the audio communication includes speech that identifies a physical location.In response to determining that the voice communication includes speech that identifies a physical location, the processing unit is configured to provide an indication that information regarding the physical location has been detected (e.g., by content item extraction unit 4909). The processing unit is also configured to detect an input via the touch sensing surface unit (e.g., by input detection unit 4911). In response to detecting the input, the processing unit is configured to (i) open an application that receives geographical location data (e.g., by application open unit 4913), and (ii) populate the application with information regarding the physical location (e.g., by application populating unit 4915).

[0278] (I20)In some embodiments of the electronic device described in I19, the processing unit (or one or more of its components such as units 4907 to 4921) is further configured to execute the method described in any one of I2 to I13.

[0279] (J1)In some embodiments, a method is performed in an electronic device having a touch sensing surface and a display (in some embodiments, a touch screen 112, where the touch sensing surface and the display are integrated as shown in FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E). The method includes presenting a text input field and a conversation record in a messaging application on the display. The method also includes determining that the next possible input from a user of the electronic device is information regarding a physical location while the messaging application is presented on the display. The method further includes analyzing content associated with the text input field and the conversation record to determine a proposed physical location, at least in part based on a portion of the analyzed content. The method further includes presenting a selectable user interface element within the messaging application on the display that identifies the proposed physical location, and receiving a selection of the selectable user interface element. The method includes presenting, in response to receiving the selection, a representation of the proposed physical location within the text input field. In this way, the content needed by the user of the electronic device is conveniently provided, and the user does not need to type anything or search for content (e.g., the user can simply select a selectable user interface element to enter their current location, access a map application to reveal their exact location, return to the messaging application, and does not need to make an explicit input to send the location information).

[0280] (J2)In some embodiments of the method of J1, the messaging application includes a virtual keyboard, and the selectable user interface element is displayed in a proposed section adjacent to and above the virtual keyboard.

[0281] (J3) In some embodiments of the method described in J1 or J2, determining that the next possible input from a user of the electronic device is information regarding a physical location involves processing text input fields and content associated with a conversation record to detect that the conversation transcript contains a question regarding the user's current location. In this way, a proposed physical location directly related to the description in the conversation transcript is provided to the user (e.g., in response to a second user's question of "Where are you?", a user interface object is presented to the user that, if selected, causes the device to send information regarding the user's current location to the second user).

[0282] (J4) In some embodiments of the method described in J3, processing the content involves applying a natural language processing algorithm to detect one or more defined keywords that form a question (e.g., "Where are you?" or "What is your address?").

[0283] (J5) In some embodiments of the method described in J3 or J4, the question is included in a message received from a second user different from the user.

[0284] (J6) In some embodiments of the method described in any one of J1 to J5, determining that the next possible input from a user of the electronic device is information regarding a physical location involves monitoring typing input received from the user in the text input portion of a messaging application.

[0285] (J7) In some embodiments of the method described in any one of J1 to J6, the method further includes ceasing to present selectable user interface elements in accordance with a determination that the user is typing and has not selected a selectable user interface element. In this way, if it is determined that the user is not interested in selecting an object, the device does not continue to present selectable user interface objects.

[0286] (J8) In some embodiments of the method according to any one of J1 to J7, the method further includes stopping presenting selectable user interface elements according to a determination that the user has provided additional input indicating that the user does not select any of the selectable user interface elements. In this way, when it can be determined that the user is not interested in selecting an object, the device will not continue to present selectable user interface objects.

[0287] (J9) In some embodiments of the method according to any one of J1 to J5, the proposed representation of the physical location includes information identifying the current geographical location of the electronic device.

[0288] (J10) In some embodiments of the method according to any one of J1 to J9, the proposed representation of the physical location is an address.

[0289] (J11) In some embodiments of the method according to any one of J1 to J9, the proposed physical location is a map object including an identifier for the proposed physical location.

[0290] (J12) In some embodiments of the method according to any one of J1 to J11, the proposed physical location corresponds to a location that the user has recently viewed in an application other than the messaging application.

[0291] (J13) In some embodiments of the method according to any one of J1 to J12, the messaging application is an email application.

[0292] (J14) In some embodiments of the method according to any one of J1 to J12, the messaging application is a text messaging application.

[0293] (J15)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch sensing surface, a display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of J1 to J14.

[0294] (J16)In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, and means for executing the method described in any one of J1 to J14.

[0295] (J17)In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface and a display, cause the electronic device to execute the method described in any one of J1 to J14.

[0296] (J18)In a further aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of J1 to J14.

[0297] (J19)In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for executing the method described in any one of J1 to J14.

[0298] In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 5001, FIG. 50), a touch sensing surface unit (e.g., touch sensing surface unit 5003, FIG. 50), and a processing unit (e.g., processing unit 5005, FIG. 50). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 50 shows the display unit 5001 and the touch sensing surface unit 5003 as being integrated into the electronic device 5000, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5007, FIG. 50), a next input determination unit (e.g., next input determination unit 5009, FIG. 50), a content analysis unit (e.g., content analysis unit 5011, FIG. 50), a selection reception unit (e.g., selection reception unit 5013, FIG. 50), a typing input monitoring unit (e.g., typing input monitoring unit 5015, FIG. 50), and a presentation end unit (e.g., presentation end unit 5017, FIG. 50). The processing unit (or one or more of its components such as units 5007 through 5017) is configured to present a text input field and a conversation record in a message application on the display (e.g., by presentation unit 5007 and / or display unit 5001). While the message application is presented on the display, the processing unit is also configured to determine (e.g., by next input determination unit 5009) that the next possible input from a user of the electronic device is information regarding a physical location.The processing unit analyzes the text input field and the content associated with the conversation record, determines the proposed physical location (e.g., by the content analysis unit 5011) based on at least a part of the analyzed content, presents a selectable user interface element that specifies the proposed physical location within a message application on the display (e.g., by the presentation unit 5007), receives the selection of the selectable user interface element (e.g., by the selection receiving unit 5013 and / or the touch sensing surface unit 5003), and is further configured to present an expression of the proposed physical location within the text input field (e.g., by the presentation unit 5007) in response to receiving the selection.

[0299] (J21)In some embodiments of the electronic device described in J20, the processing unit (or one or more of its components such as units 5007 to 5017) is further configured to execute the method described in any one of J2 to J14.

[0300] (K1)In some embodiments, the method is performed on an electronic device having a touch sensing surface and a display (in some embodiments, a touch screen 112, where the touch sensing surface and the display are integrated as shown in FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E). The method includes obtaining information identifying a first physical location (e.g., a restaurant a user searched for in an application enabling search for local businesses) that the user browsed in a first application while the first application was being displayed. The method also includes ending the first application and, after ending the first application, receiving from the user a request to open a second application different from the first application. In some embodiments, the request is received without receiving any input in the first application (e.g., the request does not include clicking a link or button within the first application). In response to receiving the request and according to a determination that the second application is capable of receiving geographical location information, the method includes presenting the second application, which includes populating the second application, at least in part, with information based on information identifying the first physical location. In this way, the user does not need to manually transfer information between two different applications. Instead, the device intelligently determines that the second application is capable of receiving geographical location information and then populates the second application directly with information regarding the physical location browsed in the first application (e.g., populates the second application with a map object that includes an identifier for the physical location).

[0301] (K2)In some embodiments of the method described in K1, receiving a request to open a second application includes detecting an input on an affordance of the second application after the first application has ended. In other words, the request does not correspond to clicking a link within the first application. Instead, the user explicitly and directly requests to open the second application, and then the device determines to populate the second application with information regarding the previously viewed (previously viewed in a different first application) physical location so that the user can further investigate or consider this previously viewed physical location in the second application.

[0302] (K3)In some embodiments of the method described in K2, the affordance of the second application is an icon displayed within the home screen of the electronic device. In some embodiments, the home screen is a system-level component of the operating system and includes icons for calling applications available on the electronic device.

[0303] (K4)In some embodiments of the method described in K2, detecting an input includes: (i) detecting a double-tap on a physical home button; (ii) displaying an application switch user interface in response to detecting the double-tap; and (iii) detecting a selection of an affordance from within the application switch user interface.

[0304] (K5)In some embodiments of the method described in any one of K1-K4, populating the second application includes displaying a user interface object that includes information based at least in part on information identifying the first physical location.

[0305] (K6) In some embodiments of the method described in K5, the user interface object includes a textual description informing the user of the most recently viewed first physical location in the first application.

[0306] (K7) In some embodiments of the method described in K6, the user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location.

[0307] (K8) In some embodiments of the method described in K6 or K7, the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard.

[0308] (K9) In some embodiments of the method described in any one of K6 to K8, obtaining information includes obtaining information regarding a second physical location, and displaying the user interface object includes displaying a user interface object having information regarding the second physical location.

[0309] (K10) In some embodiments of the method described in any one of K1 to K9, the determination that the second application can receive geographical location information includes one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services.

[0310] (K11)In some embodiments of the method described in K10, the determination that the second application can receive geographical location information includes determining that the second application includes an input receiving field that can receive and process geographical location data, and the input receiving field is a search box that enables searching within a map displayed within the second application.

[0311] (K12)In some embodiments of the method described in any one of K1 to K11, the method further includes, in response to receiving a request, determining whether the second application is associated with the first application (e.g., opened a threshold number of times after the first application is opened) based on the user's application usage history.

[0312] (K13)In some embodiments of the method described in K12, the method further includes providing the second application with access to information identifying the first physical location before presenting the second application, and before the access is provided, the second application does not have access to information identifying the first physical location. In this way, the second application can receive information regarding actions performed by the user in the first application, and then the user is provided with a way to use this information within the second application (e.g., to search for further information regarding the first physical location or to use the first physical location for some service such as a ride-sharing service available through the second application).

[0313] (K14)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of K1 to K13.

[0314] (K15) In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, and means for performing the method according to any one of K1 to K13.

[0315] (K16) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface and a display, cause the electronic device to perform the method according to any one of K1 to K13.

[0316] (K17) In a further aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method according to any one of K1 to K13.

[0317] (K18) In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for performing the method according to any one of K1 to K13.

[0318] (K19) In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 5101, FIG. 51), a touch sensing surface unit (e.g., touch sensing surface unit 5103, FIG. 51), and a processing unit (e.g., processing unit 5105, FIG. 51). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 51 shows display unit 5101 and touch sensing surface unit 5103 as being integrated into electronic device 5100, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes an information acquisition unit (e.g., information acquisition unit 5107, FIG. 51), an application termination unit (e.g., application termination unit 5109, FIG. 51), a request reception unit (e.g., request reception unit 5111, FIG. 51), an application performance determination unit (e.g., application performance determination unit 5113, FIG. 51), an application presentation unit (e.g., application presentation unit 5115, FIG. 51), an application populating unit (e.g., application populating unit 5117, FIG. 51), an input detection unit (e.g., input detection unit 5119, FIG. 51), an application switching user interface display unit (e.g., application switching user interface display unit 5121, FIG. 51), an application association determination unit (e.g., application association determination unit 5123, FIG. 51), and an access providing unit (e.g., access providing unit 5125, FIG. 51). The processing unit (or one or more of its components such as units 5107 through 5125) is configured to obtain information identifying a first physical location browsed by a user in a first application while the first application is being displayed (e.g., by information acquisition unit 5107).The processing unit is also configured to terminate the first application (e.g., by the application termination unit 5109), and after the termination of the first application, receive a request from the user to open a second application different from the first application (e.g., by the request receiving unit 5111). In response to receiving the request, and according to a determination that the second application can receive geographical location information (e.g., a determination processed or executed by the application performance determination unit 5113), present the second application (e.g., by the application presenting unit 5115), and presenting the second application includes populating the second application, at least in part, with information based on information identifying the first physical location (e.g., by the application populating unit 5117).

[0319] (K20)In some embodiments of the electronic device described in K19, the processing unit (or one or more of its components such as units 5107 to 5125) is further configured to execute the method described in any one of K2 to K13.

[0320] (L1)In some embodiments, the method is performed in an electronic device having a touch sensing surface and a display (in some embodiments, a touch screen 112, where the touch sensing surface and the display are integrated as shown in FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E). The method includes obtaining information identifying a first physical location viewed by a user in a first application and detecting a first input. In response to detecting the first input, the method includes (i) identifying a second application capable of receiving geographical location information and (ii) presenting an affordance having a proposal to open a second application different from the first application and having information regarding the first physical location on at least a portion of the display. The method also includes detecting a second input in the affordance. In response to detecting the second input in the affordance, the method includes (i) opening the second application and (ii) populating the second application to include information based at least in part on the information identifying the first physical location.

[0321] Compared with the operations related to K1 above, in the operations related to L1, before providing the user with a proposal to open a second application having information about a first physical location, no specific request to open the second application is received from the user. In this way, by providing operations related to both K1 and L1 (and combinations thereof using some processing steps of each of these methods), the electronic device can provide an efficient user experience that enables predictive use of location data either before or after the user opens an application that can receive geographical location information. Additionally, with L1, a determination that the second application can receive geographical location information is made even before the second application is opened. In this way, in embodiments of L1 where the input corresponds to a request to open an application switching user interface, the application switching user interface simply displays a proposal to open an application (e.g., the second application) having information about a first physical location when it is known that the application can receive location data.

[0322] (L2)In some embodiments of the method described in L1, the first input corresponds to a request to open an application switching user interface (e.g., the first input is a double - tap on the physical home button of the electronic device).

[0323] (L3)In some embodiments of the method described in L2, the affordance is presented within the application switching user interface.

[0324] (L4)In some embodiments of the method described in L3, presenting an affordance includes, in addition to presenting the affordance, presenting a representation of an application (e.g., a snapshot of the application content of the application) being executed on the electronic device within an application switching user interface, and presenting the affordance within a region of the display located below the representation of the application.

[0325] (L5)In some embodiments of the method described in L1, the first input corresponds to a request to open the home screen of the electronic device (e.g., the first input is a single tap on the physical home button of the electronic device).

[0326] (L6)In some embodiments of the method described in L5, the affordance is presented on a portion of the home screen.

[0327] (L7)In some embodiments of the method described in any one of L1 to L6, the proposal includes an explanation in text specific to the type associated with the second application.

[0328] (L8)In some embodiments of the method described in any one of L1 to L7, populating the second application includes displaying a user interface object that includes information based at least in part on information identifying the first physical location.

[0329] (L9)In some embodiments of the method described in L8, the user interface object includes a textual explanation informing the user that the first physical location was most recently viewed in the first application.

[0330] (L10)In some embodiments of the method described in L9, the user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location.

[0331] (L11)In some embodiments of the method described in L9 or L10, the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard.

[0332] (L12)In some embodiments of the method described in any one of L1 to L11, identifying a second application capable of receiving geographical location information includes one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services.

[0333] (L13)In some embodiments of the method described in L12, identifying that the second application can receive geographical location information includes determining that the second application includes an input receiving field capable of receiving and processing geographical location data, and the input receiving field is a search box that enables searching within a map displayed within the second application.

[0334] (L14)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of L1 to L13.

[0335] (L15) In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, and means for performing the method according to any one of L1 to L13.

[0336] (L16) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface and a display, cause the electronic device to perform the method according to any one of L1 to L13.

[0337] (L17) In a further aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method according to any one of L1 to L13.

[0338] (L18) In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for performing the method according to any one of L1 to L13.

[0339] (L19)In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 5201, FIG. 52), a touch sensing surface unit (e.g., touch sensing surface unit 5203, FIG. 52), and a processing unit (e.g., processing unit 5205, FIG. 52). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 52 shows display unit 5201 and touch sensing surface unit 5203 as being integrated into electronic device 5200, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes an information acquisition unit (e.g., information acquisition unit 5207, FIG. 52), an input detection unit (e.g., input detection unit 5209, FIG. 52), an application specific unit (e.g., application specific unit 5211, FIG. 52), an affordance presentation unit (e.g., affordance presentation unit 5213, FIG. 52), an application open unit (e.g., application open unit 5215, FIG. 52), an application populating unit (e.g., application populating unit 5217, FIG. 52), an application switch user interface presentation unit (e.g., application switch user interface presentation unit 5219, FIG. 52), and an application performance determination unit (e.g., application performance determination unit 5221, FIG. 52). The processing unit (or one or more of its components such as units 5207 through 5221) is configured to obtain information identifying a first physical location viewed by a user in a first application (e.g., by information acquisition unit 5207) and to detect a first input (e.g., by input detection unit 5209).In response to detecting the first input, the processing unit is configured to: (i) identify a second application that can receive geographical location information (e.g., by the application identification unit 5209), and (ii) present an affordance having a proposal to open a second application having information regarding a first physical location, different from the first application, on at least a portion of the display (e.g., by the affordance presentation unit 5213). The processing unit is also configured to detect a second input in the affordance (e.g., by the input detection unit 5209). In response to detecting the second input in the affordance, the processing unit is configured to: (i) open the second application (e.g., by the application open unit 5215), and (ii) populate the second application to include information based at least in part on information identifying the first physical location (e.g., by the application populating unit 5217).

[0340] (L20)In some embodiments of the electronic device described in L19, the processing unit (or one or more of its components such as units 5207 to 5221) is further configured to execute the method described in any one of L2 to L13.

[0341] (M1)In some embodiments, a method is performed in an electronic device (e.g., a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A to D (FIG. 1E)) having a touch-sensing surface and a display (in some embodiments, a touch screen 112, where the touch-sensing surface and the display are integrated as shown in FIG. 1C). The method includes obtaining information identifying a first physical location viewed by a user in a first application running on the electronic device. The method also includes determining that the user has entered a vehicle. In response to determining that the user has entered the vehicle, a prompt is provided to the user to use the first physical location as a destination for route guidance. The method includes receiving, in response to the provision of the prompt, a command from the user to use the first physical location as a destination for route guidance. The method further includes facilitating route guidance to the first physical location. In this way, a convenient proposal for destination routing is provided to the user based on a physical location previously viewed in an application on the electronic device.

[0342] (M2)In some embodiments of the method described in M1, the method includes detecting that the electronic device has received a message, where the message includes information identifying a second physical location, and providing, in response to the detection, a new prompt to the user to use the second physical location as a new destination for route guidance. In this way, based on information included in a message (e.g., text, email, voicemail, etc.), the user can dynamically add an intermediate point or add a new destination for route guidance.

[0343] (M3)In some embodiments of the method described in M2, the method further includes facilitating route guidance to the second physical location in response to receiving a command from the user to use the second physical location as a new destination.

[0344] (M4) In some embodiments of the method described in M2 or M3, detecting that a message includes information identifying a second physical location includes performing the detection while a virtual assistant available on the electronic device reads the message aloud to the user via an audio system communicating with the electronic device. In this way, when the user is listening to a message read aloud by the audio system (e.g., via a personal assistant available through the electronic device), the electronic device detects the information identifying the second physical location and uses this detected information to propose using the second physical location as a new destination. Thus, the user does not need to divert attention from the road while driving and can still dynamically adjust route guidance settings and destinations.

[0345] (M5) In some embodiments of the method described in any one of M2 to M4, determining that the user has entered the vehicle includes detecting that the electronic device has established a communication link with the vehicle.

[0346] (M6) In some embodiments of the method described in any one of M2 to M5, facilitating route guidance includes providing route guidance via a display of the electronic device.

[0347] (M7) In some embodiments of the method described in any one of M2 to M5, facilitating route guidance includes transmitting information identifying the first physical location to the vehicle.

[0348] (M8) In some embodiments of the method described in any one of M2 to M7, facilitating route guidance includes providing route guidance by an audio system (e.g., a car speaker or a built-in speaker of the device itself) communicating with the electronic device.

[0349] (M9) In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensing surface, a display, one or more processors, and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of M1 to M8.

[0350] (M10) In yet another aspect, an electronic device is provided, the electronic device including a touch-sensing surface, a display, and means for executing the method described in any one of M1 to M8.

[0351] (M11) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions, and when the executable instructions are executed by an electronic device having a touch-sensing surface and a display, the executable instructions cause the electronic device to execute the method described in any one of M1 to M8.

[0352] (M12) In a further aspect, a graphical user interface on an electronic device having a touch-sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of M1 to M8.

[0353] (M13) In a further aspect, an information processing apparatus for use in an electronic device including a touch-sensing surface and a display is provided. The information processing apparatus includes means for executing the method described in any one of M1 to M8.

[0354] In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5301, FIG. 53), a touch sensing surface unit (e.g., touch sensing surface unit 5303, FIG. 53), and a processing unit (e.g., processing unit 5305, FIG. 53). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 53 shows display unit 5301 and touch sensing surface unit 5303 as being integrated into electronic device 5300, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes an information acquisition unit (e.g., information acquisition unit 5307, FIG. 53), a vehicle entry determination unit (e.g., vehicle entry determination unit 5309, FIG. 53), a prompt providing unit (e.g., prompt providing unit 5311, FIG. 53), a command receiving unit (e.g., command receiving unit 5313, FIG. 53), a route guidance facilitation unit (e.g., route guidance facilitation unit 5315, FIG. 53), and a message detection unit (e.g., message detection unit 5317, FIG. 53). The processing unit (or one or more of its components such as units 5307 through 5317) is configured to obtain information identifying a first physical location viewed by a user in a first application running on the electronic device (e.g., by information acquisition unit 5307). The processing unit is also configured to determine that the user has entered the vehicle (e.g., by vehicle entry determination unit 5309). In response to determining that the user has entered the vehicle, the processing unit is configured to provide a prompt to the user to use the first physical location as a destination for route guidance (e.g., by prompt providing unit 5311). In response to the provision of the prompt, a command is received from the user to use the first physical location as a destination for route guidance (e.g., by command receiving unit 5313).The processing unit is further configured to facilitate route guidance to a first physical location (e.g., by the route guidance facilitation unit 5307).

[0355] (M15)In some embodiments of the electronic device described in M14, the processing unit (or one or more of its components such as units 5307 to 5317) is further configured to execute the method described in any one of M2 to M8.

[0356] (N1)In some embodiments, the method is performed on an electronic device having a touch sensing surface and a display (in some embodiments, a touch screen 112, where the touch sensing surface and the display are integrated as shown in FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E). The method includes presenting content in a first application. The method also includes receiving, from a user, a request to open a second application different from the first application, where the second application includes an input receiving field. The method includes presenting the input receiving field in the second application in response to receiving the request. The method includes providing a selectable user interface object to enable the user to paste at least a portion of the content into the input receiving field before receiving any user input in the input receiving field. The method includes pasting a portion of the content into the input receiving field in response to detecting a selection of the selectable user interface object. In this way, a proactive paste action is provided to the user in the second - application - based content previously viewed in a first action (e.g., thereby enabling the user to paste content into the second application without having to reopen the first application, perform an explicit copy action, reopen the second application, and then explicitly request to paste the copied content into the second application).

[0357] (N2)According to some embodiments of the method of N1, the method includes identifying the input receiving field as a field capable of receiving a portion of the content before providing the selectable user interface object.

[0358] (N3)According to some embodiments of the method described in N2, identifying the input receiving field as a field capable of receiving a part of the content is performed in response to detecting the selection of the input receiving field.

[0359] (N4)According to some embodiments of the method described in any one of N1 to N3, a part of the content corresponds to an image.

[0360] (N5)According to some embodiments of the method described in any one of N1 to N3, a part of the content corresponds to text content.

[0361] (N6)According to some embodiments of the method described in any one of N1 to N3, a part of the content corresponds to text content and an image.

[0362] (N7)According to some embodiments of the method described in any one of N1 to N6, the first application is a web browsing application and the second application is a messaging application.

[0363] (N8)According to some embodiments of the method described in any one of N1 to N6, the first application is a photo browsing application and the second application is a messaging application.

[0364] (N9)According to some embodiments of the method described in any one of N1 to N8, the method includes receiving a request to copy at least a part of the content before receiving a request to open the second application.

[0365] (N10)According to some embodiments of the method described in any one of N1 to N9, the selectable user interface object is displayed with an indication that a part of the content has been recently viewed in the first application. In this way, the user is provided with a clear indication of why the paste suggestion is made.

[0366] (N11)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to execute the method described in any one of N1 to N10.

[0367] (N12)In yet another aspect, an electronic device is provided, which includes a touch-sensitive surface, a display, and means for executing the method described in any one of N1 to N10.

[0368] (N13)In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions, and when the executable instructions are executed by an electronic device having a touch-sensitive surface and a display, the executable instructions cause the electronic device to execute the method described in any one of N1 to N10.

[0369] (N14)In a further aspect, a graphical user interface on an electronic device having a touch-sensitive surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method described in any one of N1 to N10.

[0370] (N15)In a further aspect, an information processing apparatus for use in an electronic device including a touch-sensitive surface and a display is provided. The information processing apparatus includes means for executing the method described in any one of N1 to N10.

[0371] In a further aspect, an electronic device is provided that includes a display unit (e.g., display unit 5401, FIG. 54), a touch sensing surface unit (e.g., touch sensing surface unit 5403, FIG. 54), and a processing unit (e.g., processing unit 5405, FIG. 54). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 54 shows display unit 5401 and touch sensing surface unit 5403 as being integrated into electronic device 5400, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5407, FIG. 54), a request receiving unit (e.g., request receiving unit 5409, FIG. 54), a user interface object providing unit (e.g., user interface object providing unit 5411, FIG. 54), a predictive pasting unit (e.g., predictive pasting unit 5413, FIG. 54), and a performance determination unit (e.g., performance determination unit 5415, FIG. 54).The processing unit (or one or more components thereof such as units 5407 to 5415) presents content in a first application (e.g., by presentation unit 5407 and / or display unit 5401), receives a request from the user to open a second application different from the first application (e.g., by request receiving unit and / or touch sensing surface unit 5403), the second application includes an input receiving field, and in response to receiving the request, presents the input receiving field in the second application (e.g., by presentation unit 5407 and / or display unit 5401), and provides a selectable user interface object (e.g., by user interface object providing unit 5411 and / or display unit 5401) to enable the user to paste at least a part of the content into the input receiving field before receiving any user input in the input receiving field, and is configured to paste a part of the content into the input receiving field (e.g., by anticipatory pasting unit 5413) in response to detecting a selection of the selectable user interface object.

[0372] (N17) In some embodiments of the electronic device described in N16, the processing unit (or one or more components thereof such as units 5407 to 5415) is further configured to execute the method described in any one of N1 to N10.

[0373] (O1)In some embodiments, the method is performed on an electronic device having a touch-sensing surface and a display (in some embodiments, a touch screen 112, where the touch-sensing surface and the display are integrated as shown in FIG. 1C), such as a portable multifunctional device 100 (FIG. 1A) configured according to any one of computing devices A - D (FIG. 1E). The method includes presenting text content associated with an application on the display. The method also includes determining that a portion of the text content is related to (i) a location, (ii) a contact, or (iii) an event. When the method determines that a portion of the text content is related to a location, it includes obtaining location information from a location sensor on the electronic device and preparing the obtained location information for display as a predicted content item. When the method determines that a portion of the text content is related to a contact, it includes searching on the electronic device for contact information related to the portion of the text content and preparing the information retrieved by the search, associated with at least one contact, for display as a predicted content item. When the method determines that a portion of the text content is related to an event, it includes performing a new search on the electronic device for event information related to the portion of the text content and preparing at least some of the information based on at least one event retrieved by the new search for display as a predicted content item. The method further includes displaying an affordance including the predicted content item within the application, detecting a selection of the affordance via the touch-sensing surface, and in response to detecting the selection, displaying the information associated with the predicted content item adjacent to the text content on the display. In this way, predicted content items that can be used by the user to complete a sentence (or, for example, answer a question posed by another user within a messaging application) are conveniently provided, and the user does not need to type anything or search through the information available on the electronic device to find the desired information.For example, the electronic device provides a phone number, current location, availability for scheduling new events, and all details related to current events, without any explicit request from the user or any extra effort, saving time while still efficiently providing the user with the desired information.

[0374] (O2) According to some embodiments of the method described in O1, a part of the text content corresponds to the text content most recently presented within the application.

[0375] (O3) According to some embodiments of the method described in O1 or O2, the application is a messaging application, and a part of the text content is a question received from a remote user of a remote device different from the electronic device within the messaging application.

[0376] (O4) According to some embodiments of the method described in O1 or O2, a part of the text content is an input provided by the user of the electronic device in an input receiving field within the application.

[0377] (O5) According to some embodiments of the method described in O1, a part of the text content is identified in response to a user input that selects a user interface object including the part of the text content.

[0378] (O6) According to some embodiments of the method described in O5, the application is a messaging application, and the user interface object is a speech bubble for a message in a conversation displayed within the messaging application.

[0379] (O7)According to some embodiments of the method described in O5 or O6, the method further includes detecting a selection of a second user interface object, and in response to detecting the selection, (i) stopping displaying with predicted affordance content items, determining that text content associated with the second user interface object is related to a location, a contact, or an event, and displaying new predicted content items within the application according to the determination. In this way, the user can easily return to the conversation of the message, select previously received messages, and still have appropriately predicted content items provided.

[0380] (O8)According to some embodiments of the method described in any one of O1 to O7, the affordance is displayed in an input receiving field adjacent to a virtual keyboard within the application.

[0381] (O9)According to some embodiments of the method described in O8, the input receiving field is a field that displays typing input received on the virtual keyboard.

[0382] (O10)According to some embodiments of the method described in any one of O1 to O9, determining includes parsing the text content when received by the application and detecting stored patterns known to be related to contacts, events, and / or locations.

[0383] (O11)In another aspect, an electronic device is provided. In some embodiments, the electronic device includes a touch-sensitive surface, a display, one or more processors, and a memory storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more programs cause the electronic device to perform the method described in any one of O1 to O10.

[0384] (O12) In yet another aspect, an electronic device is provided, the electronic device including a touch sensing surface, a display, and means for performing the method according to any one of O1 to O10.

[0385] (O13) In yet another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores executable instructions that, when executed by an electronic device having a touch sensing surface and a display, cause the electronic device to perform the method according to any one of O1 to O10.

[0386] (O14) In a further aspect, a graphical user interface on an electronic device having a touch sensing surface and a display is provided. In some embodiments, the graphical user interface includes a user interface displayed according to the method according to any one of O1 to O10.

[0387] (O15) In a further aspect, an information processing apparatus for use in an electronic device including a touch sensing surface and a display is provided. The information processing apparatus includes means for performing the method according to any one of O1 to O10.

[0388] In yet another aspect, an electronic device is provided that includes a display unit (e.g., display unit 5501, FIG. 55), a touch sensing surface unit (e.g., touch sensing surface unit 5503, FIG. 55), and a processing unit (e.g., processing unit 5505, FIG. 55). In some embodiments, the electronic device is configured according to any one of the computing devices shown in FIG. 1E (e.g., computing devices A through D). For ease of explanation, FIG. 55 shows display unit 5501 and touch sensing surface unit 5503 as being integrated into electronic device 5500, but in some embodiments, one or both of these units communicate with the electronic device even if the unit remains physically separated from the electronic device. The processing unit includes a presentation unit (e.g., presentation unit 5507, FIG. 55), a determination unit (e.g., determination unit 5509, FIG. 55), an acquisition unit (e.g., acquisition unit 5511, FIG. 55), a search execution unit (e.g., search execution unit 5513, FIG. 55), an information preparation unit (e.g., information preparation unit 5515, FIG. 55), an affordance display unit (e.g., affordance display unit 5517, FIG. 55), and a detection unit (e.g., detection unit 5519, FIG. 55).The processing unit (or one or more of its components such as units 5507 to 5519) presents text content associated with an application on a display (e.g., by presentation unit 5507 and / or display unit 5501), determines that a part of the text content is related to (i) a location, (ii) a contact, or (iii) an event (e.g., by determination unit 5509), and when it is determined that a part of the text content is related to a location, obtains location information from a location sensor on the electronic device (e.g., by acquisition unit 5511), and prepares the obtained location information as a predicted content item for display (e.g., by information preparation unit 5515). When it is determined that a part of the text content is related to a contact, searches for contact information related to a part of the text content on the electronic device (e.g., by search execution unit 5513), and prepares the information retrieved by the search and associated with at least one contact as a predicted content item for display (e.g., by information preparation unit 5515). When it is determined that a part of the text content is related to an event, performs a new search for event information related to a part of the text content on the electronic device (e.g., by search execution unit 5513), and prepares at least part of the information based on at least one event retrieved by the new search as a predicted content item for display (e.g., by information preparation unit 5515). It is configured to display an affordance including the predicted content item within the application (e.g., by affordance display unit 5517 and / or display unit 5501), detect the selection of the affordance via a touch-sensitive surface (e.g., by detection unit 5519), and in response to detecting the selection, display the information associated with the predicted content item adjacent to the text content on the display (e.g., by presentation unit 5507 and / or display unit 5501).

[0389] In some embodiments of the electronic device described in O17, the processing unit (or one or more components thereof such as units 5507 to 5519) is further configured to execute the method described in any one of O1 to O10.

[0390] As described above (and in more detail below), one aspect of the present technology is the collection and use of data available from various sources (e.g., based on speech provided during a voice communication) to improve the distribution of content that a user may be interested in to the user. The present disclosure contemplates that in some instances, the collected data may include personal information data that can be used to uniquely identify a particular person, or to communicate with or locate a particular person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, home addresses, or any other personally identifiable information.

[0391] The present disclosure recognizes that in the present technology, the use of such personal information data can be used for the benefit of the user. For example, personal information data may be used to deliver more content of interest to the user. Thus, the use of such personal information data enables intentional control of the content to be delivered. Further, other uses of personal information data that benefit the user are contemplated by the present disclosure.

[0392] The present disclosure further contemplates that entities responsible for the collection, analysis, disclosure, transfer, storage, or other uses of such personal information data comply with appropriately defined privacy policies and / or privacy practices. Specifically, such entities should always implement and use privacy policies and practices that meet or exceed industry or government requirements for maintaining strict confidentiality of personal information data. For example, personal information from users should be collected for legitimate and proper use by the entity and should not be shared or sold except for such legitimate use. Further, such collection should be carried out only after informing and obtaining consent from the user. Further, such entities should protect and secure access to such personal information data and may take any necessary measures to ensure that others having access to such personal information data comply with their privacy policies and procedures. Further, such entities may subject themselves to evaluation by a third party to prove their compliance with widely accepted privacy policies and practices.

[0393] Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which a user can selectively block the use of personal information data or access to personal information data. That is, the present disclosure contemplates that hardware elements and / or software elements can be provided to prevent or block access to such personal information data. For example, when monitoring voice communications or actions performed by a user within an application, the technology can be configured to enable the user to select an “opt-in” or “opt-out” to participate in the collection of personal information data during service registration. In another example, a user may choose not to provide location information to a targeted content delivery service. In yet another example, a user may choose not to provide accurate location information but permit the transfer of location zone information.

[0394] Accordingly, while the present disclosure broadly covers the use of personal information data for carrying out one or more various disclosed embodiments, the present disclosure also contemplates that those various embodiments may also be carried out without the need to access such personal information data. That is, the various embodiments of the present technology are not rendered inoperable by the absence of all or a portion of such personal information data. For example, content can be selected and delivered to a user by inferring preferences based on non-personal information data such as content requested by a device associated with the user or minimal necessary personal information, other non-personal information available to the content delivery service, or publicly available information.

[0395] Note that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described herein are not exhaustive, and in particular, many further features and advantages will become apparent to those skilled in the art in view of the drawings, the specification, and the claims. Further, note that the language used herein has been selected solely for readability and for the purpose of explanation, and not for the purpose of defining or limiting the subject matter of the invention.

Brief Description of the Drawings

[0396] To better understand the various embodiments described, reference should be made to the following "Detailed Description of the Invention" section in conjunction with the following drawings. Here, like reference numerals refer to corresponding parts throughout those figures.

[0397]

Figure 1A

[0398]

Figure 1B

[0399]

Figure 1C

[0400]

Figure 1D

Figure 1E

[0401]

Figure 2

[0402]

Figure 3A

Figure 3B

[0403]

Figure 4A

Figure 4B

[0404]

Figure 5

[0405]

Figure 6A

Figure 6B

[0406]

Figure 7A

Figure 7B

[0407]

Figure 8A

Figure 8B

[0408]

Figure 9A

Figure 9B

Figure 9C

Figure 9D

[0409]

Figure 10A

Figure 10B

Figure 10C

[0410]

Figure 11A

Figure 11B

Figure 11C

Figure 11D

Figure 11E

Figure 11F

Figure 11G

Figure 11H

Figure 11I

Figure 11J

[0411]

Figure 12

[0412]

Figure 13A

Figure 13B

[0413]

Figure 14

[0414]

Figure 15A

Figure 15B

[0415]

Figure 16A

Figure 16B

[0416]

Figure 17A

Figure 17B

Figure 17C

Figure 17D

Figure 17E

[0417]

Figure 18A

Figure 18B

[0418]

Figure 19A

Figure 19B

Figure 19C

Figure 19D

Figure 19E

Figure 19F

[0419]

Figure 20

[0420]

Figure 21A

Figure 21B

[0421]

Figure 22A

Figure 22B

[0422]

Figure 22C

[0423]

Figure 23A

Figure 23B

Figure 23C

Figure 23D

Figure 23E

Figure 23F

Figure 23G

Figure 23H

Figure 23I

Figure 23J

Figure 23K

Figure 23L

Figure 23M

Figure 23N

Figure 23O

[0424]

Figure 24A

Figure 24B

[0425]

Figure 25A

Figure 25B

Figure 25C

Figure 25D

Figure 25E

Figure 25F

Figure 25G

Figure 25H

Figure 25I

Figure 25J

[0426]

Figure 26A

Figure 26B

[0427]

Figure 27

[0428]

Figure 28

[0429]

Figure 29

[0430]

Figure 30A

Figure 30B

Figure 30C

Figure 30D

[0431]

Figure 31-1

[0432]

Figure 31-2

[0433]

Figure 31-3

[0434]

Figure 31-4

[0435]

Figure 31-5

[0436]

Figure 31-6

[0437]

Figure 31-7

[0438]

Figure 31-8

[0439]

Figure 31-9

[0440]

Figure 31-10

[0441]

Figure 31-11

[0442]

Figure 31-12

[0443]

Figure 31-13

[0444]

Figure 31-14

[0445]

Figure 31-15

[0446]

Figure 31-16

[0447]

Figure 31-17

[0448]

Figure 32-1

[0449]

Figure 32-2

[0450]

Figure 32-3

[0451]

Figure 32-4

[0452]

Figure 32-5

Figure 32-6

[0453]

Figure 32-7

[0454]

Figure 32-8

[0455]

Figure 32-9

[0456]

Figure 32-10

[0457]

Figure 32-11

[0458]

Figure 32-12

[0459]

Figure 32-13

[0460]

Figure 32-14

[0461]

Figure 32-15

[0462]

Figure 32-16

[0463]

Figure 32-17

[0464]

Figure 32-18

[0465]

Figure 32-19

[0466]

Figure 32-20

[0467]

Figure 32-21

[0468]

Figure 32-22

[0469]

Figure 33-1

[0470]

Figure 33-2

[0471]

Figure 33-3

[0472]

Figure 33-4

[0473]

Figure 33-5

[0474]

Figure 33-6

[0475]

Figure 33-7

[0476]

Figure 34-5A

[0477]

Figure 34-5B

[0478]

Figure 34-6A

Figure 34-6B

Figure 34-6C

Figure 34-6D

Figure 34-6E

Figure 34-6F

Figure 34-6G

[0479]

Figure 34-7A

Figure 34-7B

[0480]

Figure 34-8A

Figure 34-8B

[0481]

Figure 34-9A

Figure 34-9B

[0482]

Figure 34-10

[0483]

Figure 34-11A

Figure 34-11B

[0484]

Figure 34-12

[0485]

Figure 34-13

[0486]

Figure 34-14

[0487]

Figure 34-15

[0488]

Figure 35-1

[0489]

Figure 35-2

[0490]

Figure 35-3

[0491]

Figure 35-4

[0492]

Figure 35-5A

Figure 35-5B

Figure 35-5C

Figure 35-5D

[0493]

Figure 35-6A

Figure 35-6B

[0494]

Figure 35-7

[0495]

Figure 36-1

[0496]

Figure 36-2

[0497]

Figure 36-3

[0498]

Figure 36-4

[0499]

Figure 36-5

[0500]

Figure 36-6

[0501]

Figure 36-7A

Figure 36-7B

[0502]

Figure 37-1

[0503]

Figure 37-2

[0504]

Figure 37-3

[0505]

Figure 37-4

[0506]

Figure 37-5

[0507]

Figure 37-6

[0508]

Figure 37-7

[0509]

Figure 37-8

[0510]

Figure 37-9

[0511]

Figure 38-1

[0512]

Figure 38-2

[0513]

Figure 38-3

[0514]

Figure 38-4

[0515]

Figure 39-1

[0516]

Figure 39-2

[0517]

Figure 39-3A

[0518]

Figure 39-3B

[0519]

Figure 39-3C

[0520]

Figure 39-4A

[0521]

Figure 39-4B

[0522]

Figure 39-5A

[0523]

Figure 39-5B

[0524]

Figure 39-5C

[0525]

Figure 40-1

[0526]

Figure 40-2A

[0527]

Figure 40-2B

[0528]

Figure 40-3

[0529]

Figure 40-4

[0530]

Figure 40-5

[0531]

Figure 40-6

[0532]

Figure 40-7

[0533]

Figure 40-8A

[0534]

Figure 40-8B

[0535]

Figure 40-9A

[0536]

Figure 40-9B

[0537]

Figure 40-10A

[0538]

Figure 40-10B

[0539]

Figure 40-11

[0540]

Figure 40-12A

[0541]

Figure 40-12B

[0542]

Figure 40-13A

[0543]

Figure 40-13B

[0544]

Figure 40-14

[0545]

Figure 40-15

[0546]

Figure 40-16A

[0547]

Figure 40-16B

[0548]

Figure 40-17

[0549]

Figure 40-18

[0550]

Figure 40-19

[0551]

Figure 40-20

[0552]

Figure 40-21

[0553]

Figure 40-22

[0554]

Figure 41-1

[0555]

Figure 41-2

[0556]

Figure 41-3

[0557]

Figure 41-4A

[0558]

Figure 41-4B

[0559]

Figure 41-5

[0560]

Figure 41-6

[0561]

Figure 41-7

[0562]

Figure 41-8

[0563]

Figure 41-9

[0564]

Figure 41-10

[0565]

Figure 41-11

[0566]

Figure 41-12

[0567]

Figure 41-13

[0568]

Figure 41-14

[0569]

Figure 41-15

[0570]

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Modes for Carrying Out the Invention

[0571] As described above and as will be detailed below, there is a growing need for electronic devices that provide faster and more efficient ways and interfaces for accessing applications and desired functions within those applications. In particular, there is a growing need for devices that assist the user by eliminating repetitive tasks and providing anticipatory assistance by identifying and providing relevant information before the user explicitly requests it. Further, there is a growing need to quickly access applications and desired functions within those applications at specific times (e.g., accessing the calendar application after waking up every morning) and specific locations (e.g., accessing the music application at the gym). What is disclosed herein is a new way and interface that addresses those needs and provides the user with a way to quickly access applications and functions within those applications at those specific locations and times. Such ways and interfaces optionally complement or replace existing ways of accessing applications. Such ways and interfaces reduce the user's cognitive burden and create a more efficient human-machine interface. In the case of battery-operated devices, such ways and interfaces conserve power and extend the interval between battery charges. Further, such ways and interfaces assist in extending the life of the touch-sensitive display by reducing the number of inputs required (e.g., instead of continuously and randomly tapping the touch-sensitive display to identify desired information, the ways and interfaces disclosed herein provide information anticipatorily without the need for user input).

[0572] The following FIGS. 1A - 1E and 2 provide an illustration of a device example. FIGS. 10 and 11 provide functional block diagrams of an electronic device example. FIGS. 3A - 3B and 4A - 4B are block diagrams of example data structures used to proactively identify and surface relevant content (these data structures are used in the ways described in relation to FIGS. 6A - 6B and FIGS. 8A - 8B). FIG. 5 is a block diagram showing an example system for establishing trigger conditions used to proactively identify and surface relevant content (this system example is used in the ways described in relation to FIGS. 6A - 6B and FIGS. 8A - 8B). FIGS. 6A - 6B are flowcharts showing ways to proactively identify and surface relevant content. FIGS. 7A - 7B are an overview of a touch - sensitive display used to show example user interfaces and gestures for proactively identifying and surfacing relevant content. FIGS. 8A - 8B are flowcharts showing ways to proactively identify and surface relevant content. FIGS. 9A - 9D are an overview of a touch - sensitive display used to show additional user interfaces for proactively identifying and surfacing relevant content. FIGS. 3A - 3B, 4A - 4B, 5, and 7A - 7B are used to show the methods and / or processes of FIGS. 6A - 6B. FIGS. 3A - 3B, 4A - 4B, 5, and 9A - 9D are used to show the methods and / or processes of FIGS. 8A - 8B.

[0573] FIGS. 10A - 10C are flowcharts showing ways to proactively propose search queries based on content currently displayed on an electronic device equipped with a touch - sensitive display. FIGS. 11A - 11J are an overview of a touch - sensitive display used to show a user interface for proactively proposing search queries based on content currently displayed on the touch - sensitive display. FIGS. 11A - 11J are used to show the methods and / or processes of FIGS. 10A - 10C.

[0574] FIG. 12 is a flowchart showing a method of transitioning to a search mode based on heuristics. FIGS. 13A-13B are an overview of a touch-sensitive display used to show a user interface for transitioning to a search mode based on heuristics. FIGS. 13A-13B are used to show the method and / or process of FIG. 12.

[0575] FIG. 14 is a flowchart showing a method of proactively providing vehicle position on an electronic device having a touch-sensitive display, according to some embodiments. FIGS. 15A-15B are an overview of a touch-sensitive display used to show a user interface for proactively providing vehicle position, according to some embodiments. FIGS. 15A-15B are used to show the method and / or process of FIG. 14.

[0576] FIGS. 16A-16B are a flowchart showing a method of proactively providing neighboring point of interest (POI) information for a search query, according to some embodiments. FIGS. 17A-17E are an overview of a touch-sensitive display used to show a user interface for proactively providing neighboring point of interest (POI) information for a search query, according to some embodiments. FIGS. 16A-16B are used to show the method and / or process of FIGS. 17A-17E.

[0577] FIGS. 18A-18B are a flowchart showing a method of extracting content items from a voice communication and interacting with the extracted content items, according to some embodiments. FIGS. 19A-19F are an overview of a touch-sensitive display used to show a user interface for displaying and interacting with content items extracted from a voice communication, according to some embodiments. FIGS. 19A-19F are used to show the method and / or process of FIGS. 18A-18B.

[0578] FIG. 20 is a flowchart showing a method for determining that a voice communication includes speech that identifies a physical location and populating an application with information about that physical location, according to some embodiments. FIGS. 21A-21B are an overview of a touch-sensitive display used to show a user interface for determining that a voice communication includes speech that identifies a physical location and populating an application with information about that physical location, according to some embodiments. FIGS. 19A-19F and 21A-21B are used to illustrate the method and / or process of FIG. 20.

[0579] FIGS. 22A-22B are a flowchart showing a method for proactively suggesting a physical location for use in a messaging application, according to some embodiments. FIGS. 23A-23O are an overview of a touch-sensitive display used to show a user interface for proactively suggesting information about a location, event, or contact (e.g., for easy selection by a user and for inclusion within a messaging application), according to some embodiments. FIGS. 23A-23O are used to illustrate the method and / or process of FIGS. 22A-22B.

[0580] FIG. 22C is a flowchart showing a method for proactively suggesting information about a location, event, or contact, according to some embodiments. FIGS. 23A-23O are used to illustrate the method and / or process of FIG. 22C.

[0581] Figures 24A - 24B are flowcharts showing a method for pre - populating an application with information previously viewed by a user in a different application, in accordance with some embodiments. Figures 25A - 25J are an overview of a touch - sensitive display used to show a user interface for pre - populating (e.g., populating a rideshare application with information about a location viewed by the user in a review application) an application with information previously viewed by a user in a different application, in accordance with some embodiments. Figures 25A - 25J are used to illustrate the method and / or process of Figures 24A - 24B.

[0582] Figures 26A - 26B are flowcharts showing a method for proactively suggesting information previously viewed by a user in a first application for use in a second application, in accordance with some embodiments. Figures 25A - 25J are used to illustrate the method and / or process of Figures 26A - 26B.

[0583] Figure 27 is a flowchart showing a method for proactively suggesting a physical location for use as a destination for route guidance in a vehicle, in accordance with some embodiments. Figure 28 is an overview of a touch - sensitive display used to show a user interface for proactively suggesting a physical location for use as a destination for route guidance in a vehicle, in accordance with some embodiments. Figure 28 is used to illustrate the method and / or process of Figure 27.

[0584] Figure 29 is a flowchart showing a method for proactively suggesting a paste action, in accordance with some embodiments. Figures 30A - 30D are an overview of a touch - sensitive display used to show a user interface for proactively suggesting a paste action destination, in accordance with some embodiments. Figures 30A - 30D are used to illustrate the method and / or process of Figure 29.

[0585] Sections 1 to 11 within the "Additional Modes for Carrying Out the Invention" section describe additional details that complement the explanations provided in relation to FIGS. 1A to 30D.

[0586] Here, refer in detail to the embodiments shown in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0587] In this specification, terms such as first, second, etc. are used in some embodiments to describe various elements, but it will also be understood that these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact.

[0588] The terms used in the description of the various embodiments described herein are for the purpose of describing particular embodiments only and are not intended to be limiting. When used in the various described embodiments and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. As used herein, the term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items. The terms "includes", "including", "comprises", and / or "comprising", when used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0589] As used herein, the term "if" is optionally construed to mean "when", "upon", "in response to determining", or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are optionally construed to mean "upon determining", "in response to determining", "upon detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)", depending on the context.

[0590] The disclosure of this specification equivalently refers to the detection of touch input on, at, over, on top of, or substantially within a particular user interface element or a particular portion of a touch-sensitive display. As used herein, a touch input detected "at" a particular user interface element may also, depending on the context, be a touch input detected "on," "across," "on top of," or "substantially within" the same user interface element. In some embodiments, and as detailed below, the desired sensitivity level for detecting touch input is configured by the user of the electronic device (e.g., the user may determine (and configure and operate the electronic device) that a touch input should only be detected if it is completely within the user interface element).

[0591] Embodiments of an electronic device, a user interface for such a device, and related processes for using such a device are described. In some embodiments, the device is a portable communication device such as a mobile phone that also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunctional devices include, without limitation, the IPHONE (registered trademark), IPOD TOUCH (registered trademark), and IPAD (registered trademark) devices of APPLE Inc. of Cupertino, California. Other portable electronic devices such as laptop or tablet computers having a touch-sensitive surface (e.g., a touch-sensitive display and / or a touch pad) may optionally be used. Also, it should be understood that in some embodiments, the device is a desktop computer having a touch-sensitive surface (e.g., a touch-sensitive display and / or a touch pad) rather than a portable communication device.

[0592] In the following description, an electronic device having a display and a touch sensing surface will be described. However, it should be understood that this electronic device optionally includes one or more other physical user interface devices such as a physical keyboard, a mouse, and / or a joystick.

[0593] This device generally supports various applications such as one or more of a drawing application, a presentation application, a word processing application, a website creation application, a disk authoring application, a spreadsheet application, a game application, a telephone application, a video conferencing application, an email application, an instant messaging application, a health / fitness application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0594] The various applications executed on this device optionally use at least one common physical user interface device such as a touch sensing surface. One or more functions of the touch sensing surface, as well as the corresponding information displayed on the device, are optionally adjusted and / or changed for each application and / or within the corresponding application. Thus, the common (such as a touch sensing surface) physical architecture of the device optionally supports various applications having an intuitive and transparent user interface for the user.

[0595] Next, attention is directed to an embodiment of a portable electronic device having a touch-sensing display. FIG. 1A is a block diagram showing a portable multifunctional device 100 (also referred to interchangeably herein as electronic device 100 or device 100) having a touch-sensing display 112, according to some embodiments. The touch-sensing display 112 may be referred to herein for convenience as a "touch screen", and may also be known or referred to as a touch-sensing display system. Device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a controller 120, one or more processing units (CPUs) 122, a peripheral interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. Device 100 optionally includes one or more light sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of a contact on device 100 (e.g., a touch-sensing surface such as touch-sensing display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating a tactile output on device 100 (e.g., generating a tactile output on a touch-sensing surface such as touch-sensing display system 112 of device 100 or a touch pad of device 100). These components optionally communicate via one or more communication buses or signal lines 103.

[0596] As used in this specification and the claims, the term "intensity" of a contact on a touch-sensing surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on the touch-sensing surface, or an alternative (substitute) for the force or pressure of a contact on the touch-sensing surface. The intensity of the contact has a range of values that includes at least four distinct values and more typically hundreds of different values (e.g., at least 256). The intensity of the contact is optionally determined (or measured) using a variety of methods and a variety of sensors, or combinations of sensors. For example, one or more force sensors disposed under or adjacent to the touch-sensing surface are optionally used to measure the force at various points on the touch-sensing surface. In some implementations, the force measurements of multiple force sensors are combined (e.g., weighted average) to determine an estimated value of the contact force. Similarly, a pressure-sensing tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensing surface. Alternatively, the size and / or change thereof of the contact area detected on the touch-sensing surface, the capacitance and / or change thereof of the touch-sensing surface in proximity to the contact, and / or the resistance and / or change thereof of the touch-sensing surface in proximity to the contact are optionally used as an alternative to the force or pressure of a contact on the touch-sensing surface. In some implementations, the alternative measurements of the force or pressure of the contact are used directly to determine whether the intensity threshold is exceeded (e.g., the intensity threshold is described in units corresponding to the alternative measurements). In some implementations, the alternative measurements of the force or pressure of the contact are converted to an estimated force or pressure, and this estimated force or pressure is used to determine whether the intensity threshold is exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).

[0597] As used in this specification and the claims, the term "tactile output" refers to a physical displacement of a device relative to its previous position, a physical displacement of a component of the device (e.g., a touch-sensing surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of gravity of the device that will be detected by a user through the user's sense of touch. For example, in a situation where the device or a component of the device is in contact with a touch-sensitive surface of the user (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement is interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or the component of the device. For example, the movement of a touch-sensing surface (e.g., a touch-sensing display or a trackpad) may be optionally interpreted by the user as a "down click" or "up click" of a physical actuator button. In some cases, even when there is no movement of a physical actuator button associated with a touch-sensing surface that has been physically pressed (e.g., displaced) by a user action, the user may feel a tactile sensation such as a "down click" or "up click". As another example, the movement of a touch-sensing surface may be optionally interpreted or perceived by the user as the "roughness" of the touch-sensing surface, even when there is no change in the smoothness of the touch-sensing surface. Such an interpretation of a touch by the user is subject to the user's individual sensory perception, but there are many sensory perceptions of touch that are common to a majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of the user (e.g., "up click", "down click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of a device or a component of the device that produces the described sensory perception of a typical (or average) user.

[0598] Device 100 is merely an example of a portable multifunctional device, and it should be understood that Device 100 may optionally have more or fewer components than those shown, optionally combine two or more components, or optionally have different configurations or arrangements of components. The various components shown in FIG. 1A are implemented in hardware, software, or a combination of hardware and software, including one or more signal processing circuits and / or application specific integrated circuits.

[0599] Memory 102 optionally includes high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, or other random access solid state memory devices), and also optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state memory devices. Memory 102 optionally includes one or more storage devices located remotely from the processor(s) 122. Access to Memory 102 by other components of Device 100, such as CPU 122 and Peripheral Interface 118, is optionally controlled by Controller 120.

[0600] The Peripheral Interface 118 can be used to couple the input and output peripheral devices of the present device to the CPU 122 and the Memory 102. One or more processors 122 operate or execute various software programs and / or instruction sets stored in the Memory 102 to perform various functions for the Device 100 and to process data.

[0601] In some embodiments, the Peripheral Interface 118, the CPU 122, and the Controller 120 are optionally implemented on a single chip such as Chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0602] The RF (Radio Frequency) circuit 108 transmits and receives RF signals, also called electromagnetic signals. The RF circuit 108 converts electrical signals into electromagnetic signals or vice versa and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, etc. The RF circuit 108 optionally communicates wirelessly with the Internet, also called the World Wide Web (WWW), an intranet, and / or wireless networks such as a cellular telephone network, a wireless local area network (LAN) and / or a metropolitan area network (MAN), and other devices. The wireless communication optionally uses any of a plurality of communication standards, protocols, and technologies including but not limited to Global System for Mobile Communications (GSM) (registered trademark), Enhanced Data GSM Environment (EDGE) (registered trademark), High-Speed Downlink Packet Access (HSDPA), High-Speed Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth (registered trademark), and / or Wireless Fidelity (Wi-Fi) (registered trademark) (e.g., IEEE802.11a, IEEE802.11b, IEEE802.11g, and / or IEEE802.11n).

[0603] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts this audio data into an electrical signal, and transmits this electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits this audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 further includes a headset jack. The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device such as an output-only headset or a headset having both an output (e.g., headphones for one or both ears) and an input (e.g., a microphone).

[0604] The I / O subsystem 106 connects the input / output peripheral devices of the device 100, such as the touch screen 112 and other input control devices 116, to the peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, a light sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from and transmit electrical signals to other input or control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) 160 are optionally coupled to (or not coupled to any of) a keyboard, an infrared port, a USB port, and pointer devices such as a mouse. The one or more buttons optionally include up / down buttons for adjusting the volume of the speaker 111 and / or the microphone 113. The one or more buttons optionally include push buttons.

[0605] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touch screen 112. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

[0606] The touch screen 112 has a touch sensing surface, sensor, or set of sensors that accepts input from a user based on tactile and / or haptic contact. The touch screen 112 and the display controller 156 (along with any associated modules and / or instruction sets within the memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into an interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the area under the user's finger.

[0607] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology or OLED (organic light emitting diode) technology, although in other embodiments other display technologies are used. The touch screen 112 and the display controller 156 use any of a plurality of touch sensing technologies, including, but not limited to, capacitance technology, resistive technology, infrared technology, and surface acoustic wave technology, and other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112, whether currently known or later developed, to optionally detect contact and any movement or interruption of the contact. In one example embodiment, projected mutual capacitance sensing technology, as seen in IPHONE (registered trademark), IPOD TOUCH (registered trademark), and IPAD (registered trademark) from APPLE Inc. of Cupertino, California, is used.

[0608] The touch screen 112 optionally has a video resolution exceeding 400 dpi. In some embodiments, the touch screen 112 has a video resolution of at least 600 dpi. In other embodiments, the touch screen 112 has a video resolution of at least 1000 dpi. The user optionally contacts the touch screen 112 using any suitable object or finger, such as a stylus or finger. In some embodiments, the user interface is designed to primarily function with finger-based contact and gestures. In some embodiments, the device converts finger-based input into an accurate pointer / cursor position or a command to perform the desired action of the user.

[0609] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touch pad (not shown) for activating or deactivating certain functions. In some embodiments, the touch pad, unlike the touch screen, is a touch sensing area of the device that does not display visual output. The touch pad is optionally a separate touch sensing surface from the touch screen 112 or an extension of the touch sensing surface formed by the touch screen.

[0610] The device 100 also includes a power system 162 for powering various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharge system, a power outage detection circuit, a power converter or inverter, a power status indicator (e.g., light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power within a portable device.

[0611] Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor coupled to a light sensor controller 158 within the I / O subsystem 106. The light sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The light sensor 164 receives light from the environment projected through one or more lenses and converts the light into data representing an image. In conjunction with the imaging module 143 (also referred to as a camera module), the light sensor 164 optionally captures a still image or video. In some embodiments, the light sensor is disposed on the back of the device 100, which is opposite the touch screen 112 on the front of the device, so as to enable the touch-sensitive display to be used as a viewfinder for obtaining still images and / or videos. In some embodiments, another light sensor is disposed on the front of the device so as to optionally obtain an image of the user during a video conference while the user is viewing other video conference participants on the touch-sensitive display.

[0612] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to an intensity sensor controller 159 within the I / O subsystem 106. The contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric force sensors, optical force sensors, capacitive touch sensing surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch sensing surface). The contact intensity sensor 165 receives contact intensity information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is disposed on or proximate to a touch sensing surface (e.g., the touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is disposed on the back of the device 100, which is opposite the touch screen 112 disposed on the front of the device 100.

[0613] Device 100 also optionally includes one or more proximity sensors 166. FIG. 1A shows a proximity sensor 166 coupled to the peripheral device interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 within the I / O subsystem 106. In some embodiments, when a multifunctional device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and the touch screen 112 becomes inoperative.

[0614] Device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows a tactile output generator coupled to the tactile feedback controller 161 within the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert an electrical signal into a tactile output on the device). The contact intensity sensor 165 receives tactile feedback generation instructions from the tactile feedback module 133 and generates a tactile output on the device 100 that can be sensed by a user of the device 100. In some embodiments, at least one tactile output generator is disposed on or proximate to a touch sensing surface (e.g., the touch sensing display system 112), and optionally generates a tactile output by moving the touch sensing surface in a vertical direction (e.g., in / out of the surface of the device 100) or a lateral direction (e.g., back and forth in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is disposed on the back of the device 100, on the opposite side of the touch sensing display 112 disposed on the front of the device 100.

[0615] Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows an accelerometer 168 coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 within the I / O subsystem 106. In some embodiments, information is displayed on the touch-sensitive display in a portrait or landscape orientation based on an analysis of data received from one or more accelerometers. In addition to the accelerometer(s) 168, device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the position and orientation (e.g., portrait or landscape) of device 100.

[0616] In some embodiments, the software components stored in memory 102 include an operating system 126, a proactive module 163 (optionally including one or more application usage data tables 335, trigger condition tables 402, trigger establishment module 163-1, and / or usage data collection module 163-2), a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Further, in some embodiments, as shown in FIG. 1A, memory 102 stores a device / global internal state 157. The device / global internal state 157 includes one or more of the following: an active application state indicating which application is active if there is a currently active application, a display state indicating which application, view, or other information occupies various regions of the touch-sensitive display 112, a sensor state including information obtained from various sensors and input control devices 116 of the device, and position information regarding the position and / or orientation of the device (e.g., the orientation of the device).

[0617] The operating system 126 (e.g., an embedded operating system such as Darwin (registered trademark), RTXC (registered trademark), LINUX (registered trademark), UNIX (registered trademark), OS X (registered trademark), WINDOWS (registered trademark), or VxWorks (registered trademark)) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware components and software components.

[0618] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FIREWIRE (registered trademark), etc.) is adapted to connect directly or indirectly through a network (e.g., the Internet, a wireless LAN, etc.) to other devices. In some embodiments, the external port is the same as, or similar to, and / or compatible with a multi-pin (e.g., 30-pin) connector such as that used on some embodiments of the IPOD device from APPLE Inc. In other embodiments, the external port is the same as, or similar to, and / or compatible with a multi-pin (e.g., 8-pin) connector such as that used on the LIGHTNING connector from APPLE Inc.

[0619] The contact / motion module 130 optionally detects contact with the touch screen 112 and other touch-sensing devices (e.g., a touch pad or a physical click wheel), in conjunction with the display controller 156. The contact / motion module 130 includes various software components for performing various operations related to the detection of contact, such as determining whether contact has occurred (e.g., detecting an event of a finger being lowered), determining the intensity of the contact (e.g., the force or pressure of the contact, or an alternative to the force or pressure of the contact), determining whether there is movement of the contact, tracking movement across the touch-sensing surface (e.g., detecting one or more events of a finger being dragged), and determining whether the contact has stopped (e.g., detecting an event of a finger being raised or an interruption of the contact). The contact / motion module 130 receives contact data from the touch-sensing surface. Determining the movement of the contact point represented by a series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These operations are optionally applied to a single contact (e.g., contact with one finger) or multiple simultaneous contacts (e.g., "multi-touch" / contact with multiple fingers). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touch pad.

[0620] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by the user (e.g., to determine whether the user has selected or "clicked" on an affordance). In some embodiments, at least a subset of the intensity thresholds is determined according to software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of device 100). For example, the mouse "click" threshold of a trackpad or touch-sensitive display can be set to any of a wide range of default thresholds without changing the hardware of the trackpad or touch-sensitive display. Further, in some implementations, the user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level click "intensity" parameters).

[0621] The contact / motion module 130 optionally detects gesture inputs from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of detected contacts). Thus, gestures are optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture includes detecting an event of lowering a finger (e.g., at the location of an icon) and subsequently detecting an event of raising (lifting off) the finger at the same location (or substantially the same location) as the event of lowering the finger. As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting an event of lowering a finger and subsequently detecting one or more events of dragging the finger, and then, in some embodiments, subsequently detecting an event of raising (lifting off) the finger.

[0622] The graphic module 132 includes various known software components for rendering and displaying graphics on the touch screen 112 or other display, including components for changing the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual characteristics). As used herein, the term "graphic" includes any object that can be displayed to the user, such as text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like, but is not limited thereto.

[0623] In some embodiments, the graphic module 132 stores data representing the graphics used. Each graphic is optionally assigned a corresponding code. The graphic module 132 receives one or more codes specifying the graphics to be displayed, along with coordinate data and other graphic characteristic data as needed, from an application or the like, and then generates the image data of the screen to be output to the display controller 156.

[0624] The tactile feedback module 133 includes various software components for generating instructions to be used by the tactile output generator(s) 167 to generate tactile output at one or more locations on the device 100 in response to user interaction with the device 100.

[0625] The text input module 134 is optionally a component of the graphic module 132 and provides a soft keyboard for entering text in various applications (e.g., the contact module 137, the email client module 140, the IM module 141, the browser module 147, and any other application that requires text input).

[0626] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the application of the phone 138 for location-based dialing, to the application of the camera 143 as metadata for images / videos, and to applications that provide location-based services such as weather widgets, local yellow page widgets, and map / navigation widgets).

[0627] The applications ("apps") 136 optionally include the following modules (or sets of instructions), or subsets or supersets thereof. ● Contact module 137 (which may also be referred to as an address book or contact list), ● Phone module 138, ● Video conferencing module 139, ● Email client module 140, ● Instant message (IM) module 141, ● Health module 142, ● Camera module 143 for still and / or video images, ● Image management module 144, ● Browser module 147, ● Calendar module 148, ● Widget module 149 optionally includes one or more of weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widget 149-6, ● Search module 151, ● Video and music player module 152 optionally composed of a video player module and a music player module, ● Memo module 153, ● Map module 154, and / or ● Online video module 155.

[0628] Examples of other applications 136 that are optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, website creation applications, disk authoring applications, spreadsheet applications, JAVA (registered trademark)-compatible applications, encryption, digital rights management, voice recognition, widget creator module 149-6 for creating user-created widgets, and voice replication.

[0629] In conjunction with touch screen 112, display controller 156, contact module 130, graphic module 132, and text input module 134, contact module 137 is optionally used to manage an address book or contact list (e.g., stored in contact module 137 in memory 102 or memory 370), which includes adding names (singular or plural) to the address book, deleting names (singular or plural) from the address book, associating phone numbers (singular or plural), email addresses (singular or plural), addresses (singular or plural), or other information with names, associating images with names, classifying and sorting names, providing a phone number or email address to initiate and / or facilitate communication by phone module 138, video conferencing module 139, email module 140, or IM module 141, and the like.

[0630] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the phone module 138 is optionally used to input a series of characters corresponding to a phone number, access one or more phone numbers in the address book 137, modify the input phone number, dial the corresponding phone number, conduct a conversation, and disconnect the connection or hang up the phone when the conversation is completed. As described above, the wireless communication optionally uses any of a plurality of communication standards, protocols, and technologies.

[0631] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphic module 132, text input module 134, contact list 137, and phone module 138, the video conferencing module 139 includes executable instructions for starting, conducting, and ending a video conference between the user and one or more other participants according to the user's instructions.

[0632] In conjunction with the RF circuit 108, touch screen 112, display controller 156, contact module 130, graphic module 132, and text input module 134, the email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to the user's instructions. In conjunction with the image management module 144, the email client module 140 makes it very easy to create and send emails with still or video images captured by the camera module 143.

[0633] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphic module 132, and text input module 134, instant message module 141 includes executable instructions for entering a string corresponding to an instant message, modifying entered characters, transmitting the corresponding instant message (e.g., using the Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocol for phone-based instant messages, or using XMPP, SIMPLE, or IMPS for Internet-based instant messages), receiving an instant message, and displaying the received instant message. In some embodiments, the instant messages sent and / or received optionally include graphics, photos, audio files, video files, and / or other attached files, such as those supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant message" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0634] In conjunction with RF circuitry 108, touch screen 112, display controller 156, contact module 130, graphic module 132, text input module 134, GPS module 135, map module 154, and video and music player module 146, health module 142 includes executable instructions for creating a workout (e.g., with respect to time, distance, and / or calorie consumption goals), communicating with a workout sensor (such as a sports device like a wristwatch or pedometer), receiving workout sensor data, calibrating sensors used to monitor a workout, selecting and playing music for a workout, and displaying, storing, and transmitting workout data.

[0635] In conjunction with touch screen 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphic module 132, and image management module 144, camera module 143 includes executable instructions for capturing a still image or video (including a video stream), storing them in memory 102, modifying features of the still image or video, or deleting the still image or video from memory 102.

[0636] In conjunction with touch screen 112, display controller 156, contact module 130, graphic module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide show or album), and storing still images and / or video images.

[0637] In conjunction with RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphic module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet in accordance with user instructions, including searching for, linking to, receiving, and displaying web pages or portions thereof, and attached and other files linked to the web pages.

[0638] In conjunction with RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphic module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions for creating, displaying, modifying, and storing a calendar and data associated with the calendar (e.g., calendar items, ToDo lists, etc.) in accordance with user instructions.

[0639] In combination with RF circuit 108, touch screen 112, display system controller 156, touch module 130, graphic module 132, text input module 134, and browser module 147, widget module 149 is a mini-application (e.g., weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) that is optionally downloaded and used by the user, or a mini-application (e.g., user-created widget 149-6) created by the user. In some embodiments, the widget includes an HTML (HyperText Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript (registered trademark) file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! (registered trademark) widget).

[0640] In combination with RF circuit 108, touch screen 112, display system controller 156, touch module 130, graphic module 132, text input module 134, and browser module 147, a widget creator module (not shown) is optionally used by the user to create a widget (e.g., change a user-specified location on a web page to a widget).

[0641] In conjunction with touch screen 112, display system controller 156, contact module 130, graphic module 132, and text input module 134, search module 151 includes executable instructions for searching memory 102 for text, music, sounds, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search phrases) in accordance with a user's instructions. In some embodiments, search module 151 further includes executable instructions for displaying a search entry portion and a prediction portion (e.g., search entry portion 920 and prediction portion 930, described in detail below in connection with FIGS. 9B and 6A-9C). In some embodiments, search module 151 also, in conjunction with proactive module 163, populates the prediction portion with affordances for suggested or predicted people, actions within an application, applications, nearby locations, and / or news articles prior to receiving user input at the search entry portion (as described in detail below in connection with FIGS. 3A-9C).

[0642] In conjunction with touch screen 112, display system controller 156, contact module 130, graphic module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, video and music player module 152 includes executable instructions that enable a user to download and play recorded music or other sound files stored in one or more file formats such as MP3 or AAC files, and executable instructions for displaying, presenting, or otherwise playing videos (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally comprises the functionality of an MP3 player, such as an IPOD® from APPLE Inc.

[0643] In conjunction with the touch screen 112, the display controller 156, the touch module 130, the graphic module 132, and the text input module 134, the memo module 153 includes executable instructions for creating and managing memos, ToDo lists, etc. according to user instructions.

[0644] In conjunction with the RF circuit 108, the touch screen 112, the display system controller 156, the touch module 130, the graphic module 132, the text input module 134, the GPS module 135, and the browser module 147, the map module 154 can optionally be used to receive, display, modify, and store maps and data associated with the maps (e.g., driving instructions, data about stores near a particular location or other points of interest, and other location-based data) according to user instructions.

[0645] In conjunction with the touch screen 112, the display system controller 156, the touch module 130, the graphic module 132, the audio circuit 110, the speaker 111, the RF circuit 108, the text input module 134, the email client module 140, and the browser module 147, the online video module 155 includes instructions that enable a user to access, browse, receive (e.g., via streaming and / or downloading), play (e.g., on the touch screen or on an external display connected via the external port 124), send emails containing links to specific online videos, and manage them in other ways. In some embodiments, instead of the email client module 140, the instant message module 141 is used to send links to specific online videos.

[0646] As shown in FIG. 1A, the portable multifunctional device 100 also includes a proactive module 163 for proactively identifying relevant content and surfacing (e.g., surfacing user interface objects corresponding to actions within an application (e.g., a UI object for playing a playlist within a music app) on the lock screen or within a search interface). The proactive module 163 optionally includes the following modules (or sets of instructions), or subsets or supersets thereof. ● Application usage table 335 ● Trigger condition table 402 ● Trigger establishment module 163-1 ● Usage data collection module 163-2 ● Proactive proposal module 163-3 ● (Voice communication) content extraction module 163-4

[0647] In conjunction with application 136, GPS module 135, operating system 126, I / O subsystem 106, RF circuitry 108, external port 124, proximity sensor 166, audio circuitry 110, accelerometer 168, speaker 111, microphone 113, and peripheral device interface 118, application usage table 335 and usage data collection module 163-2 receive and / or store application usage data (e.g., from the components of device 100 identified in FIG. 1A above). In some embodiments, the usage of an application is reported to usage data collection module 163-2 and subsequently stored in application usage table 335. In some embodiments, application usage data includes all (or the most important, relevant, or predictive) contextual usage information corresponding to the usage by a user of a particular application 136. In some embodiments, each particular application stores usage data while the user is interacting with that application, and that usage data is subsequently reported to application usage data table 335 for storage (e.g., usage data 193 for a particular application 136-1 in FIG. 3B includes all sensor readings, in-application actions executed, device connection information, etc., and this usage data 193 is sent to application usage table 335 for storage as a record in the table). For example, while the user is interacting with browser module 147, application usage data receives and stores all contextual usage information including the current GPS coordinates of device 100 (e.g., determined by GPS module 135), motion data (e.g., determined by accelerometer 168), ambient light data (e.g., determined by light sensor 164), and in-application actions executed by ...

Claims

1. A non-transitory computer-readable storage medium storing executable instructions, which, when executed by an electronic device having a touch-sensitive display, cause the electronic device to execute an application on the electronic device in response to an instruction from a user of the electronic device, collect usage data while the application is being executed, the usage data including at least one or more actions performed by the user within the application, automatically obtain at least one trigger condition based on the collected usage data without human intervention, associate the at least one trigger condition with a specific action among the one or more actions performed by the user within the application, and when it is determined that the at least one trigger condition is met, provide an indication to the user that the specific action associated with the trigger condition is available. A non-transitory computer-readable storage medium.

2. A non-transitory computer-readable storage medium storing executable instructions, which, when executed by an electronic device having a touch-sensitive display, cause the electronic device to obtain at least one trigger condition based on usage data associated with a user of the electronic device, the usage data including one or more actions performed by the user within the application while the application is being executed on the electronic device, associate the at least one trigger condition with a specific action among the one or more actions performed by the user within the application, and when it is determined that the at least one trigger condition is met, provide an indication to the user that the specific action associated with the trigger condition is available. A non-transitory computer-readable storage medium.

3. The non-transitory computer-readable storage medium according to claim 1 or 2, wherein obtaining the at least one trigger condition includes transmitting the usage data to one or more servers located remotely from the electronic device and receiving the at least one trigger condition from the one or more servers.

4. The providing of the indication includes displaying, on a lock screen of the touch sensing display, a user interface object corresponding to the specific action associated with the trigger condition, the non-transitory computer-readable storage medium according to any one of claims 1 to 3.

5. The user interface object includes an explanation of the specific action associated with the trigger condition, the non-transitory computer-readable storage medium according to claim 4.

6. The user interface object further includes an icon associated with the application, the non-transitory computer-readable storage medium according to claim 5.

7. The executable instructions further cause the electronic device to detect a first gesture in the user interface object, in response to detecting the first gesture, display the application on the touch sensing display, while the application is being displayed, execute the specific action associated with the trigger condition, the non-transitory computer-readable storage medium according to any one of claims 4 to 6.

8. The first gesture is a swipe gesture on the user interface object, the non-transitory computer-readable storage medium according to claim 7.

9. The executable instructions further cause the electronic device to detect a second gesture in the user interface object, in response to detecting the second gesture and while the lock screen continues to be displayed on the touch sensing display, execute the specific action associated with the trigger condition, the non-transitory computer-readable storage medium according to any one of claims 4 to 6.

10. The second gesture is a single tap in a default area of the user interface object, the non-transitory computer-readable storage medium according to claim 9.

11. The user interface object is displayed at a default central portion of the lock screen, the non-transitory computer-readable storage medium according to any one of claims 4 to 10.

12. Providing the indication to the user that the specific action associated with the trigger condition is available includes executing the specific action, the non-transitory computer-readable storage medium according to claim 1 or 2.

13. The user interface object is an icon associated with the application, and further, the user interface object is substantially displayed at a corner of the lock screen of the touch-sensitive display, the non-transitory computer-readable storage medium according to claim 4.

14. The executable instructions further cause the electronic device to receive from the user an instruction to unlock the electronic device, in response to receiving the instruction, display a home screen of the electronic device on the touch-sensitive display, and provide on the home screen an indication to the user that the specific action associated with the trigger condition is available, the non-transitory computer-readable storage medium according to any one of claims 1 to 13.

15. The home screen includes (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device, and (ii) a second portion displayed adjacent to the first portion for launching a second set of applications available on the electronic device, the second portion being displayed on all user interface pages included in the first portion, and providing the indication on the home screen includes displaying the indication on the second portion, the non-transitory computer-readable storage medium according to claim 14.

16. The second set of applications is different from the first set of applications and smaller than the first set of applications, the non-transitory computer-readable storage medium according to claim 15.

17. Determining that the at least one trigger condition is satisfied includes determining that the electronic device is coupled to a second device different from the electronic device, the non-transitory computer-readable storage medium according to any one of claims 1 to 16.

18. Determining that the at least one trigger condition is met includes determining that the electronic device has arrived at an address corresponding to the home or workplace associated with the user, the non-transitory computer-readable storage medium according to any one of claims 1 to 17.

19. Determining that the electronic device has arrived at an address corresponding to the home or workplace associated with the user includes monitoring motion data from an accelerometer of the electronic device and determining, based on the monitored motion data, that the electronic device has not moved for a threshold amount of time, the non-transitory computer-readable storage medium according to claim 18.

20. The usage data further includes verbal instructions provided to the virtual assistant application by the user while the application is running, and further, the at least one trigger condition is further based on the verbal instructions provided to the virtual assistant application, the non-transitory computer-readable storage medium according to any one of claims 1 to 19.

21. The verbal instructions include a request to create a reminder corresponding to the current state of the application, and the current state corresponds to the state of the application when the verbal instructions are provided, the non-transitory computer-readable storage medium according to claim 20.

22. The state of the application when the verbal instructions are provided is selected from the group consisting of a page displayed within the application when the verbal instructions are provided, content being played within the application when the verbal instructions are provided, a notification displayed within the application when the verbal instructions are provided, and an active portion of the page displayed within the application when the verbal instructions are provided, the non-transitory computer-readable storage medium according to claim 21.

23. The verbal instructions include the word "this" regarding the current state of the application, the non-transitory computer-readable storage medium according to claim 21.

24. In one or more electronic devices In response to an instruction from a user of the first electronic device, execute an application on the first electronic device among the one or more electronic devices; While the application is being executed, automatically and without human intervention, collect usage data, where the usage data at least includes one or more actions executed by the user within the application; Based on the collected usage data, automatically and without human intervention, set at least one trigger condition; Associate the at least one trigger condition with a specific action among the one or more actions executed by the user within the application; When it is determined that the at least one trigger condition is satisfied, provide an indication to the user that the specific action associated with the trigger condition is available; A method comprising the above. **Claim 25** An electronic device, comprising: A touch-sensitive display; One or more processors; A memory storing one or more programs; When the one or more programs are executed by the one or more processors, cause the electronic device to: Execute an application on the electronic device in response to an instruction from a user of the electronic device; While the application is being executed, collect usage data, where the usage data at least includes one or more actions executed by the user within the application; Based on the collected usage data, automatically and without human intervention, obtain at least one trigger condition; Associate the at least one trigger condition with a specific action among the one or more actions executed by the user within the application; When it is determined that the at least one trigger condition is satisfied, provide an indication to the user that the specific action associated with the trigger condition is available; An electronic device. **Claim 26** In an electronic device having a touch-sensitive display, In response to an instruction from a user of the electronic device, execute an application on the electronic device; Collecting usage data while the application is running, the usage data including at least one or more actions performed by the user within the application, Automatically obtaining at least one trigger condition based on the collected usage data without human intervention, Associating the at least one trigger condition with a specific action among the one or more actions performed by the user within the application, When it is determined that the at least one trigger condition is satisfied, providing an indication to the user that the specific action associated with the trigger condition is available, A method including the above.

27. A non-transitory computer-readable storage medium storing executable instructions, which when executed by an electronic device touch-sensitive display, cause the electronic device to, Detect a search activation gesture of a user of the electronic device on the touch-sensitive display, In response to detecting the search activation gesture, display a search interface on the touch-sensitive display, the search interface including, (i) A search entry portion, (ii) A prediction portion displayed before receiving any user input in the search entry portion, The prediction portion including, (a) At least one affordance for contacting a person among a plurality of previously contacted persons, the person being automatically selected at least in part from among the plurality of previously contacted persons based on the current time, at least one affordance, (b) At least one affordance for performing a predicted action within an application among a plurality of applications available on the electronic device, the predicted action being automatically selected at least in part based on the application usage history associated with the user of the electronic device, at least one affordance, Populated with one or more of the above, A non-transitory computer-readable storage medium.

28. The non-transitory computer-readable storage medium according to claim 27, wherein the person is further selected at least in part based on location data corresponding to the electronic device.

29. The non - transitory computer - readable storage medium according to claim 27 or 28, wherein the application usage history and the contact information for the person are retrieved from the memory of the electronic device.

30. The non - transitory computer - readable storage medium according to claim 27 or 28, wherein the application usage history and the contact information for the person are retrieved from a server located remotely from the electronic device.

31. The prediction part is further populated with at least one affordance for executing the predicted application, and the predicted application is automatically selected at least in part based on the application usage history, the non - transitory computer - readable storage medium according to any one of claims 27 to 30.

32. The prediction part is further populated with at least one affordance of a predicted location category, and the predicted location category is automatically selected at least in part based on the current time and one or more of the location data corresponding to the electronic device, the non - transitory computer - readable storage medium according to any one of claims 27 to 31.

33. The executable instructions further cause the electronic device to detect a user input for scrolling the prediction part, in response to detecting the user input for scrolling the prediction part, scroll the prediction part according to the user input, in response to the scrolling, display at least one affordance of a predicted news article in the prediction part, the non - transitory computer - readable storage medium according to any one of claims 27 to 32.

34. The predicted news article is automatically selected at least in part based on the location data corresponding to the electronic device, the non - transitory computer - readable storage medium according to claim 33.

35. The executable instructions further cause the electronic device to detect a selection of the at least one affordance for executing the predicted action within the application, in response to detecting the selection, display the application on the touch - sensitive display and execute the predicted action within the application, the non - transitory computer - readable storage medium according to any one of claims 27 to 34.

36. The executable instructions further cause the electronic device to detect a selection of the at least one affordance for contacting the person, and in response to detecting the selection, cause the person to be contacted using the contact information for the person. The non-transitory computer-readable storage medium according to claim 29 or 30.

37. The executable instructions further cause the electronic device to detect a selection of the at least one affordance for executing the predicted application, and in response to detecting the selection, cause the predicted application to be displayed on the touch-sensitive display. The non-transitory computer-readable storage medium according to claim 31.

38. The executable instructions further cause the electronic device to detect a selection of the at least one affordance of the predicted location category, and in response to detecting the selection, cause data corresponding to at least one nearby location to be received, and cause the received data corresponding to the at least one nearby location to be displayed on the touch-sensitive display. The non-transitory computer-readable storage medium according to claim 32.

39. The executable instructions further cause the electronic device to detect a selection of the at least one affordance of the predicted news article, and in response to detecting the selection, cause the predicted news article to be displayed on the touch-sensitive display. The non-transitory computer-readable storage medium according to claim 33.

40. The search activation gesture is available from at least two different user interfaces, and further, a first user interface of the at least two different user interfaces corresponds to displaying a corresponding home screen page of a series of home screen pages on the touch-sensitive display. The non-transitory computer-readable storage medium according to any one of claims 27 to 39.

41. If the corresponding home screen page is the first home screen page among the series of home screen pages, the search activation gesture is one of the following, namely, (i) a gesture that moves substantially downward with respect to the user of the electronic device, or (ii) a continuous gesture that moves in a substantially left-to-right direction with respect to the user and substantially perpendicular to the downward direction. The non-transitory computer-readable storage medium according to claim 40.

42. If the corresponding home screen page is the second home screen page among the series of home screen pages, the search activation gesture includes the continuous gesture that moves substantially downward with respect to the user of the electronic device. The non-transitory computer-readable storage medium according to claim 41.

43. The second user interface among the at least two different user interfaces corresponds to displaying an application switching interface on the touch-sensitive display. The non-transitory computer-readable storage medium according to claim 40.

44. The search activation gesture includes contact on a predetermined search activation portion of the application switching interface on the touch-sensitive display. The non-transitory computer-readable storage medium according to claim 43.

45. An electronic device, A touch-sensitive display, One or more processors, A memory for storing one or more programs, Comprising, when the one or more programs are executed by the one or more processors, causing the electronic device to, Detect a search activation gesture of a user of the electronic device on the touch-sensitive display, In response to detecting the search activation gesture, display a search interface on the touch-sensitive display, and the search interface includes, (i) A search entry portion, (ii) A prediction portion that is displayed before receiving any user input in the search entry portion, Including, and the prediction portion is, (a) At least one affordance for contacting a person among a plurality of previously contacted persons, wherein the person is automatically selected, at least in part, based on the current time from among the plurality of previously contacted persons; at least one affordance; (b) At least one affordance for performing a predicted action within an application among a plurality of applications available on the electronic device, wherein the predicted action is automatically selected, at least in part, based on an application usage history associated with the user of the electronic device; at least one affordance; Populated with one or more of An electronic device.

46. In an electronic device having a touch-sensitive display, Detecting a search activation gesture of a user of the electronic device on the touch-sensitive display; In response to detecting the search activation gesture, displaying a search interface on the touch-sensitive display; And the search interface includes (i) A search entry portion; (ii) A prediction portion displayed before receiving any user input in the search entry portion; And the prediction portion includes (a) At least one affordance for contacting a person among a plurality of previously contacted persons, wherein the person is automatically selected, at least in part, based on the current time from among the plurality of previously contacted persons; at least one affordance; (b) At least one affordance for performing a predicted action within an application among a plurality of applications available on the electronic device, wherein the predicted action is automatically selected, at least in part, based on an application usage history associated with the user of the electronic device; at least one affordance; A method populated with one or more of.

47. An electronic device, A touch-sensitive display unit configured to display a user interface and receive a user's contact; A processing unit coupled to the touch-sensitive display unit; And the processing unit includes Execute an application on the electronic device in response to a command from a user of the electronic device, While the application is being executed, collect usage data, where the usage data includes at least one or more actions performed by the user within the application, Based on the collected usage data, automatically obtain at least one trigger condition without human intervention, Associate the at least one trigger condition with a specific action among the one or more actions performed by the user within the application, When it is determined that the at least one trigger condition is satisfied, provide an indication to the user that the specific action associated with the trigger condition is available, An electronic device configured as described above.

48. Obtaining the at least one trigger condition includes transmitting the usage data to one or more servers located remotely from the electronic device and receiving the at least one trigger condition from the one or more servers. The electronic device according to claim 47.

49. Providing the indication includes displaying a user interface object corresponding to the specific action associated with the trigger condition on a lock screen of the touch-sensitive display unit. The electronic device according to claim 47 or 48.

50. The user interface object includes an explanation of the specific action associated with the trigger condition. The electronic device according to claim 49.

51. The user interface object further includes an icon associated with the application. The electronic device according to claim 50.

52. The processing unit, Detect a first gesture in the user interface object, In response to detecting the first gesture, (i) Display the application on the touch-sensitive display unit, (ii) While the application is being displayed, execute the specific action associated with the trigger condition. The electronic device according to any one of claims 49 to 51, further configured as described above.

53. The electronic device according to claim 52, wherein the first gesture is a swipe gesture on the user interface object.

54. The processing unit detects a second gesture in the user interface object, and in response to detecting the second gesture and while the lock screen is being continuously displayed on the touch sensing display unit, executes the specific action associated with the trigger condition. The electronic device according to any one of claims 49 to 52, further configured as such.

55. The electronic device according to claim 54, wherein the second gesture is a single tap in a default area of the user interface object.

56. The electronic device according to any one of claims 49 to 55, wherein the user interface object is displayed at a default central part of the lock screen.

57. Providing the indication to the user that the specific action associated with the trigger condition is available includes executing the specific action, for the electronic device according to claim 47.

58. The electronic device according to claim 49, wherein the user interface object is an icon associated with the application, and the user interface object is substantially displayed at a corner of the lock screen of the touch sensing display unit.

59. The processing unit receives a command from the user to unlock the electronic device, and in response to receiving the command, displays the home screen of the electronic device on the touch sensing display unit, and provides the indication to the user on the home screen that the specific action associated with the trigger condition is available. The electronic device according to any one of claims 47 to 58, further configured as such.

60. The home screen includes: (i) a first portion including one or more user interface pages for launching a first set of applications available on the electronic device; and (ii) a second portion, adjacent to the first portion, for launching a second set of applications available on the electronic device, the second portion being displayed on all user interface pages included in the first portion, and providing the indication on the home screen includes displaying the indication on the second portion. The electronic device according to claim 59.

61. The electronic device according to claim 60, wherein the second set of applications is different from the first set of applications and smaller than the first set of applications.

62. Determining that the at least one trigger condition is satisfied includes determining that the electronic device is coupled to a second device different from the electronic device. The electronic device according to any one of claims 47 to 61.

63. Determining that the at least one trigger condition is satisfied includes determining that the electronic device has arrived at an address corresponding to a home or workplace associated with the user. The electronic device according to any one of claims 47 to 62.

64. Determining that the electronic device has arrived at an address corresponding to the home or workplace associated with the user includes monitoring motion data from an accelerometer of the electronic device and determining, based on the monitored motion data, that the electronic device has not moved for a threshold amount of time. The electronic device according to claim 63.

65. The usage data further includes verbal commands provided to the virtual assistant application by the user while the application is being continuously executed. Further, the at least one trigger condition is further based on the verbal commands provided to the virtual assistant application. The electronic device according to any one of claims 47 to 64.

66. The oral command includes a request to create a reminder corresponding to the current state of the application, the current state corresponding to the state of the application when the oral command was provided, the electronic device according to claim 65.

67. The state of the application when the oral command was provided is selected from the group consisting of a page displayed within the application when the oral command was provided, content being played within the application when the oral command was provided, a notification displayed within the application when the oral command was provided, and an active portion of the page displayed within the application when the oral command was provided, the electronic device according to claim 66.

68. The oral command includes the word "this" regarding the current state of the application, the electronic device according to claim 66.

69. A non-transitory computer-readable storage medium storing executable instructions, which when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to, receive at least a portion of a voice communication, the portion of the voice communication including speech provided by a remote user of a remote device different from a user of the electronic device, extract content items, at least in part, based on the speech provided by the remote user of the remote device, determine whether the content items are currently available on the electronic device, in accordance with a determination that the content items are not currently available on the electronic device, identify an application associated with the content items, display a selectable description of the content items on the display, store the content items for presentation in the identified application in response to detecting a selection of the selectable description. Non-transitory computer-readable storage medium.

70. The content items are new events, the non-transitory computer-readable storage medium according to claim 69.

71. The non-transitory computer-readable storage medium according to claim 69, wherein the content item is new event details of an event currently associated with a calendar application on the electronic device.

72. The non-transitory computer-readable storage medium according to claim 69, wherein the content item is a new contact.

73. The non-transitory computer-readable storage medium according to claim 69, wherein the content item is new contact information of an existing contact associated with a phone application on the electronic device.

74. The non-transitory computer-readable storage medium according to any one of claims 69 to 73, wherein the voice communication is a live call.

75. The non-transitory computer-readable storage medium according to any one of claims 69 to 73, wherein the voice communication is a live FaceTime call.

76. The non-transitory computer-readable storage medium according to any one of claims 69 to 73, wherein the voice communication is a recorded voicemail.

77. The non-transitory computer-readable storage medium according to any one of claims 69 to 76, wherein displaying the selectable description includes displaying the selectable description including a most recent call made using a phone application within a user interface.

78. The non-transitory computer-readable storage medium according to claim 77, wherein the selectable description is displayed with an indication that the content item is associated with the voice communication.

79. The non-transitory computer-readable storage medium according to claim 77, wherein detecting the selection includes receiving the selection while the user interface including the most recent call is being displayed.

80. When the executable instructions are executed by the electronic device, further cause the electronic device to provide feedback to the user of the electronic device that the content item has been detected, in addition to displaying the selectable description of the content item. The non-transitory computer-readable storage medium according to any one of claims 69 to 79.

81. The non - transitory computer - readable storage medium according to claim 80, wherein providing feedback includes transmitting information regarding detection of the content item to a different electronic device in the vicinity of the electronic device.

82. When the executable instructions are executed by the electronic device, further cause the electronic device to determine that the voice communication includes information regarding a first physical location, detect an input, in response to detecting the input, open an application capable of receiving location data and populate the application with the information regarding the first physical location. The non - transitory computer - readable storage medium according to any one of claims 69 to 81.

83. The application is a map application, and populating the map application with the information regarding the first physical location includes populating a map displayed within the map application with a location identifier corresponding to the first physical location. The non - transitory computer - readable storage medium according to claim 82.

84. When the executable instructions are executed by the electronic device, further cause the electronic device to determine that the voice communication includes information regarding a first physical location, detect an input, in response to detecting the input, populate a search interface with the information regarding the first physical location. The non - transitory computer - readable storage medium according to any one of claims 69 to 81.

85. Extracting the content item includes analyzing the portion of the voice communication for detection of a predefined type of content, and the analysis is performed while the voice communication is output via an audio system communicating with the electronic device. The non - transitory computer - readable storage medium according to any one of claims 69 to 84.

86. Analyzing the voice communication includes converting the speech provided by the remote user of the remote device to text, and applying a natural - language - processing algorithm to the text to determine whether the text includes one or more predefined keywords. Determining that the voice communication includes speech that describes the content item, according to a determination that the text includes a defined keyword to which the text corresponds; The non-transitory computer-readable storage medium according to claim 85, comprising: **Claim 87** The non-transitory computer-readable storage medium according to any one of claims 69 to 86, wherein receiving at least the part of the voice communication includes receiving an indication from a user of the electronic device that the part of the voice communication should be analyzed. **Claim 88** The non-transitory computer-readable storage medium according to claim 87, wherein the indication corresponds to a selection of a hardware button. **Claim 89** The non-transitory computer-readable storage medium according to claim 87, wherein the indication corresponds to a command including the phrase "hey Siri" from a user of the electronic device. **Claim 90** When the executable instructions are executed by the electronic device, the electronic device receives a second part of the voice communication, the second part including speech provided by the remote user of the remote device and speech provided by the user of the electronic device; extracts a second content item, at least in part, based on the speech provided by the remote user of the remote device and the speech provided by the user of the electronic device; according to a determination that the second content item is not currently available on the electronic device, identifies a second application associated with the second content item; displays a second selectable description of the second content item on the display; stores the second content item for presentation in the identified second application in response to detecting a selection of the second selectable description. The non-transitory computer-readable storage medium according to any one of claims 69 to 89. **Claim 91** The non-transitory computer-readable storage medium according to claim 90, wherein the selectable description and the second selectable description are displayed within a user interface, including recent calls made using a phone application. **Claim 92** In an electronic device having one or more processors, a memory, a touch-sensitive surface, and a display, receiving at least a portion of a voice communication, the portion of the voice communication including speech provided by a remote user of a remote device different from a user of the electronic device; extracting a content item, at least in part, based on the speech provided by the remote user of the remote device; determining whether the content item is currently available on the electronic device; in accordance with a determination that the content item is not currently available on the electronic device, identifying an application associated with the content item; displaying a selectable description of the content item on the display; in response to detecting a selection of the selectable description, storing the content item for presentation by the identified application; A method comprising.

93. The method of claim 92, wherein the content item is a new event.

94. The method of claim 92, wherein the content item is new event details of an event currently associated with a calendar application on the electronic device.

95. The method of claim 92, wherein the content item is a new contact.

96. The method of claim 92, wherein the content item is new contact information of an existing contact associated with a phone application on the electronic device.

97. The method according to any one of claims 92 to 96, wherein the voice communication is a live call.

98. The method according to any one of claims 92 to 96, wherein the voice communication is a live FaceTime call.

99. The method according to any one of claims 92 to 96, wherein the voice communication is a recorded voicemail.

100. The method according to any one of claims 92 to 99, wherein displaying the selectable description includes displaying the selectable description, within a user interface, including recent calls made using a phone application.

101. The method of claim 100, wherein the selectable description is displayed with an indication that the content item is associated with the voice communication.

102. The method of claim 100, wherein detecting the selection includes receiving the selection while the user interface including the most recent call is being displayed.

103. The method according to any one of claims 92 to 102, further comprising providing feedback to the user of the electronic device that the content item has been detected, in conjunction with displaying the selectable description of the content item.

104. The method of claim 103, wherein providing feedback includes transmitting information regarding detection of the content item to a different electronic device in the vicinity of the electronic device.

105. determining that the voice communication includes information regarding a first physical location; detecting an input; in response to detecting the input, opening an application capable of receiving location data and populating the application with the information regarding the first physical location; The method according to any one of claims 92 to 104, further comprising.

106. The method of claim 105, wherein the application is a map application, and populating the map application with the information regarding the first physical location includes populating a map displayed within the map application with a location identifier corresponding to the first physical location.

107. determining that the voice communication includes information regarding a first physical location; detecting an input; in response to detecting the input, populating a search interface with the information regarding the first physical location; The method according to any one of claims 92 to 104, further comprising.

108. The method according to any one of claims 92 to 107, wherein extracting the content item includes analyzing the portion of the voice communication for detection of a predetermined type of content, the analysis being performed while the voice communication is being output via an audio system communicating with the electronic device.

109. Analyzing the voice communication includes Converting the speech provided by the remote user of the remote device into text; Applying a natural language processing algorithm to the text to determine whether the text contains one or more defined keywords; Determining that the voice communication includes speech describing the content item according to the determination that the text contains the corresponding defined keyword; The method according to claim 108, comprising.

110. The method according to any one of claims 92 to 109, wherein receiving at least a portion of the voice communication includes receiving an indication from a user of the electronic device that the portion of the voice communication is to be analyzed.

111. The method according to claim 110, wherein the indication corresponds to a selection of a hardware button.

112. The method according to claim 110, wherein the indication corresponds to a command including the word "hey Siri" from a user of the electronic device.

113. Receiving a second portion of the voice communication, the second portion including speech provided by the remote user of the remote device and speech provided by the user of the electronic device; Extracting a second content item based at least in part on the speech provided by the remote user of the remote device and the speech provided by the user of the electronic device; According to the determination that the second content item is not currently available on the electronic device, Identifying a second application associated with the second content item; Displaying a second selectable description of the second content item on the display; In response to detecting a selection of the second selectable description, storing the second content item for presentation by the identified second application; The method according to any one of claims 92 to 112, further comprising.

114. The method according to claim 113, wherein the selectable description and the second selectable description are displayed within a user interface, including recent calls made using a phone application.

115. A touch sensing surface, a display, one or more processors, a memory storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs include instructions to execute the method according to any one of claims 92 to 114, An electronic device.

116. A graphical user interface on an electronic device having a memory, a touch sensing surface, a display, and one or more processors for executing one or more programs stored in the memory, wherein the graphical user interface comprises a user interface displayed according to the method according to any one of claims 92 to 114.

117. An electronic device having a touch sensing surface and a display, the electronic device comprising means for executing the method according to any one of claims 92 to 114.

118. An information processing apparatus for use in an electronic device including a touch sensing surface and a display, the information processing apparatus comprising means for executing the method according to any one of claims 92 to 114.

119. A touch sensing surface, a display, one or more processors, a memory storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs include instructions to execute the method according to any one of claims 92 to 114, An electronic device.

120. A graphical user interface on an electronic device having a memory, a touch sensing surface, a display, and one or more processors for executing one or more programs stored in the memory, wherein the graphical user interface comprises a user interface displayed according to the method according to any one of claims 92 to 114.

121. An electronic device having a touch sensing surface and a display, the electronic device comprising means for executing the method according to any one of claims 92 to 114.

122. An information processing apparatus for use in an electronic device including a touch sensing surface and a display, the information processing apparatus comprising means for performing the method according to any one of claims 92 to 114.

123. An electronic device, a touch sensing surface unit configured to receive contact from a user, a display unit configured to display a user interface, and a processing unit coupled to the touch sensing surface unit and the display unit, wherein the processing unit receives at least a portion of a voice communication, the portion of the voice communication including speech provided by a remote user of a remote device different from a user of the electronic device, extracts content items, at least in part, based on the speech provided by the remote user of the remote device, determines whether the content items are currently available on the electronic device, in accordance with a determination that the content items are not currently available on the electronic device, identifies an application associated with the content items, displays selectable descriptions of the content items on the display unit, and stores the content items for presentation by the identified application in response to detecting a selection of the selectable descriptions. A non-transitory computer-readable storage medium.

124. The electronic device according to claim 123, wherein the content item is a new event.

125. The electronic device according to claim 123, wherein the content item is new event details of an event currently associated with a calendar application on the electronic device.

126. The electronic device according to claim 123, wherein the content item is a new contact.

127. The electronic device according to claim 123, wherein the content item is new contact information of an existing contact associated with a phone application on the electronic device.

128. The electronic device according to any one of claims 123 to 127, wherein the voice communication is a live call.

129. The electronic device according to any one of claims 123 to 127, wherein the voice communication is a live FaceTime call.

130. The electronic device according to any one of claims 123 to 127, wherein the voice communication is recorded voicemail.

131. The electronic device according to any one of claims 123 to 130, wherein displaying the selectable description includes displaying the selectable description including a most recent call made using a phone application within a user interface.

132. The electronic device according to claim 131, wherein the selectable description is displayed with an indication that the content item is associated with the voice communication.

133. The electronic device according to claim 131, wherein detecting the selection includes receiving the selection while the user interface including the most recent call is being displayed.

134. The processing unit is further configured to provide feedback to the user of the electronic device that the content item has been detected, in addition to displaying the selectable description of the content item, according to any one of claims 123 to 133.

135. The electronic device according to claim 134, wherein providing feedback includes transmitting information regarding detection of the content item to a different electronic device in the vicinity of the electronic device.

136. The processing unit is determined that the voice communication includes information regarding a first physical location, detected an input, in response to detecting the input, open an application capable of receiving location data and populate the application with the information regarding the first physical location, further configured as described in any one of claims 123 to 135.

137. The electronic device according to claim 136, wherein the application is a map application, and populating the map application with the information regarding the first physical location includes populating a map displayed within the map application with a location identifier corresponding to the first physical location.

138. The processing unit is determined that the voice communication includes information regarding a first physical location, detected an input, In response to detecting the input, the search interface is populated with information regarding the first physical location. The electronic device according to any one of claims 123 to 135, further configured as such.

139. Extracting the content item includes analyzing the part of the voice communication for detecting a predefined type of content, and the analysis is performed while outputting the voice communication via an audio system communicating with the electronic device. The electronic device according to any one of claims 123 to 138.

140. Analyzing the voice communication includes: Converting the speech provided by the remote user of the remote device into text; Applying a natural language processing algorithm to the text to determine whether the text contains one or more predefined keywords; Determining that the voice communication includes speech explaining the content item according to the determination that the text contains the corresponding predefined keyword; The electronic device according to claim 139, including the above.

141. Receiving at least the part of the voice communication includes receiving an indication from the user of the electronic device that the part of the voice communication should be analyzed. The electronic device according to any one of claims 123 to 140.

142. The indication corresponds to the selection of a hardware button. The electronic device according to claim 141.

143. The indication corresponds to a command including the word "hey Siri" from the user of the electronic device. The electronic device according to claim 141.

144. The processing unit: Receives a second part of the voice communication, the second part including speech provided by the remote user of the remote device and speech provided by the user of the electronic device; Extracts a second content item based at least in part on the speech provided by the remote user of the remote device and the speech provided by the user of the electronic device. In accordance with a determination that the second content item is not currently available on the electronic device, identify a second application associated with the second content item, display a second selectable description of the second content item on the display unit, in response to detecting a selection of the second selectable description, store the second content item for presentation by the identified second application, The electronic device according to any one of claims 123 to 143, further configured as described above.

145. The electronic device according to claim 144, wherein the selectable description and the second selectable description including a most recent call made using a telephone application are displayed within a user interface.

146. A non-transitory computer-readable storage medium storing executable instructions, which when executed by an electronic device having a touch-sensitive surface and a display, cause the electronic device to receive a portion of a voice communication, the portion including speech provided by a remote user of a remote device different from a user of the electronic device, determine that the voice communication includes speech identifying a physical location, in response to determining that the voice communication includes speech identifying the physical location, provide an indication that information regarding the physical location has been detected, detect an input via the touch-sensitive surface, in response to detecting the input, open an application that accepts geographic location data, populate the application with the information regarding the physical location, A non-transitory computer-readable storage medium.

147. The non-transitory computer-readable storage medium according to claim 146, wherein the voice communication is a live call.

148. The non-transitory computer-readable storage medium according to claim 146, wherein the voice communication is a live FaceTime call.

149. The non-transitory computer-readable storage medium according to claim 146, wherein the voice communication is a recorded voicemail.

150. Providing the indication includes displaying, within a user interface, a selectable explanation of the physical location that includes a most recent call made using a telephone application, the non-transitory computer-readable storage medium according to any one of claims 146 to 149.

151. The selectable explanation indicates that the content item is associated with the voice communication, the non-transitory computer-readable storage medium according to claim 150.

152. Detecting the input includes detecting the input on the selectable explanation while the user interface including the most recent call is being displayed, the non-transitory computer-readable storage medium according to claim 150 or 151.

153. Providing the indication includes providing haptic feedback to the user of the electronic device, the non-transitory computer-readable storage medium according to any one of claims 146 to 152.

154. Providing the indication includes transmitting information regarding the physical location to a different electronic device in the vicinity of the electronic device, the non-transitory computer-readable storage medium according to any one of claims 146 to 153.

155. Determining that the voice communication includes speech that describes the physical location includes analyzing the portion of the voice communication to detect information regarding the physical location, the analysis being performed while the voice communication is being output via an audio system that is in communication with the electronic device, the non-transitory computer-readable storage medium according to any one of claims 146 to 154.

156. Receiving at least the portion of the voice communication includes receiving, from a user of the electronic device, an instruction that the portion of the voice communication is to be analyzed, the non-transitory computer-readable storage medium according to any one of claims 146 to 155.

157. The instruction corresponds to a selection of a hardware button, the non-transitory computer-readable storage medium according to claim 156.

158. The instruction corresponds to a command from a user of the electronic device that includes the word "hey Siri", the non-transitory computer-readable storage medium according to claim 156.

159. In an electronic device having one or more processors, a memory, a touch-sensing surface, and a display, receiving at least a portion of a voice communication, wherein the portion of the voice communication includes speech provided by a remote user of a remote device different from a user of the electronic device; determining that the voice communication includes speech identifying a physical location; providing an indication that information regarding the physical location has been detected in response to determining that the voice communication includes speech identifying the physical location; detecting an input via the touch-sensing surface; in response to detecting the input, opening an application that receives geographic location data; populating the application with the information regarding the physical location; A method comprising.

160. The method according to claim 159, wherein the voice communication is a live call.

161. The method according to claim 159, wherein the voice communication is a live FaceTime call.

162. The method according to claim 159, wherein the voice communication is a recorded voicemail.

163. Providing the indication includes displaying, within a user interface, a selectable explanation of the physical location that includes a most recent call made using a phone application, according to any one of claims 159 to 162.

164. The method according to claim 163, wherein the selectable explanation indicates that the content item is associated with the voice communication.

165. Detecting the input includes detecting the input on the selectable explanation while the user interface including the most recent call is being displayed, according to any one of claims 163 to 164.

166. Providing the indication includes providing haptic feedback to the user of the electronic device, according to any one of claims 159 to 165.

167. Providing the indication includes transmitting the information regarding the physical location to a different electronic device in the vicinity of the electronic device, according to any one of claims 159 to 166.

168. Determining that the voice communication includes speech that describes the physical location includes analyzing the portion of the voice communication to detect information regarding the physical location, the analyzing being performed while the voice communication is output via an audio system communicating with the electronic device, the method according to any one of claims 159 to 167.

169. Receiving at least the portion of the voice communication includes receiving, from a user of the electronic device, an instruction that the portion of the voice communication is to be analyzed, the method according to any one of claims 159 to 168.

170. The instruction corresponds to a selection of a hardware button, the method according to claim 169.

171. The instruction corresponds to a command including the phrase "hey Siri" from a user of the electronic device, the method according to claim 169.

172. A touch-sensitive surface, A display, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, Comprising, the one or more programs including instructions to perform the method according to any one of claims 159 to 171, An electronic device.

173. A graphical user interface on an electronic device having a memory, a touch-sensitive surface, a display, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 159 to 171.

174. An electronic device having a touch-sensitive surface and a display, the electronic device comprising means for performing the method according to any one of claims 159 to 171.

175. An information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing the method according to any one of claims 159 to 171.

176. An electronic device, A touch-sensitive surface unit configured to receive a user's contact, A display unit configured to display a user interface, a touch sensing surface unit and a processing unit coupled to the display unit, the processing unit receives at least a portion of a voice communication, the portion of the voice communication including speech provided by a remote user of a remote device different from a user of the electronic device determines that the voice communication includes speech identifying a physical location provides an indication that information regarding the physical location has been detected in response to determining that the voice communication includes speech identifying the physical location detects an input via the touch sensing surface unit in response to detecting the input opens an application that accepts geographical location data populates the application with the information regarding the physical location An electronic device configured as such.

177. The electronic device according to claim 176, wherein the voice communication is a live call.

178. The electronic device according to claim 176, wherein the voice communication is a live FaceTime call.

179. The electronic device according to claim 176, wherein the voice communication is a recorded voicemail.

180. Providing the indication includes displaying, within a user interface, a selectable explanation of the physical location including a most recent call made using a phone application, according to any one of claims 176 to 179.

181. The electronic device according to claim 180, wherein the selectable explanation indicates that the content item is associated with the voice communication.

182. Detecting the input includes detecting the input on the selectable explanation while the user interface including the most recent call is being displayed, according to claim 180 or 181.

183. Providing the indication includes providing haptic feedback to the user of the electronic device, according to any one of claims 176 to 182.

184. Providing the indication includes transmitting the information regarding the physical location to a different electronic device in the vicinity of the electronic device, according to any one of claims 176 to 183.

185. Determining that the voice communication includes speech that describes the physical location includes analyzing the portion of the voice communication to detect information regarding the physical location, the analysis being performed while the voice communication is output via an audio system communicating with the electronic device, the electronic device according to any one of claims 176 to 184.

186. Receiving at least the portion of the voice communication includes receiving from a user of the electronic device an instruction that the portion of the voice communication is to be analyzed, the electronic device according to any one of claims 176 to 185.

187. The instruction corresponds to selection of a hardware button, the electronic device according to claim 186.

188. The instruction corresponds to a command including the phrase "hey Siri" from a user of the electronic device, the electronic device according to claim 186.

189. A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device having a display and a touch-sensitive surface, cause the electronic device to, present a text input field and a conversation record in a message application on the display, while the message application is presented on the display, cause a determination that a next possible input from a user of the electronic device is information regarding a physical location, analyze content associated with the text input field and the conversation record to cause a determination of a proposed physical location, at least in part, based on a portion of the analyzed content, present a selectable user interface element identifying the proposed physical location within the message application on the display, receive a selection of the selectable user interface element, in response to receiving the selection, present in the text input field a representation of the proposed physical location, non-transitory computer-readable storage medium.

190. The message application includes a virtual keyboard, and the selectable user interface element is displayed in a proposed portion adjacent to and above the virtual keyboard. The non-transitory computer-readable storage medium according to claim 189.

191. Determining that the next possible input from the user of the electronic device is information regarding a physical location includes processing the text input field and the content associated with the conversation record to detect that the conversation transcript includes a question regarding the current location of the user. The non-transitory computer-readable storage medium according to claim 189 or 190.

192. Processing the content includes applying a natural language processing algorithm to detect one or more defined keywords that form the question. The non-transitory computer-readable storage medium according to claim 191.

193. The question is included in a message received from a second user different from the user. The non-transitory computer-readable storage medium according to claim 191 or 192.

194. Determining that the next possible input from the user of the electronic device is information regarding a physical location includes monitoring typing input received from the user in the text input portion of the message application. The non-transitory computer-readable storage medium according to any one of claims 189 to 193.

195. When the executable instructions are executed by the electronic device, the electronic device Stops presenting the selectable user interface element according to a determination that the user is typing and has not selected the selectable user interface element. The non-transitory computer-readable storage medium according to any one of claims 189 to 194.

196. When the executable instructions are executed by the electronic device, the electronic device Stops presenting the selectable user interface element according to a determination that the user has provided additional input indicating that the user does not select the selectable user interface element. The non-transitory computer-readable storage medium according to any one of claims 189 to 194.

197. The non - transitory computer - readable storage medium according to any one of claims 189 to 196, wherein the representation of the proposed physical location includes information identifying the current geographical location of the electronic device.

198. The non - transitory computer - readable storage medium according to any one of claims 189 to 197, wherein the representation of the proposed physical location is an address.

199. The non - transitory computer - readable storage medium according to any one of claims 189 to 197, wherein the representation of the proposed physical location is a map object including an identifier related to the proposed physical location.

200. The non - transitory computer - readable storage medium according to any one of claims 189 to 199, wherein the proposed physical location corresponds to a location that the user recently viewed in an application other than the message application.

201. The non - transitory computer - readable storage medium according to any one of claims 189 to 200, wherein the message application is an email application.

202. The non - transitory computer - readable storage medium according to any one of claims 189 to 200, wherein the message application is a text - message application.

203. In an electronic device having one or more processors, a memory, a touch - sensitive surface, and a display, presenting a text - input field and a conversation record in a message application on the display; determining that the next possible input from a user of the electronic device is information related to a physical location while the message application is presented on the display; analyzing content associated with the text - input field and the conversation record to determine a proposed physical location based at least in part on a portion of the analyzed content; presenting a selectable user - interface element within the message application on the display to identify the proposed physical location; receiving a selection of the selectable user - interface element; presenting a representation of the proposed physical location in the text - input field in response to receiving the selection; A method comprising the above steps.

204. The method according to claim 203, wherein the message application includes a virtual keyboard, and the selectable user interface element is displayed in a proposed portion adjacent to and above the virtual keyboard.

205. Determining that the next possible input from the user of the electronic device is information regarding a physical location includes processing the content associated with the text input field and the conversation record to detect that the conversation transcript includes a question regarding the current location of the user. The method according to claim 203 or 204.

206. Processing the content includes applying a natural language processing algorithm to detect one or more defined keywords that form the question. The method according to claim 205.

207. The question is included in a message received from a second user different from the user. The method according to claim 205 or 206.

208. Determining that the next possible input from the user of the electronic device is information regarding a physical location includes monitoring typing input received from the user in the text input portion of the message application. The method according to any one of claims 203 to 207.

209. The method according to any one of claims 203 to 208 further includes stopping the presentation of the selectable user interface element according to a determination that the user is typing and has not selected the selectable user interface element.

210. The method according to any one of claims 203 to 209 further includes stopping the presentation of the selectable user interface element according to a determination that the user has provided additional input indicating that the user does not select the selectable user interface element.

211. The representation of the proposed physical location includes information identifying the current geographical location of the electronic device. The method according to any one of claims 203 to 210.

212. The representation of the proposed physical location is an address. The method according to any one of claims 203 to 210.

213. The method according to any one of claims 203 to 210, wherein the representation of the proposed physical location is a map object including an identifier related to the proposed physical location.

214. The method according to any one of claims 203 to 213, wherein the proposed physical location corresponds to a location that the user has most recently viewed in an application other than the message application.

215. The method according to any one of claims 203 to 214, wherein the message application is an email application.

216. The method according to any one of claims 203 to 214, wherein the message application is a text message application.

217. A touch-sensitive surface, A display, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, Comprising, wherein the one or more programs include instructions for executing the method according to any one of claims 203 to 216, An electronic device.

218. A graphical user interface on an electronic device having a memory, a touch-sensitive surface, a display, and one or more processors for executing one or more programs stored in the memory, wherein the graphical user interface comprises a user interface displayed according to the method according to any one of claims 203 to 216.

219. An electronic device having a touch-sensitive surface and a display, the electronic device comprising means for executing the method according to any one of claims 203 to 216.

220. An information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for executing the method according to any one of claims 203 to 216.

221. An electronic device, A touch-sensitive surface unit configured to receive user contact, A display unit configured to display a user interface, A processing unit coupled to the touch-sensitive surface unit and the display unit, The processing unit is In the message application on the display, present a text input field and a conversation record, While the message application is presented on the display, determine that the next possible input from the user of the electronic device is information regarding a physical location, Analyze the content associated with the text input field and the conversation record, and determine a proposed physical location based at least in part on a portion of the analyzed content, Present a selectable user interface element within the message application on the display that identifies the proposed physical location, Receive a selection of the selectable user interface element, In response to receiving the selection, present a representation of the proposed physical location within the text input field, An electronic device configured as described above.

222. The message application includes a virtual keyboard, and the selectable user interface element is displayed in a proposed portion adjacent to and above the virtual keyboard. The electronic device according to claim 221.

223. Determining that the next possible input from the user of the electronic device is information regarding a physical location includes processing the content associated with the text input field and the conversation record to detect that the conversation transcript includes a question regarding the user's current location. The electronic device according to claim 221 or 222.

224. Processing the content includes applying a natural language processing algorithm to detect one or more defined keywords that form the question. The electronic device according to claim 223.

225. The question is included in a message received from a second user different from the user. The electronic device according to claim 223 or 224.

226. Determining that the next possible input from the user of the electronic device is information regarding a physical location includes monitoring typing input received from the user in the text input portion of the message application. The electronic device according to any one of claims 221 to 225.

227. The processing unit is further configured to stop presenting the selectable user interface element according to a determination that the user is typing and has not selected the selectable user interface element, the electronic device according to any one of claims 221 to 226. **Claim 228** The processing unit is further configured to stop presenting the selectable user interface element according to a determination that the user has provided additional input indicating that the user does not select the selectable user interface element, the electronic device according to any one of claims 221 to 226. **Claim 229** The representation of the proposed physical location includes information identifying the current geographical location of the electronic device, the electronic device according to any one of claims 221 to 228. **Claim 230** The representation of the proposed physical location is an address, the electronic device according to any one of claims 221 to 229. **Claim 231** The representation of the proposed physical location is a map object including an identifier for the proposed physical location, the electronic device according to any one of claims 221 to 229. **Claim 232** The proposed physical location corresponds to a location that the user recently viewed in an application other than the messaging application, the electronic device according to any one of claims 221 to 231. **Claim 233** The messaging application is an email application, the electronic device according to any one of claims 221 to 232. **Claim 234** The messaging application is a text messaging application, the electronic device according to any one of claims 221 to 232. **Claim 235** A non-transitory computer-readable storage medium storing executable instructions, which when executed by an electronic device communicating with a display, cause the electronic device to acquire information identifying a first physical location viewed by a user in a first application while the first application is being displayed, terminate the first application, After the termination of the first application, receive a request from the user to open a second application different from the first application. In response to receiving the request and according to the determination that the second application can receive geographical location information, present the second application, and presenting the second application includes populating the second application, at least in part, with information based on the information identifying the first physical location. A non-transitory computer-readable storage medium.

236. Receiving the request to open the second application includes detecting an input on the affordance of the second application after the termination of the first application. The non-transitory computer-readable storage medium according to claim 235.

237. The affordance of the second application is an icon displayed within the home screen of the electronic device. The non-transitory computer-readable storage medium according to claim 236.

238. Detecting the input includes detecting a double-tap on a physical home button, displaying an application switch user interface in response to detecting the double-tap, detecting a selection of the affordance from within the application switch user interface. The non-transitory computer-readable storage medium according to claim 236.

239. Populating the second application includes displaying a user interface object including information based at least in part on the information identifying the first physical location. The non-transitory computer-readable storage medium according to any one of claims 235 to 238.

240. The user interface object includes a textual explanation informing the user that the first physical location was most recently viewed in the first application. The non-transitory computer-readable storage medium according to claim 239.

241. The user interface object is a map displayed within the second application. Populating the second application includes populating the map to include an identifier of the first physical location. The non-transitory computer-readable storage medium according to claim 240. **Claim 242** The non-transitory computer-readable storage medium according to claim 240 or 241, wherein the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard. **Claim 243** The non-transitory computer-readable storage medium according to any one of claims 240 to 242, wherein obtaining the information includes obtaining information regarding a second physical location, and displaying the user interface object includes displaying the user interface object having the information regarding the second physical location. **Claim 244** The determination that the second application can receive geographical location information includes one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services. The non-transitory computer-readable storage medium according to any one of claims 235 to 243. **Claim 245** The determination that the second application can receive geographical location information includes determining that the second application includes an input receiving field capable of receiving and processing geographical location data. The non-transitory computer-readable storage medium according to claim 244, wherein the input receiving field is a search box that enables searching within a map displayed within the second application. **Claim 246** When the executable instructions are executed by the electronic device, the electronic device A non - transitory computer - readable storage medium according to any one of claims 235 to 245, which, in response to receiving the request, determines whether the second application is associated with the first application based on the application usage history of the user.

247. When the executable instructions are executed by the electronic device, the electronic device provides access to the information identifying the first physical location to the second application before presenting the second application, and before the access is provided, the second application does not have access to the information identifying the first physical location. The non - transitory computer - readable storage medium according to claim 246.

248. In an electronic device having one or more processors, a memory, a touch - sensitive surface, and a display, acquiring information identifying a first physical location browsed by a user in the first application while the first application is being displayed; ending the first application; after ending the first application, receiving a request from the user to open a second application different from the first application; in response to receiving the request and according to a determination that the second application can receive geographical location information, presenting the second application; The method includes presenting the second application, at least in part, populating the second application with information based on the information identifying the first physical location.

249. Receiving the request to open the second application includes detecting an input on the affordance of the second application after the first application has ended. The method according to claim 248.

250. The affordance of the second application is an icon displayed within the home screen of the electronic device. The method according to claim 249.

251. Detecting the input includes detecting a double - tap on a physical home button; displaying an application - switching user interface in response to detecting the double - tap. Detecting a selection of the affordance from within the application switching user interface; The method according to claim 249, comprising: **Claim 252** The method according to any one of claims 248 to 251, wherein populating the second application comprises displaying a user interface object comprising information at least in part based on the information identifying the first physical location. **Claim 253** The method according to claim 252, wherein the user interface object comprises a textual explanation informing the user that the first physical location was most recently viewed in the first application. **Claim 254** The method according to claim 253, wherein the user interface object is a map displayed within the second application, and populating the second application comprises populating the map to include an identifier of the first physical location. **Claim 255** The method according to claim 253 or 254, wherein the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard. **Claim 256** The method according to any one of claims 253 to 255, wherein obtaining the information comprises obtaining information regarding a second physical location, and displaying the user interface object comprises displaying the user interface object having the information regarding the second physical location. **Claim 257** The method according to any one of claims 248 to 256, wherein the determination that the second application is capable of receiving geographical location information comprises one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application is capable of displaying geographical location information on a map; (iii) determining that the second application is capable of using geographical location information to facilitate route guidance; and (iv) determining that the second application is capable of using geographical location information to locate and provide transportation services. **Claim 258** The determination that the second application can receive geographical location information includes determining that the second application includes an input receiving field capable of receiving and processing geographical location data. The method according to claim 257, wherein the input receiving field is a search box that enables searching within a map displayed within the second application. **Claim 259** The method according to any one of claims 248 to 258, further comprising determining, in response to receiving the request, whether the second application is associated with the first application based on the user's application usage history. **Claim 260** The method according to claim 259, further comprising providing access to the information identifying the first physical location to the second application before presenting the second application, and before the access is provided, the second application does not have access to the information identifying the first physical location. **Claim 261** A touch sensing surface, A display, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, An electronic device comprising the one or more programs including instructions for performing the method according to any one of claims 248 to 260. **Claim 262** A graphical user interface on an electronic device having a memory, a touch sensing surface, a display, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 248 to 260. **Claim 263** An electronic device having a touch sensing surface and a display, the electronic device comprising means for performing the method according to any one of claims 248 to 260. **Claim 264** An information processing apparatus for use in an electronic device including a touch sensing surface and a display, the information processing apparatus comprising means for performing the method according to any one of claims 248 to 260. **Claim 265** A touch-sensing surface unit configured to receive user contact, A display unit configured to display a user interface, A processing unit coupled to the touch-sensing surface unit and the display unit, Comprising, the processing unit is, While displaying a first application, obtain information identifying a first physical location browsed by the user in the first application, End the first application, After the end of the first application, receive a request from the user to open a second application different from the first application, In response to receiving the request and in accordance with a determination that the second application can receive geographical location information, present the second application, and presenting the second application includes populating at least a portion of the second application with information based on the information identifying the first physical location, An electronic device configured as described above.

266. Receiving the request to open the second application includes detecting an input on an affordance of the second application after the end of the first application. The electronic device according to claim 265.

267. The affordance of the second application is an icon displayed within a home screen of the electronic device. The electronic device according to claim 266.

268. Detecting the input includes, Detecting a double-tap on a physical home button, In response to detecting the double-tap, displaying an application switch user interface, Detecting a selection of the affordance from within the application switch user interface, The electronic device according to claim 266.

269. Populating the second application includes displaying a user interface object including information based at least in part on the information identifying the first physical location. The electronic device according to any one of claims 265 to 268.

270. The electronic device according to claim 269, wherein the user interface object includes an explanation in text notifying the user that the first physical location has been most recently viewed in the first application.

271. The user interface object is a map displayed within the second application, populating the second application includes populating the map to include an identifier of the first physical location, the electronic device according to claim 270.

272. The second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard, the electronic device according to claim 270 or 271.

273. Obtaining the information includes obtaining information regarding a second physical location, and displaying the user interface object includes displaying the user interface object having the information regarding the second physical location, the electronic device according to any one of claims 270 to 272.

274. The determination that the second application can receive geographical location information includes one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services, the electronic device according to any one of claims 265 to 273.

275. The determination that the second application can receive geographical location information includes determining that the second application includes an input receiving field capable of receiving and processing geographical location data, the input receiving field is a search box enabling a search within a map displayed within the second application, the electronic device according to claim 274.

276. The processing unit is, further configured to determine, in response to receiving the request and based on the user's application usage history, whether the second application is associated with the first application, the electronic device according to any one of claims 265 to 275.

277. The processing unit is, before presenting the second application, provide access to the information identifying the first physical location for the second application, and before the access is provided, the second application is further configured not to have access to the information identifying the first physical location, the electronic device according to claim 276.

278. A non-transitory computer-readable storage medium storing executable instructions, which when executed by an electronic device communicating with a display, cause the electronic device to, acquire information identifying a first physical location browsed by a user in a first application, detect a first input, in response to detecting the first input, identify a second application capable of receiving geographical location information, present, on at least a portion of the display, an affordance having a proposal to open the second application having information regarding the first physical location different from the first application, detect a second input in the affordance, in response to detecting the second input in the affordance, open the second application, populate the second application to include information based at least in part on the information identifying the first physical location, non-transitory computer-readable storage medium.

279. The first input corresponds to a request to open an application switching user interface, the non-transitory computer-readable storage medium according to claim 278.

280. The affordance is presented within the application switching user interface, the non-transitory computer-readable storage medium according to claim 279.

281. Presenting the affordance is, In addition to presenting the affordance, presenting a representation of an application running on the electronic device within the application switching user interface; Presenting the affordance within a region of the display located below the representation of the application; The non-transitory computer-readable storage medium according to claim 280, comprising:

282. The non-transitory computer-readable storage medium according to claim 278, wherein the first input corresponds to a request to open a home screen of the electronic device.

283. The non-transitory computer-readable storage medium according to claim 282, wherein the affordance is presented on a portion of the home screen.

284. The non-transitory computer-readable storage medium according to any one of claims 278 to 283, wherein the proposal includes an explanation in text specific to a type associated with the second application.

285. Populating the second application includes, at least in part, displaying a user interface object including information based on the information identifying the first physical location, according to any one of claims 278 to 284. The non-transitory computer-readable storage medium described.

286. The non-transitory computer-readable storage medium according to claim 285, wherein the user interface object includes a textual explanation informing the user that the first physical location has been most recently viewed in the first application.

287. The user interface object is a map displayed within the second application, Populating the second application includes populating the map to include an identifier of the first physical location, The non-transitory computer-readable storage medium according to claim 286.

288. The non-transitory computer-readable storage medium according to claim 286 or 287, wherein the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard.

289. Identifying the second application capable of receiving geographical location information includes one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services. The non-transitory computer-readable storage medium according to any one of claims 278 to 288.

290. Identifying that the second application can receive geographical location information includes determining that the second application includes an input receiving field capable of receiving and processing geographical location data. The input receiving field is a search box that enables searching within a map displayed within the second application. The non-transitory computer-readable storage medium according to claim 289.

291. In an electronic device having one or more processors, a memory, a touch-sensitive surface, and a display, obtaining information identifying a first physical location viewed by a user in a first application; detecting a first input; in response to detecting the first input, identifying a second application capable of receiving geographical location information; presenting an affordance having a proposal to open the second application, different from the first application, having information regarding the first physical location on at least a portion of the display; detecting a second input in the affordance; in response to detecting the second input in the affordance, opening the second application; populating the second application to include information based at least in part on the information identifying the first physical location. A method comprising.

292. The method according to claim 291, wherein the first input corresponds to a request to open an application switching user interface. **Claim 293** The method according to claim 292, wherein the affordance is presented within the application switching user interface. **Claim 294** Presenting the affordance includes presenting, within the application switching user interface, a representation of an application running on the electronic device, in addition to presenting the affordance; presenting the affordance within a region of the display located below the representation of the application; The method according to claim 293. **Claim 295** The method according to claim 291, wherein the first input corresponds to a request to open a home screen of the electronic device. **Claim 296** The method according to claim 295, wherein the affordance is presented on a portion of the home screen. **Claim 297** The method according to any one of claims 291 to 296, wherein the suggestion includes an explanation in text specific to a type associated with the second application. **Claim 298** Populating the second application includes displaying a user interface object that includes information based at least in part on the information identifying the first physical location, according to the method of any one of claims 291 to 297. **Claim 299** The method according to claim 298, wherein the user interface object includes an explanation in text informing the user that the first physical location was most recently viewed in the first application. **Claim 300** The user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location. The method according to claim 299. **Claim 301** The method according to claim 299 or 300, wherein the second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard. **Claim 302** Identifying the second application that can receive geographical location information includes one or more of: (i) determining that the second application includes an input receiving field that can receive and process geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services. The method according to any one of claims 291 to 301.

303. Identifying that the second application can receive geographical location information includes determining that the second application includes an input receiving field that can receive and process geographical location data. The input receiving field is a search box that enables searching within a map displayed within the second application. The method according to claim 302.

304. A touch-sensitive surface, A display, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, Comprising, wherein the one or more programs include instructions for executing the method according to any one of claims 291 to 303. An electronic device.

305. A graphical user interface on an electronic device having a memory, a touch-sensitive surface, a display, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 291 to 303.

306. An electronic device having a touch-sensitive surface and a display, the electronic device comprising means for executing the method according to any one of claims 291 to 303.

307. An information processing apparatus for use in an electronic device including a touch sensing surface and a display, the information processing apparatus comprising means for performing the method according to any one of claims 291 to 303.

308. A touch sensing surface unit configured to receive user contact, A display unit configured to display a user interface, A processing unit coupled to the touch sensing surface unit and the display unit, Comprising, the processing unit is Obtaining information identifying a first physical location viewed by a user in a first application, Detecting a first input, In response to detecting the first input, Identifying a second application capable of receiving geographical location information, Presenting an affordance having a proposal to open the second application having information regarding the first physical location, different from the first application, on at least a portion of the display unit, Detecting a second input in the affordance, In response to detecting the second input in the affordance, Opening the second application, Populating the second application to include information based at least in part on the information identifying the first physical location, An electronic device configured as such.

309. The electronic device according to claim 308, wherein the first input corresponds to a request to open an application switching user interface.

310. The electronic device according to claim 309, wherein the affordance is presented within the application switching user interface.

311. Presenting the affordance includes Presenting a representation of an application running on the electronic device within the application switching user interface in conjunction with presenting the affordance, Presenting the affordance within a region of the display unit located below the representation of the application, The electronic device according to claim 310, including.

312. The electronic device according to claim 308, wherein the first input corresponds to a request to open the home screen of the electronic device.

313. The affordance is the electronic device according to claim 312, presented on a part of the home screen.

314. The proposal is the electronic device according to any one of claims 308 to 313, including an explanation in text specific to the type associated with the second application.

315. Populating the second application includes, at least in part, displaying a user interface object that includes information based on the information identifying the first physical location, for the electronic device according to any one of claims 308 to 314.

316. The user interface object is the electronic device according to claim 315, including an explanation in text that informs the user that the first physical location was most recently viewed in the first application.

317. The user interface object is a map displayed within the second application, and populating the second application includes populating the map to include an identifier of the first physical location, for the electronic device according to claim 316.

318. The second application is presented with a virtual keyboard, and the user interface object is displayed above the virtual keyboard, for the electronic device according to claim 316 or 317.

319. Identifying the second application capable of receiving geographical location information includes one or more of: (i) determining that the second application includes an input receiving field capable of receiving and processing geographical location data; (ii) determining that the second application can display geographical location information on a map; (iii) determining that the second application can use geographical location information to facilitate route guidance; and (iv) determining that the second application can use geographical location information to locate and provide transportation services, for the electronic device according to any one of claims 308 to 318.

320. Identifying that the second application can receive geographical location information includes determining that the second application includes an input receiving field capable of receiving and processing geographical location data. The input receiving field is a search box that enables searching within a map displayed within the second application. The electronic device according to claim 319.

321. A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device communicating with a display, cause the electronic device to acquire information identifying a first physical location viewed by a user in a first application running on the electronic device, determine that the user has entered a vehicle, in response to determining that the user has entered the vehicle, provide a prompt to the user to use the first physical location as a destination for route guidance, in response to providing the prompt, receive from the user an instruction to use the first physical location as the destination for route guidance, facilitate route guidance to the first physical location. Non-transitory computer-readable storage medium.

322. When the executable instructions are executed by the electronic device, cause the electronic device to detect receiving a message, the message being a message and including information identifying a second physical location, in response to the detection, provide a new prompt to the user to use the second physical location as a new destination for route guidance. The non-transitory computer-readable storage medium according to claim 321.

323. When the executable instructions are executed by the electronic device, cause the electronic device to facilitate route guidance to the second physical location in response to receiving an instruction from the user to use the second physical location as the new destination, the non-transitory computer-readable storage medium according to claim 322.

324. Detecting that the message includes information identifying the second physical location includes executing the detection while a virtual assistant available on the electronic device reads the message aloud to the user via an audio system communicating with the electronic device. The non-transitory computer-readable storage medium according to any one of claims 322 to 323.

325. Determining that the user has entered the vehicle includes detecting that the electronic device has established a communication link with the vehicle. The non-transitory computer-readable storage medium according to any one of claims 322 to 324.

326. Facilitating the route guidance includes providing the route guidance via the display of the electronic device. The non-transitory computer-readable storage medium according to any one of claims 322 to 325.

327. Facilitating the route guidance includes transmitting the information identifying the first physical location to the vehicle. The non-transitory computer-readable storage medium according to any one of claims 322 to 325.

328. Facilitating the route guidance includes providing the route guidance via an audio system communicating with the electronic device. The non-transitory computer-readable storage medium according to any one of claims 322 to 327.

329. In an electronic device having one or more processors, a memory, a touch-sensitive surface, and a display, obtaining information identifying a first physical location browsed by a user in a first application running on the electronic device; determining that the user has entered a vehicle; in response to determining that the user has entered the vehicle, providing a prompt to the user to use the first physical location as a destination for route guidance; receiving, in response to the provision of the prompt, an instruction from the user to use the first physical location as the destination for route guidance; facilitating route guidance to the first physical location; A method including.

330. Detecting that the electronic device has received a message, including detecting that the message includes information identifying a second physical location. In response to the detection, providing a new prompt to the user to use the second physical location as a new destination for route guidance; The method according to claim 329, further comprising.

331. The method according to claim 330, further comprising facilitating route guidance to the second physical location in response to receiving an instruction from the user to use the second physical location as the new destination.

332. Detecting that the message includes information specifying the second physical location includes performing the detection while a virtual assistant available on the electronic device reads the message to the user via an audio system communicating with the electronic device. The method according to any one of claims 330 to 331.

333. Determining that the user has entered the vehicle includes detecting that the electronic device has established a communication link with the vehicle. The method according to any one of claims 330 to 332.

334. Facilitating the route guidance includes providing the route guidance via the display of the electronic device. The method according to any one of claims 330 to 333.

335. Facilitating the route guidance includes transmitting the information specifying the first physical location to the vehicle. The method according to any one of claims 330 to 333.

336. Facilitating the route guidance includes providing the route guidance via an audio system communicating with the electronic device. The method according to any one of claims 330 to 335.

337. A touch-sensitive surface; A display; One or more processors; A memory storing one or more programs configured to be executed by the one or more processors; Comprising, the one or more programs include instructions for performing the method according to any one of claims 329 to 336. An electronic device.

338. A graphical user interface on an electronic device having a memory, a touch sensing surface, a display, and one or more processors for executing one or more programs stored in the memory, wherein the graphical user interface comprises a user interface displayed according to the method according to any one of claims 329 to 336.

339. An electronic device having a touch sensing surface and a display, the electronic device comprising means for executing the method according to any one of claims 329 to 336.

340. An information processing apparatus for use in an electronic device including a touch sensing surface and a display, the information processing apparatus comprising means for executing the method according to any one of claims 329 to 336.

341. An electronic device, a touch sensing surface unit configured to receive a user's contact, a display unit configured to display a user interface, a processing unit coupled to the touch sensing surface unit and the display unit, wherein the processing unit obtains information identifying a first physical location viewed by the user in a first application running on the electronic device, determines that the user has entered a vehicle, in response to determining that the user has entered the vehicle, provides a prompt to the user to use the first physical location as a destination for route guidance, in response to the provision of the prompt, receives from the user an instruction to use the first physical location as the destination for route guidance, and facilitates route guidance to the first physical location. An electronic device configured as such.

342. The processing unit further includes detecting that the electronic device has received a message and detecting that the message includes information identifying a second physical location, and in response to the detection, providing a new prompt to the user to use the second physical location as a new destination for route guidance. The electronic device according to claim 341, further configured as such.

343. The processing unit The electronic device according to claim 342, further configured to facilitate route guidance to the second physical location in response to receiving an instruction from the user to use the second physical location as the new destination.

344. Detecting that the message includes information specifying the second physical location includes performing the detection while a virtual assistant available on the electronic device reads the message to the user via an audio system communicating with the electronic device. The electronic device according to claim 342 or 343.

345. Determining that the user has entered the vehicle includes detecting that the electronic device has established a communication link with the vehicle. The electronic device according to any one of claims 342 to 344.

346. Facilitating the route guidance includes providing the route guidance via the display of the electronic device. The electronic device according to any one of claims 342 to 345.

347. Facilitating the route guidance includes transmitting the information specifying the first physical location to the vehicle. The electronic device according to any one of claims 342 to 345.

348. Facilitating the route guidance includes providing the route guidance via an audio system communicating with the electronic device. The electronic device according to any one of claims 342 to 347.

349. A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device communicating with a display, cause the electronic device to present content in a first application, receive a request from the user to open a second application different from the first application, the second application including an input receiving field, and in response to receiving the request, cause the second application to present the input receiving field. Before receiving any user input in the input receiving field, provide a selectable user interface object to enable the user to paste at least a part of the content into the input receiving field. In response to detecting the selection of the selectable user interface object, cause the part of the content to be pasted into the input receiving field. Non-transitory computer-readable storage medium.

350. When the executable instructions are executed by the electronic device, the electronic device Before providing the selectable user interface object, identify the input receiving field as a field capable of receiving the part of the content, the non-transitory computer-readable storage medium according to claim 349.

351. Identifying the input receiving field as a field capable of receiving the part of the content is executed in response to detecting the selection of the input receiving field, the non-transitory computer-readable storage medium according to claim 350.

352. The part of the content corresponds to an image, the non-transitory computer-readable storage medium according to any one of claims 349 to 351.

353. The part of the content corresponds to text content, the non-transitory computer-readable storage medium according to any one of claims 349 to 351.

354. The part of the content corresponds to text content and an image, the non-transitory computer-readable storage medium according to any one of claims 349 to 351.

355. The first application is a web browsing application, and the second application is a messaging application, the non-transitory computer-readable storage medium according to any one of claims 349 to 354.

356. The first application is a photo browsing application, and the second application is a messaging application, the non-transitory computer-readable storage medium according to any one of claims 349 to 354.

357. When the executable instructions are executed by the electronic device, the electronic device Causing a request to copy at least a portion of the content to be received before receiving the request to open the second application A non-transitory computer-readable storage medium according to any one of claims 349 to 356

358. The non-transitory computer-readable storage medium according to any one of claims 349 to 357, wherein the selectable user interface object is displayed with an indication that the portion of the content has been recently viewed in the first application

359. In an electronic device having one or more processors, a memory, a touch-sensitive surface, and a display Presenting content in a first application Receiving, from the user, a request to open a second application different from the first application, the second application including an input receiving field In response to receiving the request, presenting the input receiving field in the second application Providing a selectable user interface object to enable the user to paste at least a portion of the content into the input receiving field before receiving any user input in the input receiving field Pasting the portion of the content into the input receiving field in response to detecting a selection of the selectable user interface object A method comprising

360. The method according to claim 359, further comprising identifying the input receiving field as a field capable of receiving the portion of the content before providing the selectable user interface object

361. The method according to claim 360, wherein identifying the input receiving field as a field capable of receiving the portion of the content is performed in response to detecting a selection of the input receiving field

362. The method according to any one of claims 359 to 361, wherein the portion of the content corresponds to an image

363. The method according to any one of claims 359 to 361, wherein the portion of the content corresponds to text content

364. The method according to any one of claims 359 to 361, wherein the part of the content corresponds to text content and images.

365. The method according to any one of claims 359 to 364, wherein the first application is a web browsing application and the second application is a messaging application.

366. The method according to any one of claims 359 to 364, wherein the first application is a photo browsing application and the second application is a messaging application.

367. The method according to any one of claims 359 to 366, further comprising receiving a request to copy at least the part of the content before receiving the request to open the second application.

368. The method according to any one of claims 359 to 367, wherein the selectable user interface object is displayed with an indication that the part of the content has been recently viewed in the first application.

369. A touch-sensitive surface, A display, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, An electronic device comprising the one or more programs including instructions for performing the method according to any one of claims 359 to 368.

370. A graphical user interface on an electronic device having a memory, a touch-sensitive surface, a display, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 359 to 368.

371. An electronic device having a touch-sensitive surface and a display, the electronic device comprising means for performing the method according to any one of claims 359 to 368.

372. An information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing the method according to any one of claims 359 to 368.

373. A touch-sensing surface unit configured to receive user contact; A display unit configured to display a user interface; A processing unit coupled to the touch-sensing surface unit and the display unit; The processing unit is configured to: Present content in a first application; Receive a request from the user to open a second application different from the first application, the second application including an input receiving field; In response to receiving the request, present the input receiving field in the second application; Provide a selectable user interface object to enable the user to paste at least a portion of the content into the input receiving field before receiving any user input in the input receiving field; In response to detecting a selection of the selectable user interface object, paste the portion of the content into the input receiving field; An electronic device configured as such.

374. When the executable instructions are executed by the electronic device, the electronic device is caused to: Identify the input receiving field as a field capable of receiving the portion of the content before providing the selectable user interface object, as claimed in claim 373.

375. The electronic device according to claim 374, wherein identifying the input receiving field as a field capable of receiving the portion of the content is performed in response to detecting a selection of the input receiving field.

376. The electronic device according to any one of claims 373 to 375, wherein the portion of the content corresponds to an image.

377. The electronic device according to any one of claims 373 to 375, wherein the portion of the content corresponds to text content.

378. The electronic device according to any one of claims 373 to 375, wherein the portion of the content corresponds to text content and an image.

379. The electronic device according to any one of claims 373 to 378, wherein the first application is a web browsing application and the second application is a messaging application.

380. The electronic device according to any one of claims 373 to 378, wherein the first application is a photo browsing application and the second application is a messaging application.

381. When the executable instructions are executed by the electronic device, the electronic device receives a request to copy at least a part of the content before receiving the request to open the second application. The electronic device according to any one of claims 373 to 380.

382. The electronic device according to any one of claims 373 to 381, wherein the selectable user interface object is displayed with an indication that a part of the content has been recently viewed in the first application.

383. A non-transitory computer-readable storage medium storing executable instructions, which, when executed by an electronic device having a display and a touch-sensitive surface, cause the electronic device to present text content associated with an application on the display, determine that a part of the text content is related to (i) a location, (ii) a contact, or (iii) an event, when it is determined that the part of the text content is related to a location, obtain location information from a location sensor on the electronic device and prepare to display the obtained location information as a predicted content item, when it is determined that the part of the text content is related to a contact, perform a search for contact information related to the part of the text content on the electronic device and prepare to display the information retrieved by the search and associated with at least one contact as the predicted content item. When it is determined that the part of the text content is related to an event, cause a new search to be performed on event information related to the part of the text content on the electronic device, and cause at least a part of the information based on at least one event retrieved by the new search to be prepared for display as the predicted content item. Cause an affordance including the predicted content item to be displayed within the application. Detect a selection of the affordance via the touch-sensitive surface. In response to detecting the selection, cause information associated with the predicted content item to be displayed on the display adjacent to the text content. Non-transitory computer-readable storage medium.

384. The non-transitory computer-readable storage medium according to claim 383, wherein the part of the text content corresponds to the text content most recently presented within the application.

385. The non-transitory computer-readable storage medium according to claim 383 or 384, wherein the application is a messaging application, and the part of the text content is a question received from a remote user of a remote device different from the electronic device within the messaging application.

386. The non-transitory computer-readable storage medium according to claim 383 or 384, wherein the part of the text content is an input provided by the user of the electronic device in an input receiving field within the application.

387. The non-transitory computer-readable storage medium according to claim 383, wherein the part of the text content is specified in response to a user input that selects a user interface object including the part of the text content.

388. The non-transitory computer-readable storage medium according to claim 387, wherein the application is a messaging application, and the user interface object is a speech bubble for a message in a conversation displayed within the messaging application.

389. When the executable instructions are executed by the electronic device, cause the electronic device to Detect a selection of a second user interface object. In response to detecting the selection, Stop displaying the affordance together with the predicted content item, Cause the text content associated with the second user interface object to be determined to be related to a location, contact, or event, Cause a new predicted content item to be displayed within the application according to the determination, The non-transitory computer-readable storage medium according to claim 387 or 388.

390. The non-transitory computer-readable storage medium according to any one of claims 383 to 389, wherein the affordance is adjacent to a virtual keyboard within the application.

391. The information associated with the predicted content item is displayed within an input receiving field, and the input receiving field is a field that displays typing input received on the virtual keyboard. The non-transitory computer-readable storage medium according to claim 390.

392. The determining includes parsing the text content when received by the application and detecting a stored pattern known to be related to a contact, event, and / or location. The non-transitory computer-readable storage medium according to any one of claims 383 to 391.

393. In an electronic device having one or more processors, a memory, a touch sensing surface, and a display, Presenting text content associated with an application on the display; Determining that a portion of the text content is related to (i) a location, (ii) a contact, or (iii) an event; When it is determined that the portion of the text content is related to a location, obtaining location information from a location sensor on the electronic device and preparing the obtained location information to be displayed as a predicted content item; When it is determined that the portion of the text content is related to a contact, performing a search on the electronic device for contact information related to the portion of the text content and preparing the information retrieved by the search and associated with at least one contact to be displayed as the predicted content item; When it is determined that a part of the text content is related to an event, on the electronic device, a new search is performed for event information related to the part of the text content, and at least part of the information based on at least one event retrieved by the new search is prepared for display as the predicted content item. Displaying an affordance including the predicted content item within the application. Detecting a selection of the affordance via the touch-sensitive surface. In response to detecting the selection, displaying information associated with the predicted content item adjacent to the text content on the display. A method including the above.

394. The method according to claim 393, wherein the part of the text content corresponds to the text content most recently presented within the application.

395. The method according to claim 393 or 394, wherein the application is a messaging application, and the part of the text content is a question received from a remote user of a remote device different from the electronic device within the messaging application.

396. The method according to claim 393 or 394, wherein the part of the text content is an input provided by the user of the electronic device in an input receiving field within the application.

397. The method according to claim 393, wherein the part of the text content is specified in response to a user input for selecting a user interface object including the part of the text content.

398. The method according to claim 397, wherein the application is a messaging application, and the user interface object is a speech bubble for a message in a conversation displayed within the messaging application.

399. Detecting a selection of a second user interface object. In response to detecting the selection, Stopping displaying the affordance together with the predicted content item. determining that text content associated with the second user interface object is related to a location, contact, or event; displaying, within the application, a new predicted content item according to the determination; The method according to claim 397 or 398, further comprising.

400. The method according to any one of claims 393 to 399, wherein the affordance is displayed adjacent to a virtual keyboard within the application.

401. The information associated with the predicted content item is displayed within an input receiving field, and the input receiving field is a field that displays typing input received on the virtual keyboard. The method according to claim 400.

402. The determining includes parsing the text content when received by the application and detecting a stored pattern known to be related to a contact, event, and / or location. The method according to any one of claims 393 to 401.

403. A touch-sensitive surface; A display; One or more processors; A memory storing one or more programs configured to be executed by the one or more processors; Comprising, the one or more programs include instructions for performing the method according to any one of claims 393 to 402. An electronic device.

404. A graphical user interface on an electronic device having a memory, a touch-sensitive surface, a display, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 393 to 402. A graphical user interface.

405. An electronic device having a touch-sensitive surface and a display, the electronic device comprising means for performing the method according to any one of claims 393 to 402.

406. An information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, the information processing apparatus comprising means for performing the method according to any one of claims 393 to 402.

407. An electronic device, A touch sensing surface unit configured to receive user contact; A display unit configured to display a user interface; A processing unit coupled to the touch sensing surface unit and the display unit; Comprising, wherein the processing unit: Presents text content associated with an application on the display; Determines that a portion of the text content is related to (i) a location, (ii) a contact, or (iii) an event; When it is determined that the portion of the text content is related to a location, obtains location information from a location sensor on the electronic device and prepares the obtained location information to be displayed as a predicted content item; When it is determined that the portion of the text content is related to a contact, searches for contact information related to the portion of the text content on the electronic device and prepares the information retrieved by the search and associated with at least one contact to be displayed as the predicted content item; When it is determined that the portion of the text content is related to an event, performs a new search for event information related to the portion of the text content on the electronic device and prepares at least some of the information based on at least one event retrieved by the new search to be displayed as the predicted content item; Displays an affordance including the predicted content item within the application; Detects a selection of the affordance via the touch sensing surface; In response to detecting the selection, displays the information associated with the predicted content item on the display adjacent to the text content; An electronic device configured as such.

408. The electronic device according to claim 407, wherein the portion of the text content corresponds to the text content most recently presented within the application.

409. The electronic device according to claim 407 or 408, wherein the application is a messaging application and the portion of the text content is a question received from a remote user of a remote device different from the electronic device within the messaging application.

410. The electronic device according to claim 407 or 408, wherein a part of the text content is an input provided by the user of the electronic device in an input receiving field within the application.

411. The electronic device according to claim 407, wherein the part of the text content is specified in response to a user input that selects a user interface object including the part of the text content.

412. The electronic device according to claim 411, wherein the application is a messaging application, and the user interface object is a speech bubble for a message in a conversation displayed within the messaging application.

413. The processing unit detects a selection of a second user interface object, in response to detecting the selection, stops displaying the affordance together with the predicted content item, determines that text content associated with the second user interface object is related to a location, a contact, or an event, and in accordance with the determination, displays a new predicted content item within the application. The electronic device according to claim 411 or 412, further configured as described above.

414. The electronic device according to any one of claims 407 to 413, wherein the affordance is displayed adjacent to a virtual keyboard within the application.

415. The information associated with the predicted content item is displayed within an input receiving field, and the input receiving field is a field that displays typing input received on the virtual keyboard. The electronic device according to claim 414.

416. A non-transitory computer-readable storage medium storing executable instructions, wherein when the executable instructions are executed by an electronic device having a touch-sensitive surface and a display, the electronic device displays content associated with an application running on the electronic device on the display, detects a swipe gesture that, when detected via the touch-sensitive surface, causes the electronic device to transition to a search mode different from the application. In response to detecting the swipe gesture, shift to the search mode, where the search mode includes a search interface displayed on the display. In conjunction with shifting to the search mode, cause at least one proposed search query to be determined based on at least some of the information associated with the content. Populate the displayed search interface with the at least one proposed search query before receiving any user input in the search interface. Non-transitory computer-readable storage medium.

417. The non-transitory computer-readable storage medium according to claim 416, wherein detecting the swipe gesture includes detecting the swipe gesture on at least a portion of the currently displayed content.

418. When the executable instructions are executed by the electronic device, cause the electronic device to Detect an input corresponding to a request to view the home screen of the electronic device before detecting the swipe gesture. In response to detecting the input, stop displaying the content associated with the application and display the corresponding page of the home screen of the electronic device. The corresponding page is the initial page among a series of home screen pages. The swipe gesture is detected while the initial page of the home screen is being displayed on the display. The non-transitory computer-readable storage medium according to claim 416 or 417.

419. The non-transitory computer-readable storage medium according to any one of claims 416 to 418, wherein the search interface is displayed semi-transparently overlapping the application.

420. When the executable instructions are executed by the electronic device, cause the electronic device to Determine the at least one proposed search query based on at least a portion of the text content according to a determination that the content includes text content. The non-transitory computer-readable storage medium according to any one of claims 416 to 419.

421. Determining the at least one proposed search query based at least in part on the text content includes detecting one or more defined keywords used to analyze the text content to determine the at least one proposed search query, the non-transitory computer-readable storage medium according to claim 420.

422. Determining the at least one proposed search query includes determining a plurality of proposed search queries, Populating the search interface includes populating the search interface with the plurality of proposed search queries, The non-transitory computer-readable storage medium according to any one of claims 416 to 421.

423. When the executable instructions are executed by the electronic device, the electronic device, Detect a new swipe gesture on the newly displayed content via the touch-sensitive surface, In response to detecting the new swipe gesture, transitioning to the search mode includes causing the search interface to be displayed on the display to transition to the search mode, In conjunction with transitioning to the search mode and in accordance with the determination that the new content does not include text content, cause the search interface to be populated with search queries proposed based on a set of selected past search queries by a user of the electronic device. The non-transitory computer-readable storage medium according to any one of claims 416 to 422.

424. Based on location information provided by a second application different from the application, the search interface is displayed together with a target location, the non-transitory computer-readable storage medium according to claim 423.

425. The search interface further includes one or more proposed applications, the non-transitory computer-readable storage medium according to claim 423 or 424.

426. The set of past search queries is selected at least in part based on the frequency of recent search queries, the non-transitory computer-readable storage medium according to any one of claims 423 to 425.

427. When the executable instructions are executed by the electronic device, the electronic device, The non - transitory computer - readable storage medium according to any one of claims 416 to 426, wherein, in conjunction with transitioning to the search mode, information associated with the content is obtained by using one or more accessibility functions available on the electronic device.

428. Using the one or more accessibility functions includes (i) applying a natural - language processing algorithm to text content currently displayed within the application; and (ii) determining one or more keywords for explaining the content using data obtained from the natural - language processing algorithm, wherein the at least one proposed search query is determined based on the one or more keywords, thereby generating information associated with the content using the one or more accessibility functions. The non - transitory computer - readable storage medium according to claim 427.

429. Determining the one or more keywords for explaining the content includes (i) extracting metadata corresponding to non - text content currently displayed within the application; and (ii) determining the one or more keywords using the extracted metadata in addition to the data obtained from the natural - language processing algorithm. The non - transitory computer - readable storage medium according to claim 428, which also includes this.

430. The non - transitory computer - readable storage medium according to any one of claims 416 to 429, wherein the search interface further includes one or more trend queries.

431. The non - transitory computer - readable storage medium according to claim 430, wherein the search interface further includes one or more applications that the user of the electronic device is expected to be interested in.

432. In an electronic device including one or more processors, a memory, a touch - sensitive surface, and a display, displaying content associated with an application running on the electronic device on the display; and detecting a swipe gesture via the touch - sensitive surface that, when detected, causes the electronic device to transition to a search mode different from the application. Upon detecting the swipe gesture, transitioning to the search mode, wherein the search mode includes a search interface displayed on the display In conjunction with transitioning to the search mode, determining at least one proposed search query based on at least some information associated with the content Populating the displayed search interface with the at least one proposed search query before receiving any user input in the search interface A method comprising the above

433. The method according to claim 432, wherein detecting the swipe gesture includes detecting the swipe gesture on at least a part of the currently displayed content

434. Before detecting the swipe gesture, detecting an input corresponding to a request to view the home screen of the electronic device In response to detecting the input, stopping displaying the content associated with the application and displaying the corresponding page of the home screen of the electronic device of the electronic device Including, the corresponding page being the initial page among a series of home screen pages The swipe gesture is detected while the initial page of the home screen is being displayed on the display The method according to claim 432 or 433

435. The method according to any one of claims 432 to 434, wherein the search interface is displayed semi-transparently overlapping the application

436. The method according to any one of claims 432 to 435, including determining the at least one proposed search query based on at least some of the text content in accordance with a determination that the content includes text content

437. The method according to claim 436, wherein determining the at least one proposed search query based on at least some of the text content includes detecting one or more predefined keywords used to analyze the text content to determine the at least one proposed search query

438. Determining the at least one proposed search query includes determining a plurality of proposed search queries. Populating the search interface includes populating the search interface with the plurality of proposed search queries. The method according to any one of claims 432 to 437. **Claim 439** Detecting a new swipe gesture on currently displayed new content via the touch-sensitive surface; In response to detecting the new swipe gesture, transitioning to the search mode, wherein transitioning to the search mode includes displaying the search interface on the display; In conjunction with transitioning to the search mode and in accordance with a determination that the new content does not include text content, populating the search interface with search queries proposed based on a set of selected past search queries by a user of the electronic device; The method according to any one of claims 432 to 438, comprising: **Claim 440** The method according to claim 439, wherein the search interface is displayed with a target location based on location information provided by a second application different from the application. **Claim 441** The method according to claim 439 or 440, wherein the search interface further includes one or more proposed applications. **Claim 442** The method according to any one of claims 439 to 441, wherein the set of past search queries is selected at least in part based on the frequency of recent search queries. **Claim 443** The method according to any one of claims 432 to 442, further comprising obtaining the information associated with the content by using one or more accessibility features available on the electronic device in conjunction with transitioning to the search mode. **Claim 444** Using the one or more accessibility features includes: (i) applying a natural language processing algorithm to text content currently displayed within the application; (ii) determining one or more keywords for explaining the content using data obtained from the natural language processing algorithm, wherein the at least one proposed search query is determined based on the one or more keywords; The method according to claim 443, further comprising generating the information associated with the content using the one or more accessibility features.

445. Determining the one or more keywords for explaining the content comprises: (i) retrieving metadata corresponding to non-text content currently displayed within the application; (ii) determining the one or more keywords using the retrieved metadata in addition to the data obtained from the natural language processing algorithm; The method according to claim 444, further comprising the above.

446. The method according to any one of claims 432 to 445, wherein the search interface further comprises one or more trend queries.

447. The method according to claim 446, wherein the search interface further comprises one or more applications that the user of the electronic device is expected to be interested in.

448. A touch-sensitive surface, A display, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, An electronic device, wherein the one or more programs include instructions for performing the method according to any one of claims 432 to 447.

449. A graphical user interface on an electronic device having a touch-sensitive surface, a display, a memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 432 to 447.

450. A touch-sensitive surface, A display, Means for performing the method according to any one of claims 432 to 447, An electronic device comprising the above.

451. An information processing apparatus for use in an electronic device including a touch-sensitive surface and a display, An information processing apparatus comprising means for executing the method according to any one of claims 432 to 447.

452. An electronic device, A display unit configured to display a user interface, A touch sensing unit configured to detect a user's contact through a touch sensing surface of the touch sensing display unit, A processing unit coupled to the touch sensing surface unit and the display unit, Comprising, the processing unit Display on the display unit content associated with an application being executed on the electronic device, Detect a swipe gesture that, when detected via the touch sensing surface unit, causes the electronic device to transition to a search mode different from the application, In response to detecting the swipe gesture, transition to the search mode, the search mode including a search interface displayed on the display unit, In conjunction with transitioning to the search mode, determine at least one proposed search query based on at least some information associated with the content, Populate the displayed search interface with the at least one proposed search query before receiving any user input in the search interface. An electronic device configured as described above.

453. The electronic device according to claim 452, wherein detecting the swipe gesture includes detecting the swipe gesture on at least a portion of the currently displayed content.

454. The processing unit Before detecting the swipe gesture, detect an input corresponding to a request to view the home screen of the electronic device, In response to detecting the input, stop displaying the content associated with the application and display the corresponding page of the home screen of the electronic device. And is further configured as The corresponding page is the initial page of a series of home screen pages, The swipe gesture is detected while the initial page of the home screen is displayed on the display unit. The electronic device according to claim 452 or 453.

455. The electronic device according to any one of claims 452 to 454, wherein the search interface is displayed so as to overlap the application semi-transparently.

456. The processing unit The electronic device according to any one of claims 452 to 455, further configured to determine at least a part of the at least one proposed search query based on the text content according to a determination that the content includes text content.

457. Determining the at least one proposed search query based at least in part on the text content includes detecting one or more defined keywords used to analyze the text content to determine the at least one proposed search query. The electronic device according to claim 456.

458. Determining the at least one proposed search query includes determining a plurality of proposed search queries, Populating the search interface includes populating the search interface with the plurality of proposed search queries. The electronic device according to any one of claims 452 to 457.

459. The processing unit Detect a new swipe gesture on the newly displayed content via the touch sensing surface unit, In response to detecting the new swipe gesture, transitioning to the search mode includes displaying the search interface on the display unit, transitioning to the search mode, In conjunction with transitioning to the search mode and according to a determination that the new content does not include text content, the search interface is populated with search queries proposed based on a set of selected past search queries by the user of the electronic device. The electronic device according to any one of claims 452 to 458, further configured as described above.

460. The electronic device according to claim 459, wherein the search interface is displayed together with a target location based on location information provided by a second application different from the application.

461. The electronic device according to claim 459 or 460, wherein the search interface further includes one or more proposed applications. **Claim 462** The electronic device according to any one of claims 459 to 461, wherein the set of past search queries is selected at least in part based on the frequency of recent search queries. **Claim 463** The processing unit is further configured to obtain the information associated with the content by using one or more accessibility functions available on the electronic device in conjunction with transitioning to the search mode, according to any one of claims 452 to 462. **Claim 464** Using the one or more accessibility functions includes (i) applying a natural language processing algorithm to text content currently displayed within the application; and (ii) determining one or more keywords for explaining the content using data obtained from the natural language processing algorithm, wherein the at least one proposed search query is determined based on the one or more keywords, thereby generating the information associated with the content using the one or more accessibility functions, according to claim 463. **Claim 465** Determining the one or more keywords for explaining the content includes (i) extracting metadata corresponding to non-text content currently displayed within the application; and (ii) determining the one or more keywords using the extracted metadata in addition to the data obtained from the natural language processing algorithm, also included in the electronic device according to claim 464. **Claim 466** The electronic device according to any one of claims 452 to 465, wherein the search interface further includes one or more trend queries. **Claim 467** The electronic device according to claim 466, wherein the search interface further includes one or more applications that a user of the electronic device is expected to be interested in. **Claim 468** A non-transitory computer-readable storage medium storing executable instructions, wherein when the executable instructions are executed by an electronic device having a touch-sensitive surface and a display, the electronic device is caused to Via the touch sensing surface, detect a swipe gesture on the user interface, where the swipe gesture causes the electronic device to transition to a search mode when the swipe gesture is detected. In response to detecting the swipe gesture, transitioning to the search mode includes populating a search interface different from the user interface with a first content item before receiving any user input within the search interface. According to a determination that the user interface includes content associated with an application different from a home screen including selectable icons for calling applications, populating the search interface with the first content item includes populating the search interface, at least in part, with at least one proposed search query based on the content associated with the application. According to a determination that the user interface is associated with a page of the home screen, populating the search interface with the first content item includes populating the search interface with an affordance including a selectable description of at least one target location within a threshold distance of the current location of the electronic device. Non-transitory computer-readable storage medium.

469. Populating the search interface with the affordance includes displaying a search entry portion of the search interface on the touch sensing surface. When the instructions are executed by the electronic device, the electronic device further Detect an input in the search entry portion, In response to detecting the input in the search entry portion, stop displaying the affordance and display the at least one proposed search query within the search interface. The non-transitory computer-readable storage medium according to claim 468.

470. In an electronic device including one or more processors, a memory, a touch sensing surface, and a display, Detecting a swipe gesture on a user interface via the touch sensing surface, wherein the swipe gesture causes the electronic device to transition to a search mode when detected. In response to detecting the swipe gesture, transitioning to the search mode, which includes populating a search interface different from the user interface with a first content item before receiving any user input within the search interface. Comprising: In accordance with a determination that the user interface includes content associated with an application different from a home screen including selectable icons for invoking applications, populating the search interface with the first content item includes populating the search interface, at least in part, with at least one proposed search query based on the content associated with the application. In accordance with a determination that the user interface is associated with a page of the home screen, populating the search interface with the first content item includes populating the search interface with affordances including selectable descriptions of at least one target location within a threshold distance of the current location of the electronic device.

471. Populating the search interface with the affordances includes displaying a search entry portion of the search interface on the touch sensing surface. The method further comprises: Detecting an input in the search entry portion; In response to detecting the input in the search entry portion, ceasing to display the affordances and displaying the at least one proposed search query within the search interface. The method according to claim 470, further comprising:

472. A touch sensing surface; A display; One or more processors; A memory storing one or more programs configured to be executed by the one or more processors; An electronic device, wherein the one or more programs include instructions for performing the method according to claim 470 or 471.

473. A graphical user interface on an electronic device having a touch sensing surface, a display, a memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of claim 470 or 471.

474. A touch sensing surface, A display, Means for executing the method according to claim 470 or 471, An electronic device comprising.

475. An information processing apparatus for use in an electronic device including a touch sensing surface and a display, The information processing apparatus comprising means for executing the method according to any one of claims 470 to 471.

476. An electronic device, A display unit configured to display a user interface, A touch sensing surface unit configured to receive a user's contact, A processing unit coupled to the touch sensing surface unit and the display unit, Comprising, the processing unit, Via the touch sensing surface unit, detecting a swipe gesture on the user interface, the swipe gesture causing the electronic device to transition to a search mode when the swipe gesture is detected, In response to detecting the swipe gesture, transitioning to the search mode includes populating a search interface different from the user interface with a first content item before receiving any user input within the search interface. Configured as such, according to the determination that the user interface includes content associated with an application different from a home screen including selectable icons for calling applications, populating the search interface with the first content item includes populating the search interface, at least in part, with at least one proposed search query based on the content associated with the application. In accordance with the determination that the user interface is associated with a page of the home screen, populating the search interface with the first content item includes populating the search interface with an affordance that includes a selectable description of at least one target location within a threshold distance of the current location of the electronic device. An electronic device.

477. Populating the search interface with the affordance includes displaying a search entry portion of the search interface. The processing unit detects an input in the search entry portion, and in response to detecting the input in the search entry portion, stops displaying the affordance and displays the at least one proposed search query within the search interface. The electronic device according to claim 476, further configured as described above.

478. A non-transitory computer-readable storage medium storing executable instructions, wherein when the executable instructions are executed by an electronic device having a touch-sensitive surface, a display, and a position sensor, the electronic device automatically and without an instruction from the user determines that the user of the electronic device is in a vehicle stopped at a geographical location, when it is determined that the user has left the vehicle at the geographical location, determines whether position information retrieved from the position sensor to identify the geographical location meets an accuracy criterion, when it is determined that the position information does not meet the accuracy criterion, provides a prompt to the user to input information regarding the geographical location, and in response to providing the prompt, receives information about the geographical location from the user and stores the information as vehicle position information. A non-transitory computer-readable storage medium.

479. When the executable instructions are executed by the electronic device, the electronic device stores the position information as the vehicle position information automatically and without an instruction from the user when it is determined that the position information meets the accuracy criterion. The non-transitory computer-readable storage medium according to claim 478.

480. When the executable instructions are executed by the electronic device, the electronic device According to the determination that the user is heading towards the geographical location, display a user interface object including the vehicle position information. The non-transitory computer-readable storage medium according to claim 479.

481. The non-transitory computer-readable storage medium according to claim 480, wherein the user interface object is a map object including an identifier regarding the current position of the user and another identifier regarding the geographical location.

482. The non-transitory computer-readable storage medium according to claim 481, wherein the user interface object is displayed on the lock screen of the electronic device.

483. The non-transitory computer-readable storage medium according to claim 481, wherein the user interface object is displayed in response to a swipe gesture that causes the electronic device to transition to a search mode.

484. The non-transitory computer-readable storage medium according to claim 483, wherein determining whether the user is heading towards the geographical location is performed in response to receiving the swipe gesture.

485. The prompt is an audio prompt provided by a virtual assistant available via the electronic device. Receiving the information from the user includes receiving an oral description from the user that specifies the geographical location. Displaying the user interface object includes, when selected, displaying a selectable affordance that causes the device to play back the oral description. The non-transitory computer-readable storage medium according to any one of claims 478 to 484.

486. The prompt is displayed on the display of the electronic device. Receiving the information from the user includes receiving a text description from the user that specifies the geographical location. Displaying the user interface object includes displaying the text description from the user. The non-transitory computer-readable storage medium according to any one of claims 478 to 484.

487. Determining whether the user is heading towards the geographical location includes determining that the electronic device is moving towards the geographical location using new location information received from the location sensor, the non-transitory computer-readable storage medium according to any one of claims 478 to 484.

488. Determining whether the user is heading towards the geographical location (i) determining that the electronic device has remained at a different geographical location for a threshold time; and (ii) determining that the new location information indicates that the electronic device is moving away from the different geographical location and towards the geographical location, the non-transitory computer-readable storage medium according to claim 487.

489. Determining that the user is in the vehicle stopped at the geographical location includes (i) determining that the user is in the vehicle by determining that the electronic device is moving at a speed exceeding a threshold speed; and (ii) (a) determining that the electronic device has remained at the geographical location for a threshold time, (b) determining that the communication link between the electronic device and the vehicle has been disconnected, and (c) determining that the geographical location corresponds to a location within a parking lot, to determine that the vehicle has stopped at the geographical location, the non-transitory computer-readable storage medium according to any one of claims 478 to 488.

490. Determining that the vehicle has stopped at the geographical location includes determining that the electronic device has remained at the geographical location for a threshold time, the non-transitory computer-readable storage medium according to claim 489.

491. Determining that the vehicle has stopped at the geographical location includes determining that the communication link between the electronic device and the vehicle has been disconnected, the non-transitory computer-readable storage medium according to claim 489 or 490.

492. Determining that the vehicle has stopped at the geographical location includes determining that the geographical location corresponds to a location within a parking lot, the non-transitory computer-readable storage medium according to any one of claims 489 to 491.

493. The non-transitory computer-readable storage medium according to any one of claims 478 to 492, wherein the accuracy criterion includes a criterion that is satisfied when the accuracy of the GPS reading value associated with the position information exceeds an accuracy threshold level.

494. In an electronic device including one or more processors, a memory, a touch-sensitive surface, a display, and a position sensor, automatically and without an instruction from a user, determine that the user of the electronic device is in a stopped vehicle at a geographical location; when it is determined that the user has left the vehicle at the geographical location, determine whether position information retrieved from the position sensor to identify the geographical location satisfies an accuracy criterion; when it is determined that the position information does not satisfy the accuracy criterion, provide a prompt to the user to input information regarding the geographical location; in response to the provision of the prompt, receive information regarding the geographical location from the user and store the information as vehicle position information; A method comprising the steps of.

495. When it is determined that the position information satisfies the accuracy criterion, automatically and without an instruction from a user, store the position information as the vehicle position information. The method according to claim 494.

496. The method according to claim 495, comprising displaying a user interface object including the vehicle position information in accordance with a determination that the user is heading towards the geographical location.

497. The method according to claim 496, wherein the user interface object is a map object including an identifier regarding the current position of the user and another identifier regarding the geographical location.

498. The method according to claim 497, wherein the user interface object is displayed on a lock screen of the electronic device.

499. The method according to claim 497, wherein the user interface object is displayed in response to a swipe gesture that causes the electronic device to transition to a search mode.

500. The method according to claim 499, wherein determining whether the user is heading towards the geographical location is performed in response to receiving the swipe gesture.

501. The prompt is an audio prompt provided by a virtual assistant available via the electronic device, Receiving the information from the user includes receiving an oral description from the user that identifies the geographical location, Displaying the user interface object includes displaying a selectable affordance that, when selected, causes the device to play the oral description, The method according to any one of claims 494 to 500.

502. The prompt is displayed on the display of the electronic device, Receiving the information from the user includes receiving a text description from the user that identifies the geographical location, Displaying the user interface object includes displaying the text description from the user, The method according to any one of claims 494 to 500.

503. Determining whether the user is heading towards the geographical location includes determining, using new location information received from the location sensor, that the electronic device is moving towards the geographical location, according to the method of any one of claims 494 to 500.

504. Determining whether the user is heading towards the geographical location includes (i) determining that the electronic device has remained at a different geographical location for longer than a threshold time, and (ii) determining that the new location information indicates that the electronic device is leaving the different geographical location and heading towards the geographical location, The method according to claim 503, including.

505. Determining that the user is in the vehicle stopped at the geographical location includes (i) determining that the user is in the vehicle by determining that the electronic device is moving at a speed exceeding a threshold speed, and (ii) (a) determining that the electronic device has remained at the geographical location for longer than a threshold time, (b) determining that the communication link between the electronic device and the vehicle has been disconnected, and (c) determining that the geographical location corresponds to a location within a parking lot, and one or more of the above, (ii) determining that the vehicle has stopped at the geographical location, according to the method of any one of claims 494 to 504.

506. The method according to claim 505, wherein determining that the vehicle has stopped at the geographical location includes determining that the electronic device has remained at the geographical location for a time exceeding a threshold time.

507. The method according to claim 505 or 506, wherein determining that the vehicle has stopped at the geographical location includes determining that a communication link between the electronic device and the vehicle has been disconnected.

508. The method according to any one of claims 505 to 507, wherein determining that the vehicle has stopped at the geographical location includes determining that the geographical location corresponds to a location within a parking lot.

509. The method according to any one of claims 494 to 508, wherein the accuracy criterion includes a criterion that is satisfied when the accuracy of a GPS reading associated with the location information exceeds a threshold level of accuracy.

510. A touch sensing surface, A display, A position sensor, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, Comprising, wherein the one or more programs include instructions for executing the method according to any one of claims 494 to 509, An electronic device.

511. A graphical user interface on an electronic device having a touch sensing surface, a display, a position sensor, a memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 494 to 509.

512. A touch sensing surface, A position sensor, A display, Means for executing the method according to any one of claims 494 to 509, An electronic device.

513. An information processing apparatus for use in an electronic device including a touch sensing surface, a display, and a position sensor, Comprising means for executing the method according to any one of claims 494 to 509, An information processing apparatus.

514. An electronic device, A display unit configured to display a user interface, A touch sensing surface unit configured to receive a user's contact, A position sensor unit configured to obtain position information of the electronic device; A touch sensing surface unit, the display unit, and a processing unit coupled to the position sensor unit; Comprising, the processing unit is configured to: Automatically and without an instruction from the user, Determine that the user of the electronic device is in a vehicle stopped at a geographical location, When it is determined that the user has left the vehicle at the geographical location, determine whether the position information retrieved from the position sensor unit to identify the geographical location meets an accuracy criterion; When it is determined that the position information does not meet the accuracy criterion, provide a prompt to the user to input information regarding the geographical location; In response to the provision of the prompt, receive information regarding the geographical location from the user and store the information as vehicle position information. An electronic device configured as described above.

515. The processing unit is further configured to: When it is determined that the position information meets the accuracy criterion, automatically and without an instruction from the user, store the position information as the vehicle position information, as described in claim 514.

516. The processing unit is further configured to: In accordance with the determination that the user is heading towards the geographical location, display a user interface object including the vehicle position information, as described in claim 515.

517. The user interface object is a map object including an identifier regarding the user's current position and another identifier regarding the geographical location, as described in claim 516.

518. The user interface object is displayed on the lock screen of the electronic device, as described in claim 517.

519. The user interface object is displayed in response to a swipe gesture that causes the electronic device to transition to a search mode, as described in claim 517.

520. Determining whether the user is heading towards the geographical location is performed in response to receiving the swipe gesture, as described in claim 519.

521. The prompt is an audio prompt provided by a virtual assistant available via the electronic device, Receiving the information from the user includes receiving from the user an oral description that identifies the geographical location, Displaying the user interface object includes displaying a selectable affordance that, when selected, causes the device to reproduce the oral description, An electronic device according to any one of claims 514 to 520.

522. The prompt is displayed on the display of the electronic device, Receiving the information from the user includes receiving from the user a text description that identifies the geographical location, Displaying the user interface object includes displaying the text description from the user, An electronic device according to any one of claims 514 to 520.

523. Determining whether the user is heading towards the geographical location includes determining, using new location information received from the position sensor unit, that the electronic device is moving towards the geographical location, An electronic device according to any one of claims 514 to 520.

524. Determining whether the user is heading towards the geographical location is (i) determining that the electronic device has remained at a different geographical location for a threshold time, and (ii) determining that the new location information indicates that the electronic device is leaving the different geographical location and heading towards the geographical location, An electronic device according to claim 523, comprising.

525. Determining that the user is in the vehicle stopped at the geographical location includes: (i) determining that the user is in the vehicle by determining that the electronic device is moving at a speed exceeding a threshold speed; and (ii) (a) determining that the electronic device has remained at the geographical location for a time exceeding a threshold time, (b) determining that a communication link between the electronic device and the vehicle has been disconnected, and (c) determining that the geographical location corresponds to a location within a parking lot, and determining that the vehicle has stopped at the geographical location, and the electronic device according to any one of claims 514 to 524.

526. Determining that the vehicle has stopped at the geographical location includes determining that the electronic device has remained at the geographical location for a time exceeding a threshold time, for the electronic device according to claim 525.

527. Determining that the vehicle has stopped at the geographical location includes determining that a communication link between the electronic device and the vehicle has been disconnected, for the electronic device according to claim 525 or 526.

528. Determining that the vehicle has stopped at the geographical location includes determining that the geographical location corresponds to a location within a parking lot, for the electronic device according to any one of claims 525 to 527.

529. The accuracy criterion includes a criterion satisfied when the accuracy of the GPS reading associated with the position information exceeds an accuracy threshold level, for the electronic device according to any one of claims 514 to 528.

530. A non-transitory computer-readable storage medium storing executable instructions, wherein when the executable instructions are executed by an electronic device having a touch-sensitive surface, a display, and a position sensor, the electronic device is caused to without receiving any instructions from a user of the electronic device, use the position sensor to monitor the geographical location of the electronic device, based on the monitored geographical location, determine that the electronic device is within a threshold distance of a target location of a predetermined type, in accordance with determining that the electronic device is within the threshold distance of the target location, identify at least one currently popular activity at the target location Causing information about the target location to be retrieved, including retrieving information about at least one currently popular activity at the target location. Causing a first input to be detected via the touch sensing surface, which, when detected, causes the electronic device to transition to a search mode. In response to detecting the first input, transitioning to the search mode includes presenting, via the display, an affordance that includes (i) the information about the at least one activity and (ii) an indication that the at least one activity is currently popular at the target location, before receiving any user input at the search interface. Non-transitory computer-readable storage medium.

531. When the executable instructions are executed by the electronic device, causing the electronic device to: Detect a second input. In response to detecting the second input, updating the affordance to include available information about current activities at a second target location different from the target location, where the target location is within the threshold distance range of the electronic device. The non-transitory computer-readable storage medium according to claim 530.

532. The affordance further includes selectable categories of target locations. When the executable instructions are executed by the electronic device, causing the electronic device to: Detect a selection of a corresponding selectable category. In response to detecting the selection, updating the affordance to include information about additional target locations located within a second threshold distance range of the device. The non-transitory computer-readable storage medium according to claim 530 or 531.

533. The non-transitory computer-readable storage medium according to any one of claims 530 to 532, wherein the target location is an amusement park and the retrieved information includes the current waiting time for rides at the amusement park.

534. The non-transitory computer-readable storage medium according to claim 533, wherein the retrieved information includes information about the waiting time for rides located within a default distance range of the electronic device.

535. The non-transitory computer-readable storage medium according to any one of claims 530 to 532, wherein the target location is a restaurant, and the information to be retrieved includes information regarding popular menu items of the restaurant.

536. The non-transitory computer-readable storage medium according to claim 535, wherein the information to be retrieved is retrieved from a social network associated with the user of the electronic device.

537. The non-transitory computer-readable storage medium according to any one of claims 530 to 532, wherein the target location is a movie theater, and the information to be retrieved includes information regarding showtimes at the movie theater.

538. The non-transitory computer-readable storage medium according to claim 537, wherein the information to be retrieved is retrieved from a social network associated with the user of the electronic device.

539. The non-transitory computer-readable storage medium according to any one of claims 530 to 538, wherein after unlocking the electronic device, the affordance is available in response to a substantially horizontal swipe on the initial page of the home screen of the electronic device.

540. In an electronic device including one or more processors, a memory, a touch-sensitive surface, a display, and a position sensor, without receiving any commands from a user of the electronic device, using the position sensor to monitor the geographical location of the electronic device; determining, based on the monitored geographical location, that the electronic device is within a threshold distance of a target location of a predetermined type; in accordance with determining that the electronic device is within the threshold distance of the target location, identifying at least one currently popular activity at the target location; retrieving information regarding the target location, including retrieving information regarding at least one currently popular activity at the target location; detecting, via the touch-sensitive surface, a first input that, when detected, causes the electronic device to transition to a search mode; transitioning to the search mode in response to detecting the first input; including, transitioning to the search mode includes presenting, via the display, an affordance including (i) the information regarding the at least one activity and (ii) an indication that the at least one activity is currently popular at the target location, before receiving any user input at the search interface. A method including the above.

541. Detecting a second input; In response to detecting the second input, updating the affordance to include available information regarding current activities at a second target location different from the target location; including, the target location being within the threshold distance of the electronic device. The method according to claim 540.

542. The affordance further includes selectable categories of target locations, and the method includes: Detecting a selection of a corresponding selectable category; In response to detecting the selection, updating the affordance to include information regarding additional target locations located within a second threshold distance of the device. The method according to claim 540 or 541, further including the above.

543. The method according to any one of claims 540 to 542, wherein the target location is an amusement park and the retrieved information includes the current waiting time of the rides in the amusement park.

544. The method according to claim 543, wherein the retrieved information includes information regarding the waiting time of rides located within a default distance range of the electronic device.

545. The method according to any one of claims 540 to 542, wherein the target location is a restaurant and the retrieved information includes information regarding popular menu items of the restaurant.

546. The method according to claim 545, wherein the retrieved information is retrieved from a social network associated with the user of the electronic device.

547. The method according to any one of claims 540 to 542, wherein the target location is a movie theater and the retrieved information includes information regarding the screening times of the movie theater.

548. The method according to claim 547, wherein the retrieved information is retrieved from a social network associated with the user of the electronic device.

549. The method according to any one of claims 540 to 548, wherein after unlocking the electronic device, the affordance is available in response to a substantially horizontal swipe on the initial page of the home screen of the electronic device.

550. A touch-sensing surface, A display, A position sensor, One or more processors, A memory storing one or more programs configured to be executed by the one or more processors, Comprising, wherein the one or more programs include instructions for performing the method according to any one of claims 540 to 549, An electronic device.

551. A graphical user interface on an electronic device having a touch-sensing surface, a display, a position sensor, a memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method according to any one of claims 540 to 549.

552. A touch-sensing surface, A position sensor, A display, Means for performing the method according to any one of claims 540 to 549, An electronic device.

553. An information processing apparatus for use in an electronic device including a touch-sensing surface, a display, and a position sensor, The information processing apparatus comprising means for performing the method according to any one of claims 540 to 549.

554. An electronic device, A display unit configured to display a user interface, A touch-sensing surface unit configured to receive a user's touch, A position sensor unit configured to obtain a geographical location, A processing unit coupled to the touch-sensing display unit, the processing unit Without receiving any commands from the user of the electronic device, Using the position sensor unit to monitor the geographical location of the electronic device, Based on the monitored geographical location, determining that the electronic device is within a threshold distance of a predefined type of target location, In accordance with determining that the electronic device is within the threshold distance of the target location, Identifying at least one currently popular activity at the target location, Retrieving information about the target location, including retrieving information about at least one currently popular activity at the target location, Detecting, via the touch sensing surface unit, a first input that, when detected, causes the electronic device to transition to a search mode, In response to detecting the first input, transitioning to the search mode includes presenting, via the display unit, an affordance that includes (i) the information about the at least one activity and (ii) an indication that the at least one activity is currently popular at the target location, before receiving any user input at the search interface. An electronic device configured as such.

555. The processing unit, Detects a second input, And in response to detecting the second input, is further configured to update the affordance to include available information about current activities at a second target location different from the target location, where the target location is within the threshold distance range of the electronic device. The electronic device according to claim 554.

556. The affordance further includes selectable categories of target locations, and the processing unit, Detects a selection of a corresponding selectable category, And in response to detecting the selection, updates the affordance to include information about additional target locations located within a second threshold distance range of the device. The electronic device according to claim 554 or 555, further configured as such.

557. The target location is an amusement park, and the retrieved information includes the current waiting time for rides at the amusement park. The electronic device according to any one of claims 554 to 556.

558. The retrieved information includes information about the waiting time for rides located within a predefined distance range of the electronic device. The electronic device according to claim 557.

559. The target location is a restaurant, and the retrieved information includes information about popular menu items at the restaurant. The electronic device according to any one of claims 554 to 556.

560. The electronic device according to claim 559, wherein the information to be retrieved is retrieved from a social network associated with the user of the electronic device.

561. The electronic device according to any one of claims 554 to 556, wherein the target location is a movie theater, and the information to be retrieved includes information regarding the screening time of the movie theater.

562. The electronic device according to claim 561, wherein the information to be retrieved is retrieved from a social network associated with the user of the electronic device.

563. The electronic device according to any one of claims 554 to 562, wherein after the electronic device is unlocked, the affordance is available in response to a substantially horizontal swipe on the initial page of the home screen of the electronic device.

Images