Uniform Resource Identifiers and Image Sharing for Context Information Display

Abstract

The present invention relates to uniform resource identifiers and image sharing for context information display. A method, system, and apparatus, including a computer program encoded on a computer storage medium, the method, system, and apparatus for generating a message including an image of a specific environment instance of a native application and a uniform resource identifier of the specific environment instance of the native application in response to a single input operating system command invoked independent of the native application, the image being displayed upon the single input operating system command.

Description

[0001] This application is a divisional application of a patent application for invention with an application date of August 31, 2017, an application number of 201780069465.4, and an invention title of "Uniform Resource Identifiers and Image Sharing for Context Information Display". Technical Field

[0002] This application relates to uniform resource identifiers and image sharing. Background Art

[0003] The Internet provides access to various information. For example, digital image files, video and / or audio files, and web page resources for specific topics or specific news articles can be accessed through the Internet. Regarding web page resources, many of these resources are designed to facilitate the execution of specific functions (such as banking, booking hotel reservations, shopping, etc.) or provide structured information (such as online encyclopedias, movie databases, etc.). Search engines crawl and index these resources to facilitate resource search.

[0004] In addition, with the emergence of tablet computers and smart phones, native applications that facilitate the performance of the same functions promoted by using web page resources are now abundantly provided. Additionally, native applications without websites having synchronized content (such as games) are also very popular on tablet computers and smart phones. Therefore, search engines now also facilitate the search for these native applications.

[0005] A user may expect information related to a web page resource or a native application that is active on the user's device. Such information is referred to as "contextual information" because the information expected by the user is in the context of the active web page or native application.

[0006] In the context of a native application, a user may also expect to share information about the native application with other users. Some systems allow a user to take a screenshot of a specific environmental instance of the native application and send the screenshot to other users. The user can also manually input or select information about the native application and also send that information to other users. Summary of the Invention

[0007] This specification relates to sharing context information of an active native application on a first user device with other user devices.

[0008] Generally, one innovative aspect of the subject matter described in this specification can be embodied in a method including the following actions, which include: displaying, on a display of a user device, a particular environment instance of a native application that can display multiple different environment instances, where the native application is an application operable to generate environment instances for display within the native application on the user device and operates independently of a browser application that can operate on the user device; receiving, at the user device, a single input operating system command when the particular environment instance is the active focus of the display, the single input operating system command being a command invoked independently of the native application; in response to the single input operating system command: displaying, on the display, a context information card including a share command and at least one or more additional commands; in response to a selection of the share command: determining a recipient address specifying a recipient device separate from the user device, generating an image of the particular environment instance of the native application, selecting a uniform resource identifier of the particular environment instance of the native application, generating a message including the image of the particular environment instance and the uniform resource identifier, and sending the message including the image of the particular environment instance and the uniform resource identifier to the recipient address. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each computer system, apparatus, and computer program being configured to perform the above actions of the method. A system of one or more computers can be configured to perform particular operations or actions by installing software, firmware, hardware, or a combination thereof on the system, the software, firmware, hardware, or a combination thereof causing the system to perform the above actions in operation. One or more computer programs can be configured to perform particular operations or actions by including instructions that, when run by a data processing apparatus, cause the apparatus to perform the above actions.

[0009] Specific embodiments of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. Context information including a screenshot of an active native application and a Uniform Resource Identifier (URI) of a specific environmental instance of the native application captured in the screenshot can be shared by using an intuitive and easy-to-use interaction model. This enables a user to quickly and effectively share desired information about the native application with other users, and enables other users to instantiate (e.g., launch, or install and launch) the native application on their receiving devices to process the URI and generate the URI (or, similarly, the environmental instance depicted in the screenshot). The shared context information represents an efficient use of information because it is not necessary to share the running code of the native application in order to recreate the environmental instance on the receiving user's device. The network traffic between users during the sharing process is reduced, and the load on the sharing user's device (from the perspective of generating the information to be shared) and the receiving user's device (from the perspective of interpreting and processing the received information) is reduced. Nevertheless, the receiving user can still easily determine the description of the environmental instance and the manner in which the native application should be launched, or installed and launched, to generate the same or a similar environmental instance.

[0010] Additionally, because the input of a query-independent request for context information does not require query input and the sharing command does not require manual selection of a URI, the system does not need to perform text-to-speech processing or process typed input. This can reduce input errors and incorrect input queries. Thus, when considered in the aggregate, thousands of errors as well as inaccurate queries and messages are avoided, which in turn provides a more efficient use of network resources. This improves the technical field of search processing and information sharing between users and is thus another significant advantage achieved by the systems and methods described below. Using a query-independent request results in an efficient input mechanism because it reduces the amount of input to be processed compared to an input mechanism that must process the format or content of a specific query in order to obtain appropriate context information. Additionally, it is not necessary to separately configure the processing of requests for context information for different scenarios because the requests can be independent of a specific environment or native application. This further improves the efficiency with which context information can be obtained.

[0011] Details of one or more embodiments of the subject matter described in this specification are set forth in conjunction with the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the specification, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A and Figure 1BA block diagram of an example environment that provides context information for a native application and shares screenshots and uniform resource identifiers (URIs) of the native application.

[0013] Figure 2 A flowchart of an example process for sharing an image and URI of a native application.

[0014] Figure 3 A block diagram of an example environment that provides context information for active content.

[0015] Figure 4 A block diagram of an example mobile computing device.

[0016] Like reference numerals and names in the various figures represent like elements. Detailed Description

[0017] A user device implementing the systems and methods described below enables capturing screenshots of an active native application and the uniform resource identifier of a particular environment instance of the native application captured in the screenshot. The capture and sharing of such information is achieved through the use of an intuitive and easy-to-use context interaction model.

[0018] As used herein, a native application generates an environment instance for display on a user device within the environment of the native application and operates independently of a browser application on the user device. A native application is an application specifically designed to run on a particular user device operating system and machine firmware. Thus, a native application is different from browser-based applications and resources presented by a browser. The latter require all or at least some elements or instructions to be downloaded from a web server each time they are instantiated or presented. Additionally, browser-based applications and resources presented by a browser can be processed by all network-capable mobile devices within a browser and are thus not specific to an operating system. Each particular environment instance of a native application is specified by a "deep link", which otherwise may be referred to as a uniform resource identifier (URI). The URI can be constructed according to a particular standard or can be a structure unique to a particular native application. The URI enables the instantiation of a native application instance that specifies the native application.

[0019] By using a single input operating system command, a user can receive context information related to an active environment instance of a native application. The user can then select a share command that generates a message including a screenshot of the active environment instance of the native application, as well as a "deep link" to the active environment instance, e.g., a Uniform Resource Identifier. The user can then send the screenshot and the deep link in a single message to one or more other users. Upon receipt, each receiving user can view the screenshot. If the viewing user desires to instantiate the same native application on the receiving device, the user can select a command (such as "long press", e.g., interacting with the image in some way) to instantiate the native application on the receiving device. The native application can then process the URI to generate the same (or a similar) environment instance.

[0020] These features and additional features are described in more detail below.

[0021] Figure 1A and Figure 1B is a block diagram of an example environment in which context information is provided for a native application and a screenshot and Uniform Resource Identifier (URI) of the native application are shared. Specifically, Figure 1A depicts requesting and receiving a context information card when a particular environment instance of the native application is active. Specifically, Figure 1B depicts sharing a screenshot of a particular environment instance of the native application and a deep link to the particular environment instance.

[0022] Referring to Figure 1A , a user uses a user device 110 to obtain context information for a native application. The user device 110 is an electronic device capable of requesting and receiving resources over a network. Example user devices 110 include mobile communication devices (e.g., smart phones or tablets), and other devices that can send and receive data over a network. The native application is operating, and a particular environment instance 112 is the active focus (e.g., the active window) and immediately responds to input received at the display of the user device 110.

[0023] The user device can also display other information, such as network resources. These may include HTML pages, electronic documents, image files, video files, text message conversations, emails, graphical user interfaces of applications, etc. Similar to the native application environment instance, a resource can also be the active focus of the display. For simplicity, an active resource or an active native application environment instance is referred to as "active content".

[0024] The user device 110 detects context information of the user's desired activity content. For example, the user device 110 may be displaying a particular native application environment instance 112, and the activity focus may be on the environment instance 112. For example, the environment instance 112 may include a graphic 114 and text 116. The user may generate an indication of the context information (e.g., by pressing a button 111 (e.g., the "home" button) for three seconds or by clicking on the screen according to a predefined click pattern, etc.). In some embodiments, such a command is a single input operating system command invoked independent of the native application. In other words, the same command may be used in multiple different native applications and network resources.

[0025] For illustrative purposes, assume that the user performs a long press, which indicates that the user desires context information for a particular native application environment instance 112. In response, the user device 110 provides a query-independent request 113 to the context information server 150 to obtain context information for the particular native application environment instance 112. For example, the user device 110 may provide a request that includes a screenshot of the particular native application environment instance, information about the native application, a URI of the particular native application environment instance, etc.

[0026] This request may be considered a query-independent request because the user device 110 provides the request to the context information server 150 without the user (either verbally, physically, or some other interaction) entering query terms. For example, after the user device 110 detects that the user has long pressed a button, the user device 110 may provide the request to the context information server 150 without requesting additional information from the user.

[0027] In response to providing the request to the context information server 150, the user device 110 then receives one or more user interface elements from the context information server 150. As Figure 1A shown, the received information is a context information card 160. However, other user interface elements may be used, such as chat bubbles, selectable link annotations, or footnotes, etc. The context information card 160 may include context information (such as images, text, audio, and video) about the native application and / or a particular environment instance of the native application, as well as one or more commands 162. As Figure 1A shown, the commands 162 include a search command that can invoke a separate search in the search user interface, a review command that can retrieve reviews of the native application, and a share command that can generate a screenshot of the environment instance 112 and the URI of the environment instance and send the screenshot and URI to another user device.

[0028] Figure 1BDepicts the process flow in response to selecting the share command in the context information card 160. Main reference Figure 2 Describes the process flow Figure 2 Is a flowchart of an example process 200 for sharing images and URIs of native applications.

[0029] In Figure 1B the context information card 160 has the active focus of the user device 110 as it is displayed in the foreground. To further emphasize the active focus on card 162, a specific environment instance 112 on the user device 110 can be obscured or shown beneath a slightly opaque layer, as indicated by the dashed lines of the image elements.

[0030] To reach the Figure 1B state, the user device first displays a specific environment instance (202) of the native application and receives a single input operating system command when the specific environment instance is in the active focus state, where the single input operating system command is a command invoked independent of the native application (204). For example, in Figure 1A the user device 110 displays the environment instance 112, and when the environment instance 112 is the active focus of the user device, the user device receives a long press on the home button 111.

[0031] In response to the single input operating system command, the user device displays a context information card (206) including the share command and at least one or more additional commands. For example, in Figure 1B the user device 110 receives the context information card 160 and displays card 160. Card 160 includes the "Share" command, and as shown by the bold outline, the share command has been selected.

[0032] In response to the selection of the share command, the user device generates a message (208) including an image 122 of the specific environment instance 112 of the native application and a uniform resource identifier 124 of the specific environment instance. Image 122 can be generated by invoking a system command that generates an image from the viewport data of the native application.

[0033] For example, the uniform resource identifier 124 of the specific environment instance 112 of the native application can be selected from the user device cache of recently used uniform resource identifiers. For example, the operating system of the user device 110 can track the uniform resource identifiers most recently selected by each native application in a local history cache. In a variant of this embodiment, each native application can be configured to provide the uniform resource identifiers it processes to the cache.

[0034] Optionally, the operating system of the user device 10 may request a uniform resource identifier 124 for a specific environment instance 112 of the native application from the native application and from the operating system. For example, the operating system of the user device 10 may send a request through an application programming interface (API) configured to communicate with the native application and receive the uniform resource identifier in response to the request.

[0035] The recipient address for sharing is determined, such as through the user's selection or a default address. Multiple recipient addresses may also be selected. Then, the user device 110 sends a message (210) including the image 122 of the specific environment instance 112 and the uniform resource identifier 124 to the recipient address. For example, the message may be sent as a text message, a notification, or as a message processed by a digital assistant. Other message types may also be used.

[0036] In some embodiments, the user device 110 generates the message by embedding the uniform resource identifier 124 in the image metadata of the image 122. Thus, when received by a user device such as Figure 1B the user device 130, the image 122 is displayed, but the uniform resource identifier 124 is not easily displayed to the user. In other embodiments, the user device includes the uniform resource identifier 124 and the image 122 as separate items that are displayed separately from each other in the display of the user device 130.

[0037] The user device 130 may process a navigation command in response to the message. For example, if the uniform resource identifier 124 is embedded in the image 122, the user of the user device may "long press" the image 122, or if the uniform resource identifier is displayed separately from the image, the uniform resource identifier 124 may be selected.

[0038] In response to the navigation command, the user device 130 instantiates the native application at the specific environment instance 112 addressed by the uniform resource identifier 124. This is shown in the user device 130-B, which has been transformed from the image 122 to the actual environment instance 112.

[0039] In some embodiments, the environment instance 112 in the receiving user device 130 will be the same as the environment instance 112 in the sending user device 110. This occurs when the native application 112 does not require specific user information and the information in the environment instance 112 is published information (e.g., not personal information obtained through a login process specific to the sending user). The uniform resource identifier 124 may also include query parameters and other parameters processed by the native application to reach the specific environment instance 112.

[0040] The process of instantiating a native application involves launching the native application and having the native application process a Uniform Resource Identifier 124. If the native application is not installed on the user device 130, the user device can automatically install the native application from the native application provider as part of the instantiation, and then, launch the native application. Optionally, the user can be prompted to confirm the installation of the native application.

[0041] Although the sharing of the image of the environment instance 122 and the corresponding Uniform Resource Identifier 124 has been described in the context of a shared command from a context information card, the sharing of such information can also be provided independently of the context information card. For example, another type of single-input operating system command can be used to generate a message that includes the image 122 of a specific environment instance 112 and the Uniform Resource Identifier 124, without the need to display the context card 160 with the shared command. An example of such a command is pressing the button 111 in a predetermined sequence (e.g., four short taps within two seconds).

[0042] In another embodiment, the message that includes the image 122 and the Uniform Resource Identifier 124 can also include context information from the context card 160. For example, the context information can be a summary of the native application, a comment, or other information. Upon receiving the message, the user device 130-A can allow the user to "slide" the image 122 to the side to reveal the Uniform Resource Identifier and the context information. Then, the user can choose to launch the native application (or install and launch the native application) by selecting the Uniform Resource Identifier, or can instead view the context information and interact with the context information.

[0043] Figure 3 is a block diagram of an example environment that provides context information for active content. The environment 300 includes a client context module 310 on a user device 110 and a context information server 312. The context information server 312 includes an item identification engine 320, an item knowledge graph 322, a relevance scoring engine 330, an item selection engine 340, and a context card provider 350.

[0044] The client context module 310 determines the context information for the active content that the user desires. In response to determining that the user desires the context information for the active content, the client context module 310 generates a request for the context information of the active content to the context information server 312.

[0045] The client context module 310 can include information about the active content in the request. For example, the client context module 310 can generate a screenshot of the active content, provide a tree-based document object model that defines the content presented in the currently foregrounded application, and include the model in the request, etc.

[0046] In some embodiments, the client context module 310 can include information about the active content in the request by additionally or alternatively determining additional information about the request and including the additional information in the request. The additional information about the request can include one or more of a uniform resource identifier of the active content, metadata describing the active content, and the like.

[0047] Then, the client context module 310 provides the request to the item identification engine 320 without user input of a query. For example, the client context module 310 provides the request to the item identification engine 320 in response to an indication by the user providing context information of the user-desired active content (e.g., three quick presses of a button, a long press of a button, or some other indication), without the user providing any further information (e.g., a query term) after providing the indication.

[0048] In response to providing the request to the item identification engine 320, the client context module 320 receives and presents a context card. For example, a context card 160 of Figure 1A and Figure 1B can be received and displayed.

[0049] To generate the card 160, the item identification engine 320 receives a request for context information and identifies one or more search terms from the content in the active resource. The search terms can be referenced by text fragments (e.g., terms or phrases), and the search terms can be distinguished from each other (e.g., based on context). For example, the search terms can be keywords, as well as instances of specific entities, proper names, entities or relationships modeled by nodes or edges in a knowledge graph, and the like.

[0050] The relevance scoring engine 330 can receive the search terms identified by the item identification engine 320 and determine a relevance score for each of the search terms. Various relevance scoring processes can be used. For example, the relevance score can reflect the confidence that the search term is relevant to the user, i.e., the confidence that the user wishes to see context information about the search term. The relevance scoring engine 330 can also determine the relevance score of the search terms based on the appearance of the text, where the search terms are identified from the appearance of the text. For example, the relevance scoring engine 330 can determine that the active content is a text message conversation, and in response, determine a relevance score that reflects a higher confidence in the search terms identified from the text that appears closer to the bottom of the screen, as that text may be included in more recent messages. In another example, the relevance scoring engine 330 can determine that the active content is an applicant instance, and in response, determine a relevance score that reflects a higher confidence in the search terms identified from the text that appears closer to the top of the screen, as that text may be more relevant to the native application. Other scoring techniques can also be used.

[0051] The project selection engine 340 selects a subset of search terms based on relevance scores and provides the selected search terms to the context card provider 350. The context card provider 350 can obtain context cards including context information of the selected search terms and provide the context cards to the client context module 310. For example, the context card provider 350 can obtain a description of a native application and a link to reviews of the native application, and then generate a context information card 160 including the description and the link to the reviews.

[0052] In cases where the systems discussed herein collect personal information about a user or can make use of personal information, the user may be provided with an opportunity to control whether the program or feature collects user information (e.g., information about the user's social network, social actions or activities, profession, user preferences, or user's current location), or to control whether and / or how content that may be more relevant to the user is received from a content server. Additionally, certain data may be processed in one or more ways before it is stored or used to remove personally identifiable information. For example, the identity of a user may be processed so that the user's personally identifiable information cannot be determined, or the user's geographical location may be generalized where location information is obtained (such as to a city, ZIP code, or state level) so that the user's specific location cannot be determined. Thus, the user can control how information about the user is collected and used by the content server.

[0053] Figure 4 is a block diagram of an example mobile computing device. In this illustration, the mobile computing device 410 is depicted as a handheld mobile phone (e.g., a smartphone or feature phone), which includes a touchscreen display device 412 for presenting content to a user of the mobile computing device 410 and receiving touch-based user input. Other visual, tactile, and auditory output components (e.g., LED lights, a vibration mechanism for tactile output, or a speaker for providing tonal, voice-generated, or recorded output) may also be provided, as well as a variety of different input components.

[0054] An example visual output mechanism in the form of the display device 412 may take the form of a display with resistive or capacitive touch capabilities. The display device can be used to display video, graphics, images, and text, as well as to coordinate the position of user touch input with the position of the information being displayed, such that the device 410 can associate the user contact at the position of the displayed item with the item. The mobile computing device 410 may also take alternative forms, including a laptop computer, a tablet or slate computer, a personal digital assistant, an embedded system (e.g., an automotive navigation system), a desktop personal computer, or a computerized workstation.

[0055] The mobile computing device 410 may be able to determine the location of physical contact with the touchscreen display device 412 (e.g., the location of contact by a finger or a stylus). Using the touchscreen 412, various "virtual" input mechanisms may be generated, where the user interacts with the graphical user interface elements depicted on the touchscreen 512 by contacting the graphical user interface elements. An example of a "virtual" input mechanism is a "software keyboard", where the keyboard is displayed on the touchscreen, and the user selects keys by pressing the areas of the touchscreen 412 corresponding to each key.

[0056] The mobile computing device 410 may include mechanical or touch-sensitive buttons 418a-d. Additionally, the mobile computing device may include buttons for adjusting the volume output by one or more speakers 420, and buttons for turning the mobile computing device on or off. The microphone 422 allows the mobile computing device 410 to convert audible sound into an electrical signal that may be digitally encoded and stored in a computer-readable memory, or transmitted to another computing device. The mobile computing device 410 may also include a digital compass, an accelerometer, a proximity sensor, and an ambient light sensor.

[0057] The operating system may provide an interface between the hardware of the mobile computing device (e.g., input / output mechanisms and a processor that runs instructions retrieved from a computer-readable medium) and the software. The operating system may provide a platform for running application programs that facilitate interaction between the computing device and the user.

[0058] The mobile computing device 410 may utilize the touchscreen 412 to present a graphical user interface. The graphical user interface is a collection of one or more graphical interface elements, and may be static (e.g., the display appears to remain the same over a period of time), or may be dynamic (e.g., the graphical user interface includes graphical interface elements that are active without user input).

[0059] The graphical interface elements may be text, lines, shapes, images, or a combination thereof. For example, the graphical interface elements may be icons displayed on a desktop and the associated text of the icons. In some examples, the graphical interface elements may be selected using user input. For example, the user may select a graphical interface element by pressing the area of the touchscreen corresponding to the display of the graphical interface element. In some examples, the user may manipulate a trackball to highlight a single graphical interface element as having focus. The user selection of a graphical interface element may invoke a predefined action of the mobile computing device. The user selection of a button may invoke a predefined action.

[0060] The mobile computing device 410 may include other applications, computing subsystems, and hardware. The speech recognition service 472 may receive speech communication data received by the microphone 422 of the mobile computing device and convert the speech communication into corresponding text data or perform speech recognition. The processed speech data may be input into a command model stored in the command model data 122 to determine whether the speech input for generating the speech data invokes a specific action of a specific application as described above. One or more of the following applications, services, and units may have corresponding actions invoked by such a voice command.

[0061] The call processing unit may receive an indication of an incoming telephone call and provide the user with the ability to answer the incoming telephone call. The media player may allow the user to listen to music or play movies stored in the local memory of the mobile computing device 410. The mobile device 410 may include a digital camera sensor, as well as corresponding image and video capture and editing software. The Internet browser may enable the user to view content from a web page by typing an address corresponding to the web page or selecting a link to the web page.

[0062] A service provider operating a network of base stations may connect the mobile computing device 410 to the network 450 to enable communication between the mobile computing device 410 and other computing systems providing the service 460. Although the service 460 may be provided over different networks (e.g., the service provider's internal network, the public switched telephone network, and the Internet), the network 450 is shown as a single network. The service provider may operate a server system 452 that routes information packets and voice data between the mobile computing device 410 and the computing systems associated with the service 460.

[0063] The network 450 may connect the mobile computing device 410 to a Public Switched Telephone Network (PSTN) 462 to establish voice or fax communication between the mobile computing device 410 and another computing device. For example, the service provider server system 452 may receive an indication of an incoming call to the mobile computing device 410 from the PSTN 462. Conversely, the mobile computing device 410 may send a communication to the service provider server system 452 using a telephone number associated with a device accessible through the PSTN 462 to initiate a telephone call.

[0064] The network 450 can connect the mobile computing device 410 to a Voice over Internet Protocol (VoIP) service 464 that routes voice communications over an IP network rather than the PSTN. For example, a user of the mobile computing device 410 can invoke a VoIP application and initiate a call using the program. The service provider server system 452 can forward the voice data from the call to the VoIP service, and the VoIP service can route the call over the Internet to the corresponding computing device, possibly using the PSTN for the last leg of the connection.

[0065] The app store 466 can provide the user of the mobile computing device 410 with the ability to browse a list of remotely stored applications, which the user can download over the network 450 and install on the mobile computing device 410. The app store 466 can serve as a repository for applications developed by third-party app developers. The applications installed on the mobile computing device 410 can be able to communicate with a server system designated for the application over the network 450. For example, a VoIP application can be downloaded from the app store 466, enabling the user to communicate with the VoIP service 464.

[0066] The mobile computing device 410 can access content on the Internet 468 over the network 450. For example, a user of the mobile computing device 410 can invoke a web browser application that requests data from a remote computing device accessible at a specified uniform resource location. In various examples, some of the services 460 can be accessed over the Internet.

[0067] The mobile computing device 410 can communicate with a personal computer 470. For example, the personal computer 470 can be the home computer of the user of the mobile computing device 410. Thus, the user can be able to stream media from his personal computer 470. The user can also view the file structure of his personal computer 470 and transfer selected documents between the computerized devices.

[0068] The mobile computing device 410 can communicate with a social network 474. The social network can include many members, some of which have agreed to be associated as acquaintances. An application on the mobile computing device 410 can access the social network 474 to retrieve information based on the user's acquaintances of the mobile computing device. For example, an "address book" application can retrieve the phone numbers of the user's acquaintances. In various examples, content can be delivered to the mobile computing device 410 based on the social network distance and connection relationships from the user to other members in the social network graph of the members. For example, advertisements and news article content can be selected for the user based on the interaction levels of members "close" to the user (e.g., members who are "friends" or "friends of friends") with such content.

[0069] The mobile computing device 410 can access a personal contact set 476 via a network 450. Each contact can identify an individual and include information about that individual (e.g., phone number, email address, and birthday). Since the contact set is remotely hosted to the mobile computing device 40, the user can access and maintain the contacts 476 across multiple devices as a common contact set.

[0070] The mobile computing device 410 can access cloud-based applications 478. Cloud computing provides applications (e.g., a word processor or an email program) that are remotely hosted from the mobile computing device 410 and can be accessed by the device 410 using a web browser or a dedicated program.

[0071] A map service 480 can provide street maps, route planning information, and satellite images to the mobile computing device 40. The map service 480 can also receive queries and return location-specific results. For example, the mobile computing device 410 can send an estimated location of the mobile computing device and a query of the user input for "pizza shop" to the map service 480. The map service 480 can return a street map with "markers" superimposed on the map, where the "markers" identify the geographical locations of nearby "pizza shops".

[0072] A turn-by-turn service 482 can provide turn-by-turn navigation guidance to a destination provided by the user to the mobile computing device 410. For example, the turn-by-turn service 482 can stream a street-level view of the estimated location of the device along with audio commands and superimposed arrows for guiding the user of the device 410 to the destination to the device 410.

[0073] The mobile computing device 410 can request various forms of streaming media 484. For example, the computing device 410 can request a stream for a pre-recorded video file, a live television program, or a live radio program.

[0074] The Weibo service 486 can receive a post from the mobile computing device 410 that does not identify the recipient of the post. The Weibo service 486 can disseminate the post to other members of the Weibo service 486 who have agreed to subscribe to the user.

[0075] The search engine 488 can receive a text or verbal query entered by a user from the mobile computing device 410, determine a set of Internet-accessible documents responsive to the query, and provide information to the device 410 to display a list of search results for the responsive documents. In an example of receiving a verbal query, the speech recognition service 472 can translate the received audio into a text query that is sent to the search engine.

[0076] These and other services can be implemented in the server system 852. The server system can be a combination of hardware and software that provides a service or a collection of services. For example, a collection of physically separated and networked computerized devices can operate together as a logical server system unit to handle the operations necessary to provide services to hundreds of computing devices. The server system is also referred to herein as a computing system.

[0077] In various embodiments, an operation performed "in response to" or "due to" another operation (e.g., a determination or an identification) is not performed if the previous operation was unsuccessful (e.g., if the determination was not performed). An operation that is performed "automatically" is an operation that is performed without user intervention (e.g., intervening user input). Features in this document described using conditional language can describe alternative embodiments. In some examples, "transmitting" from a first device to a second device includes the first device placing data on the network for receipt by the second device, but may not include the second device receiving the data. Conversely, "receiving" from a first device can include receiving data from the network, but may not include the first device sending the data.

[0078] "Determining" by a computing system can include a computing system requesting another device to perform the determination and providing the result to the computing system. Additionally, "displaying" or "presenting" by a computing system can include the computing system sending data for causing another device to display or present the referenced information.

[0079] Embodiments of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs encoded on a computer storage medium for running by, or to control the operation of, a data processing apparatus, i.e., one or more modules of computer program instructions. Optionally or additionally, the program instructions can be encoded on an artificially generated propagated signal that is generated to encode information for transmission to a suitable receiver apparatus for running by the data processing apparatus, e.g., a machine-generated electrical, optical, or electromagnetic signal. A computer storage medium can be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them, or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, although a computer storage medium is not a propagated signal, a computer storage medium can be the source or destination of computer program instructions encoded in an artificially generated propagated signal. A computer storage medium can also be one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices) or be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0080] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0081] The term “data processing apparatus” encompasses all types of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, a system on a chip, or multiple or combinations of the foregoing. The apparatus can include dedicated logic circuitry, e.g., an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit). In addition to hardware, the apparatus can also include code that creates an execution environment for the computer programs being discussed, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and the execution environment can implement various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0082] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages, declarative or procedural languages), and can be deployed in any form (including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for a computing environment). A computer program may or may not correspond to a file in a file system. The program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to run on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0083] The processes and logical flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logical flows can also be performed by special-purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and the apparatus can also be implemented as special-purpose logic circuitry.

[0084] Processors suitable for running a computer program include, for example, both general and special purpose microprocessors, as well as any one or more processors of any type of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The basic elements of a computer are a processor for performing actions in accordance with the instructions and one or more memory devices for storing the instructions and data. Generally, a computer will also include one or more mass storage devices for storing data (e.g., magnetic disks, magneto-optical disks, or optical disks) or be operatively coupled to receive data from or transfer data to one or more mass storage devices for storing data or both. However, a computer need not have such devices. In addition, a computer may be embedded in another device, for example, a mobile phone, a Personal Digital Assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a Universal Serial Bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0085] To provide for interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) monitor) for providing information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user may provide input to the computer. Other types of devices may also be used to provide for interaction with the user; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input received from the user may be in any form, including acoustic, speech, or tactile input. Additionally, a computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page in response to a request received from a web browser to a web browser on a user's user device.

[0086] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component (e.g., as a data server), or a middleware component (e.g., an application server), or a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with the embodiments described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), the Internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network).

[0087] A computing system can include users and servers. Users and servers are typically remote from each other and generally interact through a communication network. The relationship between users and servers arises by virtue of computer programs that run on respective computers and have a user-server relationship with each other. In some embodiments, a server transmits data (e.g., an HTML page) to a user device (e.g., for the purpose of displaying data to a user interacting with the user device and receiving user input from the user interacting with the user device). Data generated at the user device (e.g., the result of a user interaction) can be received from the user device at the server.

[0088] Although this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, in some cases one or more features from a claimed combination can be deleted from the claimed combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination.

[0089] Similarly, although the operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order, or that all of the illustrated operations be performed, to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system components in the foregoing embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems generally may be integrated together in a single software product or packaged into multiple software products.

[0090] Accordingly, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the acts recited in the claims can be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the figures need not be in the particular order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method implemented in a user device, the method comprising: Displaying, on a display of the user device, a specific environment instance of a native application capable of displaying multiple different environment instances, wherein the native application is an application operable to generate environment instances for display within the native application on the user device and operates independently of a browser application operable on the user device; Displaying, in the foreground on the display and in an environment separate from the native application, a context information card, the context information card including a share command and at least one or more additional commands that are not share commands; and In response to a selection of the share command during the display of the context information card: Determining a recipient address specifying a recipient device separate from the user device; Generating an image of the specific environment instance of the native application; Selecting a uniform resource identifier of the specific environment instance of the native application, wherein the uniform resource identifier is selected from a local user device cache of recently used uniform resource identifiers; Generating a message including the image of the specific environment instance and the uniform resource identifier; and Sending the message including the image of the specific environment instance and the uniform resource identifier to the recipient address.

2. The computer-implemented method according to claim 1, wherein, Generating a message including the image of the specific environment instance and the uniform resource identifier includes embedding the uniform resource identifier in image metadata of the image, wherein, at a user device receiving the message, in response to a navigation command on the image, the user device will instantiate the native application at the specific environment instance addressed by the uniform resource identifier.

3. The computer-implemented method according to claim 1, wherein, Generating a message including the image of the specific environment instance and the uniform resource identifier includes generating the message including the uniform resource identifier and the image as separate display items, and wherein, at a user device receiving the message, in response to a navigation command on the uniform resource identifier, the user device will instantiate the native application at the specific environment instance addressed by the uniform resource identifier.

4. A user device, comprising: Data processing means; A display device; And A non-transitory computer-readable storage medium that communicates data with the data processing means and stores instructions executable by the data processing means, and when the instructions are run, causes the data processing means to perform operations, the operations including: Displaying, on a display of the user device, a specific environment instance of a native application capable of displaying multiple different environment instances, wherein the native application is an application operable to generate environment instances for display within the native application on the user device and operates independently of a browser application operable on the user device; Displaying, in the foreground on the display and in an environment separate from the native application, a context information card, the context information card including a share command and at least one or more additional commands that are not share commands; and In response to a selection of the share command during the display of the context information card: Determine a recipient address that specifies a recipient device separate from the user device; Generate an image of a particular environment instance of the native application; Select a uniform resource identifier of the particular environment instance of the native application, wherein the uniform resource identifier is selected from a local user device cache of recently used uniform resource identifiers; Generate a message that includes the image of the particular environment instance and the uniform resource identifier; and Send a message that includes the image of the particular environment instance and the uniform resource identifier to the recipient address.

5. The user equipment according to claim 4, wherein, Generating a message that includes the image of the particular environment instance and the uniform resource identifier includes embedding the uniform resource identifier in the image metadata of the image, wherein at a user device that receives the message, in response to a navigation command on the image, the user device will instantiate the native application at the particular environment instance addressed by the uniform resource identifier.

6. The user equipment according to claim 4, wherein, Generating a message that includes the image of the particular environment instance and the uniform resource identifier includes generating the message that includes the uniform resource identifier and the image as separate display items, and wherein at a user device that receives the message, in response to a navigation command on the uniform resource identifier, the user device will instantiate the native application at the particular environment instance addressed by the uniform resource identifier.

7. A non-transitory computer-readable storage medium storing instructions executable by a user device, and when the instructions are run, causing the user device to perform operations, the operations including: Display, on a display of the user device, a particular environment instance of a native application that is capable of displaying multiple different environment instances, wherein the native application is an application operable to generate environment instances for display within the native application on the user device and operates independently of a browser application operable on the user device; Display, in a foreground on the display and in an environment separate from the native application, a context information card that includes a share command and at least one or more additional commands that are not share commands; and In response to a selection of the share command during the display of the context information card: Determine a recipient address that specifies a recipient device separate from the user device; Generate an image of the particular environment instance of the native application; Select a uniform resource identifier of the particular environment instance of the native application, wherein the uniform resource identifier is selected from a local user device cache of recently used uniform resource identifiers; Generate a message that includes the image of the particular environment instance and the uniform resource identifier; and Send a message that includes the image of the particular environment instance and the uniform resource identifier to the recipient address.

8. The non-transitory computer-readable storage medium according to claim 7, wherein, Generating a message that includes an image of the specific environment instance and the uniform resource identifier includes embedding the uniform resource identifier in the image metadata of the image, wherein, at a user device that receives the message, in response to a navigation command on the image, the user device will instantiate the native application at the specific environment instance addressed by the uniform resource identifier.

9. The non-transitory computer-readable storage medium according to claim 7, wherein, Generating a message that includes an image of the specific environment instance and the uniform resource identifier includes generating the message that includes the uniform resource identifier and the image as separate displayable items, and wherein, at a user device that receives the message, in response to a navigation command on the uniform resource identifier, the user device will instantiate the native application at the specific environment instance addressed by the uniform resource identifier.

10. A computer-implemented method implemented in a user device, the method comprising: Displaying, on a display of the user device, active content of a specific environment instance of a native application that is capable of displaying multiple different environment instances, wherein the native application is an application operable to generate environment instances for display within the native application on the user device, Receiving, at the user device, a single input operating system command when the active content of the specific environment instance is the active focus of the display, the single input operating system command being a command invoked independent of the native application; In response to the single input operating system command: Generating a query-independent request for context information related to the active content; Sending the query-independent request to a context information server; and Receiving context information from the context information server, wherein the context information is in response to the query-independent request and includes at least one of text content, audio content, and video content; Displaying, on the display, a context information card that includes a share command and the context information; and In response to a selection of the share command during the display of the context information card: Determining a recipient address that specifies a recipient device separate from the user device; Selecting a uniform resource identifier of the specific environment instance of the native application; Generating a message that includes an image of the active content of the specific environment instance and the uniform resource identifier; and Sending the message that includes the image of the active content of the specific environment instance and the uniform resource identifier to the recipient address.

11. The computer-implemented method according to claim 10, wherein, Generating a message that includes an image of the active content of the specific environment instance and the uniform resource identifier includes embedding the uniform resource identifier in the image metadata of the image, wherein, at a user device that receives the message, in response to a navigation command on the image, the user device will instantiate the native application at the specific environment instance addressed by the uniform resource identifier.

12. The computer-implemented method according to claim 10, wherein, Generating a message that includes an image of the active content of the specific environment instance and the uniform resource identifier includes generating the message that includes the uniform resource identifier and the image as separate displayable items, and wherein, at a user device that receives the message, in response to a navigation command on the uniform resource identifier, the user device instantiates the native application at the specific environment instance addressed by the uniform resource identifier.

13. The computer-implemented method according to claim 10, wherein, Selecting a uniform resource identifier for a specific environment instance of the native application includes selecting the uniform resource identifier from a user device cache of recently used uniform resource identifiers.

14. The computer-implemented method according to claim 10, wherein, Selecting a uniform resource identifier for a specific environment instance of the native application includes the operating system of the user device requesting the uniform resource identifier for the specific environment instance from the native application.

15. A user device, comprising: A non-transitory computer-readable storage medium storing instructions; A display; And A data processing device that runs the instructions to: Display on the display active content of a specific environment instance of a native application that is capable of displaying multiple different environment instances, wherein the native application is an application operable to generate environment instances for display within the native application on the user device, and when the active content of the specific environment instance is the active focus of the display, receive, at the user device, a single input operating system command that is a command invoked independent of the native application; In response to the single input operating system command: Generate a query-independent request for context information related to the active content; Send the query-independent request to a context information server; and Receive context information from the context information server, wherein the context information is in response to the query-independent request and includes at least one of text content, audio content, and video content; Display on the display a context information card that includes a share command and the context information; and In response to a selection of the share command during the display of the context information card: Determine a recipient address specifying a recipient device separate from the user device; Select a uniform resource identifier for a specific environment instance of the native application; Generate a message that includes the uniform resource identifier for the specific environment instance of the native application and the context information; and Send to the recipient address a message that includes the uniform resource identifier for the specific environment instance and the context information.

16. The user equipment according to claim 15, wherein, In response to a selection of the uniform resource identifier of the message by a recipient user of the recipient device, the recipient device instantiates the native application at the specific environment instance addressed by the uniform resource identifier at the recipient device.

17. The user equipment according to claim 16, wherein, The uniform resource identifier for the specific environment instance of the native application included in the message includes query parameters processed by the native application installed on the recipient device to reach the specific environment instance.

18. The user equipment according to claim 15, wherein, Selecting a uniform resource identifier for a specific environment instance of the native application includes selecting the uniform resource identifier from a user device cache of recently used uniform resource identifiers.

19. The user equipment according to claim 15, wherein, Selecting a uniform resource identifier for a specific environment instance of the native application includes the operating system of the user device requesting the uniform resource identifier for the specific environment instance from the native application.

Images