In-application history search

Abstract

The invention relates to in-application history search. An apparatus of a method and apparatus for performing a search using a plurality of application states is described. In one exemplary embodiment, the device receives a plurality of application states from a plurality of applications running on the device. The device also creates an index of the plurality of application states. In addition, the device receives a query to search for data stored on the device. Further, the device searches for a plurality of application states using the index and the query. The device additionally determines a match to the query for one of the plurality of application states and returns a match to the matched application state.

Description

[0001] This application is a divisional application of the invention patent application with application number 201680031614.3 and name “Historical Search within Application” filed on April 20, 2016. Technical Field

[0002] The present invention generally relates to search technology and, more particularly, to searching application history on a device and across multiple devices and generating application views for use in a search index. Background Art

[0003] Users on a device often perform query searches to find information on the Web or from some other data source, such as objects stored locally on the device. A query search begins by receiving a query string from a client, which is sent to a search service. The search service receives the query string and searches one or more search indexes for results that match the query string. The indexes can reference web pages or other objects available on the network, or can include information about objects locally located on the device (e.g., files, media, contacts, and / or other types of objects stored locally on the device). In addition, many queries performed by users result in web history results. For example, a query entered in a web browser results in results from the web history 60% of the time.

[0004] Therefore, being able to search web browser history can be useful. In addition, searching the history of other applications on the device would be useful. This is because the user will have a routine for using the device. For portable devices, this routine may include using applications, particularly non-web browser applications. For example, the average smartphone user spends 86% of their time using non-web browser applications. However, being able to search the history of non-web browser applications can be difficult because the data for application usage history is difficult to access (if it can be accessed at all) and may be in a proprietary format. Therefore, searching application history is difficult. Summary of the Invention

[0005] A method and apparatus for performing a search using multiple application states is described. In one exemplary embodiment, a device receives multiple application states from multiple applications running on the device. The device also creates an index of the multiple application states. Additionally, the device receives a query to search data stored on the device. Furthermore, the device searches the multiple application states using the index and the query. The device also determines a match for the query against one of the multiple application states and returns a match for the matching application state.

[0006] In another embodiment, the device performs a query using a plurality of application states on the device. In this embodiment, the device performs the query using an index stored on the device. The device also receives a plurality of results that match the query. Additionally, the device determines a subset of the plurality of results corresponding to application states that correspond to native applications installed on the device. Furthermore, the device presents, for each result in the subset of the plurality of results, the result and a representation of the native application that corresponds to the result.

[0007] In yet another embodiment, a device selects an application state for use in a multi-device search. In this embodiment, the device detects on the device that the application state has been selected as a query result for a device-level search on the device. The device further sends the application state to a server, where the application state is indexed along with other application states from other devices.

[0008] In another embodiment, the device performs a search on the first device using the application state received from the second device. In this embodiment, the device receives multiple application states from multiple applications running on multiple devices. The device also creates an index of the multiple application states. The device also receives a query to search data stored on the device. In addition, the device searches the multiple application states using the index and the search query and returns matches for matching application states.

[0009] In yet another embodiment, a device performs a search. In this embodiment, the device sends a query to a server and receives a plurality of results matching the query. The device further determines a subset of the plurality of results, the subset comprising an application state generated on another device corresponding to a native application installed on the device. Additionally, the device presents a link and a representation of the native application for each result in the subset of the plurality of results.

[0010] In another embodiment, the device indexes the application state into the search query index. In this embodiment, the device receives the application state of the application from another device coupled to the server. The device also generates an application view corresponding to the application state, wherein the view is a representation of a user interface of the application corresponding to the application state. Additionally, the device indexes the view into the search query index.

[0011] In yet another embodiment, the device retrieves an application state having a view associated with a query result. In this embodiment, the device sends a query to a server. The device further receives a query result from the server, the result including a view of the application state of an application corresponding to the result, where the view is a representation of a user interface of the application corresponding to the application state. The device also presents the result and an indication of the view.

[0012] Other methods and apparatus are also described. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0014] Figure 1 is a block diagram of one embodiment of a system for indexing application state for use in a local device search index.

[0015] Figure 2 is a block diagram of one embodiment of a system for searching application state using an on-device application state search index.

[0016] Figure 3 is a block diagram of an embodiment of a user interface that displays application state query results among other query results.

[0017] Figure 4A is a flow diagram of one embodiment of a process for indexing application state received from multiple different applications on a device.

[0018] Figure 4B is a flow diagram of one embodiment of a process for determining query results for a query using an application state index.

[0019] Figure 5 is a flow diagram of one embodiment of a process for receiving and presenting application state as part of query results.

[0020] Figure 6 is a block diagram of one embodiment of a system for indexing application state for use in a remote search index.

[0021] Figure 7 is a block diagram of one embodiment of a system for searching application state using a remote application state search index.

[0022] Figure 8 is a flow diagram of one embodiment of a process for adding application state to an application state index.

[0023] Figure 9is a flow diagram of one embodiment of a process for exporting application state to an application state indexing service.

[0024] Figure 10 is a flow diagram of one embodiment of a process for performing a query search using an application state index.

[0025] Figure 11 is a flow diagram of one embodiment of a process for receiving and presenting application state as part of query results.

[0026] Figure 12 is a block diagram of one embodiment of a system for indexing application state views for use in remote search indexing.

[0027] Figure 13 is a block diagram of one implementation of an application view.

[0028] Figure 14 is a flow diagram of one embodiment of a process for using application state to generate an application state view.

[0029] Figure 15 is a flow diagram of one embodiment of a process for receiving and presenting application state, including an application state view, as part of query results.

[0030] Figure 16 is a block diagram of one embodiment of an application state indexing module that indexes application state received from multiple different applications on a device.

[0031] Figure 17 is a block diagram of one embodiment of a results module that uses an application state index to determine query results for a query.

[0032] Figure 18 is a block diagram of one embodiment of a presentation module that receives and presents application state as part of query results.

[0033] Figure 19 is a block diagram of one embodiment of an application state module that adds application state to an application state index.

[0034] Figure 20 is a block diagram of one embodiment of an application state export module that exports application state to an application state indexing service.

[0035] Figure 21 is a block diagram of one embodiment of a query module that performs query searches using an application state index.

[0036] Figure 22is a block diagram of one embodiment of another presentation module that receives and presents application state as part of query results.

[0037] Figure 23 is a block diagram of one embodiment of an application state view module that uses application state to generate an application state view.

[0038] Figure 24 is a block diagram of one embodiment of an application state view rendering module that receives and renders application state including an application state view as part of query results.

[0039] Figure 25 An example of a typical computer system that may be used with the embodiments described herein is shown.

[0040] Figure 26 An example of a data processing system that may be used with an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0041] Methods and apparatus for performing searches using multiple application states are described. In the following description, numerous specific details are provided to provide a thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known components, structures, and techniques have not been shown in detail in order to avoid obscuring the description.

[0042] The phrase "one embodiment" mentioned in this specification means that a particular feature, structure or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment.

[0043] In the following description and claims, the terms "coupled" and "connected" and their derivatives may be used. It should be understood that these terms are not intended to be synonymous with each other. "Coupled" is used to indicate that two or more elements that may or may not be in direct physical or electrical contact with each other cooperate or interact with each other. "Connected" is used to indicate the establishment of communication between two or more elements that are coupled to each other.

[0044] The processes illustrated in the following figures are performed by processing logic components that may include hardware (e.g., circuitry, dedicated logic components, etc.), software (such as software running on a general-purpose computer system or a dedicated machine), or a combination of both. Although the processes are described below as operating in certain sequential order, it should be understood that certain operations described may be performed in a different order. Furthermore, certain operations may be performed in parallel rather than sequentially.

[0045] The terms "server," "client," and "device" are intended to refer generally to data processing systems and not specifically to a particular form factor of a server, client, and / or device.

[0046] An apparatus and method for performing a search using multiple application states is described. As described above, being able to search web browser history is useful because users have a digital routine of using a web browser. This digital routine may further include repeatedly accessing the same applications and using those applications for the same types of operations. As described above, smartphone users spend an average of 86% of their time using non-web browser applications. However, being able to search the history of non-web browser applications can be difficult because the data for application usage history is difficult to access (if at all) and is in a proprietary format. Therefore, it is difficult to search application history.

[0047] In one embodiment, a device generates and stores application states for executing applications. The device further indexes these application states so that a local search service running on the device can search the indexed application states for query results. In this embodiment, an application state is a snapshot in time of an application. Application states are similar to web browser states stored in a web browser's history. In one embodiment, application states are for non-web browser applications. In one embodiment, an application's application state may include a title, a view, the date displayed in the view, associated metadata, and / or other state information for the state. For example, and in one embodiment, the application may be a review application that displays reviews of different businesses and services in a geographic area. In this example, each application state may be a set of reviews and related information for a particular business or service (e.g., the name, address, contact information, hours of operation, a description of the business or service, a set of reviews submitted by visitors or users of the service or business, and / or any other type of information related to the business or service). Each application state may be displayed on a single user interface page or on multiple user pages, where the content of each page is organized for display. In one embodiment, each executing application exports one or more application states, which the device indexes into an application state index.

[0048] By indexing application states, users can search the application's history. This allows users to search and find previous application states. Using the found application state, the user can launch the corresponding application using this application state, which brings the application to the point in execution when the application exported the application state. Users can use the indexed application state to return the application to a previously used state using a common mechanism across multiple different applications. For example, and in one embodiment, the application state may be a page in a transportation application for a specific route in a transportation system. In this example, the user can navigate to a specific route in the transportation application, such as local bus route 7. By navigating to this specific route, the transportation application exports the application state for the local bus route page to the application state index. Given that the application state is indexed, the user can retrieve the application state via a query. For example, and in one embodiment, the user can enter "bus route 7" in a query, and the application state for local bus route 7 will appear as a query result. After selecting the application state, the transportation application will load the application state for local bus route 7, and the local bus route 7 page in the transportation application will be displayed to the user. Thus, in this example, the transportation application enters the same state as the previous execution.

[0049] In another embodiment, the device can export application states to a remote application state indexer, which can be used to support queries from devices that did not generate the application states. In this embodiment, the device exports user-occupied application states, where an occupied application state is an application state returned as a query result in response to a user query on the device, and the user has selected the application state. In addition, the device cleans up the application state by removing private information before exporting the application state. If the remote application state indexer has received the application state the required number of times, the remote application state indexer receives the application state and indexes the application state. In this embodiment, by indexing the application state after the required number of times, the application state has been crowdsourced, such that many different users and / or devices have occupied the application state in local searches. In one embodiment, requiring a certain number of occupations of the application state increases the likelihood that the application state will be useful to other users. After indexing, the remote search service can search the remote application state index to determine whether there is an application state that matches the query. For each match, the remote search service returns the matching application state to the client. On the client, the user can select the application state, in which case the corresponding application is launched and brought to the point in the application's execution when the application exported the application state.

[0050] In yet another embodiment, the device generates application state views for different application states. In this embodiment, the application state view is a representation of the user interface of the application corresponding to the application state. For example, and in one embodiment, a review application that has access to the content of thousands or millions of reviews of a business or service can have a view for each of these thousands or millions of reviews. These views can be used to preview the application state and generally the application. In one embodiment, these application state views can be used to preview the application state returned in a set of results for a query, or can be used to preview the application generally. In one embodiment, collecting multiple application state views for an application can be used to preview the application in an application store. For example, and in one embodiment, a review application can have dozens of application state views available for the application.

[0051] Figure 1 is a block diagram of one embodiment of a system for indexing application state for use in a local device search index. Figure 1, device 100 includes a plurality of applications 102 coupled to an application state indexer 104. In one embodiment, device 100 is any type of device that can transmit network data to another device (e.g., a personal computer, a laptop, a server, a mobile device (e.g., a phone, a smartphone, a smartwatch, a personal gaming device, etc.), another network element, etc.). In one embodiment, device 100 can be a virtual machine or can be a device that hosts one or more virtual machines. In one embodiment, device 100 further includes an application state search index 108. In one embodiment, each application 102 is an executing program that goes through a series of states while the application is running. For example and in one embodiment, application 102 can be a word processing application, a spreadsheet, contacts, mail, a phone, a web browser, a media player, a review application, a classified ads application, a social network, productivity, utilities, games, real estate, photos, videos, e-commerce, a storefront, coupons, an operating system, and / or any other type of application that can run on the device.

[0052] As described above, each application 102 goes through a series of states while the application is executing. In one embodiment, one of these application states is a snapshot in time of the application. In one embodiment, the application state of an application 102 may include a title, a user interface state, data displayed in the user interface, associated metadata, and / or other state information for the state. In another embodiment, the application state includes information describing how the state should be presented in search results. For example, and in one embodiment, the application 102 may be a review application that displays reviews of different businesses and services in a certain geographic area. In this example, each application state may be a set of reviews and related information for a certain business or service (e.g., name, address, contact information, business hours, description of the business or service, a set of reviews submitted by visitors or users of the service or business, and / or any other type of information related to the business or service). In one embodiment, the application state title is a title given to the application state, such as, in the case of a review application, the name of the business or service. The application state view is a representation of the user interface of the application 102 that may correspond to the application state. In this embodiment, the user interface state may include a representation of the user interface, to which position the user interface is scrolled or which component of the user interface is activated, and / or what mode the application may be in (e.g., application 102 may have different modes for presenting information to the user). In yet another embodiment, the application may be small enough to include a title plus a uniform resource locator or application identifier and version number of the application that is compatible with the state.

[0053] In one embodiment, each application state includes a title, searchable data and / or metadata, and application-specific opaque data. In this embodiment, the searchable data and / or metadata is data designated by the application 102 as accessible by the search indexing service and / or the query search service, wherein the searchable data and / or metadata can be used to index the application state and can also be used to return the application state as a query result. For example, and in one embodiment, the searchable data and / or metadata can be the content in the application state (e.g., the application state title, the content displayed in the user interface state, media data, location data, time data, or other types of data or metadata that can be used for search indexing). In one embodiment, the application-specific opaque data is application-specific data used to return the application to its previous state and may or may not be searchable data. In this embodiment, the application state is loaded by the corresponding application 102 to return the application to the application state. For example, and in one embodiment, the application-specific opaque data can include user interface state, user interface mode, and / or references to resources. The user interface mode can be the type of mode currently being used by the user interface. For example, and in one embodiment, a word processing program may be in draft layout view or print layout view; and an image editing program may be in library mode, image editing mode, or print mode. In one embodiment, the referenced resource may be the file being viewed or edited, or a uniform resource locator for a resource on the device or another device (such as a server on a network). In one embodiment, data that is part of the application state may be in a dictionary having (key, value) pairs.

[0054] In one embodiment, one or more applications 102 each export one or more application states to an application state indexer 104. In this embodiment, the applications 102 may each export the application state on a fixed or variable schedule. For example and in one embodiment, the applications 102 may export the application state on a fixed time, export the application state for each new user interface state, export the application state after one or more interactions with a user, or according to some other metric. As another example and in another embodiment, a review application may navigate to a new review or review search. In this example, by navigating to the new review or review search, a new view is generated, and a new application state is created and exported to the application state indexer 104. The application state indexer receives the application state and adds the application state to the application index 108. By adding the application state to the index, the new application state is available to a local search service to match queries received by the local search service. In another embodiment, the application state may be exported to a remote search application state index 108, which is described below. Figures 6 to 11 is described in .

[0055] Figure 2 is a block diagram of one embodiment of a system for searching application state using an on-device application state search index. Figure 2 In the embodiment, the device 200 includes an application 204 coupled to a local search service 208. The local search service 208 is further coupled to an application state search index 212 and a local search index 214. In one embodiment, the device 200 is such that Figure 1 The device shown. In one embodiment, the application 204 includes a search input field 206. In this embodiment, the search input field is used to enter a query that can be used by the local search service to perform a search using the query. If a query is entered into the search input 206, the application 204 sends the query to the local search service 208. The local search service 208 receives the query and generates ranked results by searching the local search index 214 and / or the application state search index 212 to determine a set of results for the query. In addition, the local search service 208 ranks the results and sends them back to the application 204.

[0056] In this embodiment, the search may include searching objects stored on device 200. For example, and in one embodiment, the objects may be documents, pictures, music, applications, emails, calendar entries, and / or other objects stored in a local search index. In one embodiment, the search is based on an index maintained by a search module. In this embodiment, the index is an index of metadata stored in objects on the device. In an alternative embodiment, local search service 208 may also apply a query to application state search index 212. In this embodiment, local search service 208 applies the query to application state search index 212 to determine whether there are any application states that match the query. For example, and in one embodiment, local search service 208 applies the query to the searchable data for each application state stored in index 212. In this example, if there are matches for the query for one or more application states in index 212, local search service 208 returns a set of results to application 204 that include these one or more application states. Application 204 displays the ranked results. If one of the ranked results displayed is an application state, the application may display the application's icon, application state title, and application state summary. In one embodiment, upon selecting a displayed application state, an application corresponding to the application state is loaded with the application state. In this embodiment, by loading the application state for an application, the application is loaded into an execution state corresponding to the application state. For example, in one embodiment, if the application state is a specific coupon for a coupon application (e.g., "50% off weekend car rentals!"), the coupon application is loaded with the application state, and the application state displays the specific coupon as if the user had navigated to the coupon.

[0057] Figure 3 is a block diagram of an embodiment of a user interface for displaying application status query results among other query results. Figure 3, three different possible user interfaces 300A to 300C for displaying application states on a device are shown. In one embodiment, user interface 300A includes a search input 302, an application state display 314A, other actions 310A, and an on-screen keyboard 312A. In one embodiment, search input 302A is used by a user of the device to enter a query. In this embodiment, a partial or entire query can be entered and sent to a local search service to determine one or more sets of query results. In one embodiment, the results of the query are returned when one or more characters of the search are entered. In addition, application state display 314A includes an application icon 304A, an application state title 306A, and an application state summary 308A. In one embodiment, application icon 304A is an icon representing the application corresponding to the application state. In this embodiment, application icon 304A can be a portion of the application state returned from the query or retrieved based on information stored in the application state. In one embodiment, application state title 306A is the title of the application state stored in the application state. In addition, application state summary 308A is a summary of the application state. For example and in one embodiment, application state summary 308A includes a description of the application state, such as a description of the content of the application state. In this example, application state summary 308A can indicate to the user the content associated with the application state.

[0058] In one embodiment, in addition to displaying the query results including the application state 314A, the user interface 300A may also include other actions 310A. For example, in one embodiment, the other actions 310A may include links for searching the web using the query or searching an online encyclopedia using the query. The user interface 300A may also include an on-screen keyboard 312A for the user to enter a search query. Alternatively, the query may be entered via other means (e.g., via a microphone coupled to the device, through another device coupled to the device, such as a smart watch coupled to a portable device). In one embodiment, the icon 304A may be an image thumbnail specific to the application state provided by the application. Additionally, the icon 304A may also be a video or video preview. In yet another embodiment, the application state summary may include an "action" button, such as a phone call icon, play, directions, buy.

[0059] In one embodiment, there are many different types of application states that can be displayed as query results. For example, in one embodiment, an application state can be a transportation application's view of a specific route in a transportation system. In this example, a user can navigate to a specific route in the transportation application, such as local bus line 7. By navigating to this specific route, the transportation application will export the application state of the local bus line view to the application state index. Given that the application state is indexed, the user can retrieve the application state via a query. For example, and in one embodiment, a user can enter "bus line 7" in a query, and the application state of local bus line 7 will appear as a query result. Upon selecting the application state, the transportation application will load the application state of local bus line 7, and the user interface of local bus line 7 in the transportation application will be displayed to the user. Thus, in this example, the transportation application enters the same state as the previously viewed state.

[0060] As another example and in another embodiment, a user may be using a food ordering application, and the user simply wants to reorder one of their previous orders. In this example, the user can use the local restaurant's dedicated application to order Vietnamese pho soup from the local restaurant. In this order, the local restaurant application will export an application state corresponding to the order for Vietnamese pho soup. This application state will be indexed and accessible by the local search service. The user can later enter the query "Vietnamese pho soup," "Vietnamese restaurants," or the name of the local restaurant, and the application state corresponding to the order may be one of the results. The application state may also be a preference for the result. After selecting the application state, the local restaurant application will be launched and display previous pho soup orders that the user can use to complete the order for pho soup.

[0061] In yet another example and embodiment, a user may maintain a picture board to plan their next wilderness trip. In this example, the user uses a picture board application to link pictures and comments about the next trip. The user will use the picture board application to return to this specific picture board to add a link from the device clipboard. The picture board application will export this application state of the image board for the wilderness trip in this application state to be provided by the local search service. By searching for this application state, such as the name of a travel location, the user can quickly navigate to the wilderness trip picture board within the picture board application via a query rather than launching the picture board application and navigating to the specific picture board view.

[0062] In one embodiment, saving application state can be used to quickly access a specific view of a utility that may be difficult to navigate to. For example, and in one embodiment, a device settings application can have multiple options that are many levels deep. In this example, a user can go to the battery usage page in the settings application to see which application is consuming the most battery power. The battery usage page can be four or more levels deep and difficult to access. By exporting the application state to implement a better usage page for the settings application, the user can enter the query "battery usage," "battery power," "battery," or some other prefix of the word "battery usage" to have the application state for the battery usage page of the settings application displayed as a result of the query. This will provide quick access to pages in the settings application that may be difficult to navigate to.

[0063] In another embodiment, the application status results for a query can be displayed along with other query results from other domains, such as the one above. Figure 2 The local search index described in . Figure 3 , user interface 300B displays application state 314B and other query results 310B. In the user interface 300B, a search input 302B is displayed together with the application state 314B, the other query results 310B, and the on-screen keyboard 312B. In one embodiment, the search input 302B and the on-screen keyboard 312C are the same as those described above for user interface 300A. In addition, the application state 314B includes an application icon 304B, an application state title 306B, and an application state summary 308B, which are the same as the application icon, application state title, and application state summary described above for user interface 300A. In addition, user interface 300B includes other query results 310B, which may be other query results from other domains or application states. For example and in one embodiment, the other query results 310B for the query "battery" may include an index of objects in a local search index that matches the word "battery", other application states that match the word "battery", or other query results that match a local or remote search index (e.g., a web search index) that matches the word "battery".

[0064] As described above, application state can also be saved for utility applications running on the device. For example, and in one embodiment, the device's settings applications used to configure the device can also export application state. User interface 300C is an example of a query result that includes application state for a settings application. Figure 3, user interface 300C, search input 302C is displayed along with application state 314B, other actions 310C, and on-screen keyboard 312C. In one embodiment, search input 302C and on-screen keyboard 312C are the same as described above for user interface 300A. Additionally, application state 314C includes application icon 304B, application state title 306C for setting components of the application (e.g., battery usage), and application state summary 308C for setting components of the application (battery usage).

[0065] As described above, to make the application state accessible to the local search service, the application state is added to an index accessible to the local search service. Figure 4A is a flow chart of one embodiment of a process 400 for indexing application states received from a plurality of different applications on a device. In one embodiment, process 400 is performed by an application state indexer, such as described above in Figure 1 The application state indexer 104 described in . Figure 4A , process 400 begins by receiving a plurality of application states from a plurality of applications on a device at block 402. For example, and in one embodiment, process 400 may receive application states from a plurality of applications, such as a word processing application, a spreadsheet, contacts, mail, phone, a web browser, a media player, a review application, a classifieds application, a social network, productivity, utilities, games, real estate, photos, video, e-commerce, a storefront, coupons, an operating system, and / or any other type of application that may be run on the device. In one embodiment, the applications may send application states to processor 400 simultaneously, serially, and / or in combination. At block 404, for each application state received by process 400, process 400 adds the application states to an application state index. In one embodiment, process 400 adds the application states to the application state index by adding an application state identifier, indexable text, application identifier, and / or insertion time to a search index data structure (e.g., an inverted index and completion attempts).

[0066] By adding multiple application states to the application state index, these indexed application states are made available to query searches performed by the local search service. Figure 4B is a flow chart of one embodiment of a process 450 for determining query results for a query using an application state index. In one embodiment, process 450 is performed by a local search service to use an application state index (such as the one above) to determine query results for a query. Figure 2The local search service 208 described in the preceding text is used to determine the query results for the query. Figure 4B In one embodiment, process 450 begins by receiving a request at block 452. In one embodiment, the query is a search string entered by a user in an application and sent to process 450. In one embodiment, the input can be entered through text, spoken words, automatically generated, and / or some other means of entering a query prefix. For example, and in one embodiment, the user can enter the query in a web browser or file browser. At block 454, process 450 uses the local application state index to determine a set of query results for the query. In one embodiment, process 450 uses information in the query to determine matching application states in the local application state index. At block 456, process 450 ranks the set of query results. In one embodiment, the ranking is based on the score of each application state that matches the query. Process 450 returns the ranked set of query results at block 458. In one embodiment, process 450 sends the ranked set of query results back to the application that sent the query to process 450.

[0067] Figure 5 is a flow chart of one embodiment of a process 500 for receiving and presenting application state as part of a query result. In one embodiment, the process 500 is performed by an application to receive and present application state as part of a query result, such as described above in Figure 2 The application described in 204. Figure 5In the example, process 500 begins by sending a query to a local search service at block 502. In one embodiment, the query can be a search string entered by a user in an application and sent to the local search service. In this embodiment, the input can be text, spoken words, automatically generated, received from a coupled device (e.g., a smartwatch coupled to a portable device), and / or some other method for entering a search string. In another embodiment, the query can be suggested by the search system, and the user can select a query from multiple options. Alternatively, the query can be extracted from the context. For example, and in one embodiment, a user is reading a text message and performing a search, and the query is extracted by the data detection system and automatically issued, or suggested to the user. Alternatively, the query can be issued by following a link from another application. At block 504, process 500 receives a set of results, wherein these results include application states. In one embodiment, the set of results is sorted with the highest-ranked results as the first choice. At block 506, process 500 presents the application states in the user interface. In this embodiment, the application state includes an application state title, a summary, and an indication of the icon corresponding to the application for the application state. In one embodiment, process 500 is shown above Figure 3 The application state described. In response to an application being selected, process 500 uses the selected application state to start the corresponding application. For example and in one embodiment, if the application state is a review of a local restaurant in a review application, process 500 starts a review application with the application state of the local restaurant. In this example, the review application will be started so that the view presented to the user is one of the local restaurants in the review application. In one embodiment, if the application is not installed on the device, process 500 can download the application from a remote source such as an application store, a web page or other remote server. In this embodiment, process 500 will install the application and start the application using the selected application state.

[0068] As mentioned above, multiple applications can export application states that are indexed locally on the devices that execute these applications.In one embodiment, these application states can further be exported to a remote application state indexer and be used to support queries from devices that do not generate these application states. Figure 6 is a block diagram of one embodiment of a system 618 for indexing application state for use in remote search indexing. Figure 6In FIG. 6 , a device 600 is coupled to a remote application state indexer 610. In one embodiment, each device 600 includes a plurality of applications 602 executing on the device 600, and each application 602 is coupled to an application state module 604 on the device. In this embodiment, each application 602 exports one or more application states 612 to the application state module 604. In this embodiment, the application state module 604 is described above in Figures 1 to 5 6. In another embodiment, the application states may be sent to a remote application state indexer 610. By sending the application states to the remote application state indexer 610, the application states may be made available to support queries from other devices not shown. Thus, in this embodiment, indexed application states from multiple applications running on multiple devices may be used in query results in response to queries sent by devices that did not generate the application states.

[0069] In one embodiment, each application state that is indexed locally can also be exported to a remote application state indexer 610. In this embodiment, thousands or millions of application states can be generated and sent to a remote application state indexer 610. However, when these many application states are exported and indexed, this can create an application state index that is too large and / or has many useless, spurious entries. In addition, some or all of these exported application states may include private information that is not desired to be included in the indexed application state 614.

[0070] In one embodiment, if the application states are already occupied on the device, the application state module 604 exports the application states to the remote application state indexer 610. In this embodiment, to occupy an application state, the application state module determines whether the application state has been returned as a query result in response to a query by a user on the device and the user has selected the application state. In one embodiment, occupying the application state means that the user has sent a query to a local search service, the local search service has returned the application state in a set of query results, and the user has selected or viewed the application state. In one embodiment, occupying the application state indicates to the application state module 604 that this particular application state may be more important than other application states generated by the device 600. For each occupied application state, the application state module 604 exports the application state to the remote application state indexer 610.

[0071] In yet another embodiment, before exporting the occupied application state to the remote application state indexer 610, the application state module 604 cleans up the application state by removing any possible private information that may be in the application state. In one embodiment, the application state may include private information such as a username, private contact information, location, access times, social security number, bank account number, and / or any other type of private information that may be in the application state. In one embodiment, the application that created the application state may mark certain information as private information stored in the application state. In another embodiment, the device may add private information to the application state. Alternatively, the application state module 604 may know that certain information is private regardless of whether the information is marked as private. In any of these embodiments, the application state module 604 will remove the private information.

[0072] In one embodiment, the remote application state indexer 610 receives application states from multiple devices 600. The remote application state indexer 610 can receive application states from a small number of devices or from as many as thousands or millions of devices. In addition and in one embodiment, the remote application state indexer 610 maintains two sets of application states. One set of application states is indexed application states 614. This is the set of application states that have been indexed and available to the search service. The other set of application states is unindexed application states 616. In one embodiment, once an application state has been occupied a required number of times by one or more devices, the remote application state indexer 610 adds the application state to the indexed set of application states. For example and in one embodiment, if an application state has been occupied 50 times, the application state is added to the indexed set of application states. In alternative embodiments, if the application state has been occupied more or fewer times, the application state can be added to the indexed set of application states. In one embodiment, the required number of times and application states must be occupied before being included in the application index can vary depending on the type of application state. For example and in one embodiment, an application state that includes geo-located information (e.g., an application state for regional coupons) may need to be populated fewer times than an application state that does not have geo-located information.

[0073] In this embodiment, indexing the application state after it has been engaged a necessary number of times increases the likelihood that the application state will be useful to other users. For example, and in one embodiment, many different users on different devices use a local transportation application and generate an application state for local bus line 7. In this example, this is a popular line, so the application state is engaged by users accessing the application state via the local search service. The application state is indexed by remote application state indexer 610 and made available to the remote search service.

[0074] In one embodiment, remote application indexer 610 determines whether the application state has been sent before by computing a hash of the application state. If the hash matches other hashes stored by remote application indexer 610, remote application indexer 610 increments the number of times the remote application indexer has received the application state. If the required number of times has been received, remote application indexer 610 indexes the application state. In the following Figure 9 Indexing application state is further described in .

[0075] Alternatively, device 600 may also send the application state directly to other devices that device 600 trusts. In one embodiment, device 600 sends the application state to other devices that device 600 trusts. For example and in one embodiment, device 600 may be in a circle of trust with other devices. Device 600 may send the application state as if it were generated by device 600, or the device may send the application state that has been occupied and / or cleared by the user as described above. Other devices may use these received application states for query searches by adding these received application states to a locally maintained application state index. Alternatively, each other device may index the application state that it has received the necessary number of times.

[0076] Figure 7 is a block diagram of one embodiment of a system for searching application state using a remote application state search index. Figure 7 , device 702 is coupled to a remote application state search service 714. In one embodiment, device 702 includes an application 704 coupled to a local search service 708. Local search service 708 is further coupled to an application state search index 716. In one embodiment, device 702, application 704, local search service 708, and application state search index 716 are as described above. Figure 2. In another embodiment, local search service 708 can forward the query to remote application search service 714, where remote application search service 714 determines whether there is a set of results for the query. Remote application search service 714 returns the set of results to local search service 708, which in turn returns the set of results to application 704. Alternatively, application 704 can send the query to remote application search service 714, which in turn sends the set of query results back to application 704.

[0077] As described above, remote application state search service 714 receives a query from device 702 and returns a set of query results for the query to device 702. In one embodiment, remote application state search service 714 receives the query, searches indexed application states 712 for application states that match the received query, scores each matching application state, ranks the set of results, and returns the ranked results to the application. Application 704 displays the results including the application state. In one embodiment, application 704 displays an icon for the application, an application state title, and an application state summary, as described above. Figure 3 After selecting a displayed application state, the application is loaded with the application state. In one embodiment, if the application state is stored locally on the device, the application is in the same state as if the user had occupied the application state.

[0078] Figure 8 is a flow chart of one embodiment of a process 800 for adding application state to an application state index. In one embodiment, the process 800 is performed by an application state exporter module to add application state to an application state index, such as the one above. Figure 6 The application state exporter module 606 described in . Figure 8, process 800 begins by receiving an application state at block 802. In one embodiment, the application state is received by process 800 from one or more applications running on the device on which process 800 is executing. At block 802, process 800 determines whether the application state is already occupied. In one embodiment, if the application state is returned in a set of results that match the query and the user has selected the application state to load into the application, then the application state is occupied. If the application state is not occupied, execution continues to block 802 above. If the application state is already occupied, process 800 cleans up the application state at block 806. In one embodiment, process 800 performs a cleanup of the application state as described above in Figure 6 The application state is cleaned by removing private information associated with and / or stored in the application state as described in

[15] . At block 808, process 800 sends the cleaned application state to a remote application state indexing service. In one embodiment, the remote application state indexing service may add the application state to an application state index.

[0079] Figure 9 is a flow chart of one embodiment of a process 900 for indexing application state by an application state indexing service. In one embodiment, process 900 is performed by a remote application state indexer to index application state, such as the one above. Figure 6 The remote application state indexer 610 described in . Figure 6In block 902, process 900 begins by receiving an indication of an application state from a device. In one embodiment, the indication of the application state is a hash of the application state. By receiving the application state hash rather than the complete application state, process 900 does not receive the application state until the application state is shared by multiple clients or has been consumed a requisite number of times. At block 904, process 900 increments the number of occurrences of the application state. In one embodiment, process 900 maintains a counter for the application state hash. If this is the first time process 900 has received the indication, the counter is 1. At block 906, process 900 determines whether the number of occurrences exceeds a threshold. In one embodiment, if process 900 has received the application state the requisite number of times, it means that the application state has been consumed multiple times and is a candidate for indexing and use by shared service queries. For example, and in one embodiment, if the application state for a particular coupon in a coupon application has been consumed 50 times, the application state can be made available for application state indexing. If the number of occurrences exceeds the threshold, process 900 sends a request for the complete application state to the device that sent the last application state indication. Process 900 receives the application state at block 910. Process 900 indexes the application state at block 910. By indexing the application state, process 900 makes the application state available as part of a set of results for a query.

[0080] In another embodiment, rather than requesting the full application state upon the last application state indication received, process 900 begins incrementally building the application state until process 900 receives and indexes the last segment of the application state. For example, and in one embodiment, process 900 requests the last M clients to send 1 / M of the application state to process 900. In this example, since the application state generates the same application state hash, this is the same application state. This means that these M segments of the application state can be joined together by process 900. This embodiment can provide additional privacy because sending portions of the application state each time allows process 900 to build the complete application state.

[0081] Figure 10 is a flow chart of one embodiment of a process 1000 for performing a query search using an application state index. In one embodiment, process 1000 is performed by a remote application state search service to perform a query search using an application state index, such as described above in Figure 7 Remote application state search service 714 as described in . Figure 10In

[10] , process 1000 begins by receiving a query from a client at block 1002. In one embodiment, the query is a search string entered by a user in an application and sent to a remote search service as described above. At block 1004, process 1000 uses the query to search an application state index. In one embodiment, process 1000 determines whether there are any application states that match the query. At block 1006, process 1000 determines a set of results for the query. In one embodiment, this set of results includes one or more application states that match some or all of the text in the query. At block 1008, process 1000 ranks the set of results. In one embodiment, process 1000 ranks the set of results by determining a score for each result and using that score to rank the results. At block 1010, process 1000 combines the set of results with results from other search domains. In one embodiment, if the search is a consolidated search, where the same query is used to search different indices, process 1000 combines the results from the other search domains with the set of results determined using the application state index. For example, and in one embodiment, the query may be used to search an application state index, a general web search index, and / or a different index (e.g., a media index, an application store index, a map index, an online encyclopedia index, and / or another type of index). At block 1012, process 1000 returns a set of ranked results to the client along with the other results generated in block 1010.

[0082] Figure 11 is a flow chart of one embodiment of a process 1100 for receiving and presenting application state as part of a query result. In one embodiment, the process 1100 is performed by an application to receive and present application state as part of a query result, such as described above in Figure 7 The application described in 704. Figure 11 In the embodiment, process 1100 begins by sending a query to a remote search service at block 1102. In one embodiment, the query can be a search string entered by a user in an application and sent to the remote search service. In this embodiment, the input can be input through text, spoken words, automatically generated, received from a coupled device (e.g., a smartwatch coupled to a portable device), and / or some other means of entering a search string. At block 1104, process 1100 receives a set of results, where the results include an application state. In this embodiment, the application state is as described above. Figure 6In one embodiment, the set of results is sorted with the highest ranked results as the first choice. At block 1106, process 1100 presents the application state in a user interface. In this embodiment, the application state includes an application state title, a summary, and an indication of an icon corresponding to the application for the application state. In one embodiment, process 1100 displays the application state as shown above. Figure 3 The application state described. In response to the application being selected, process 1100 uses the selected application state to start the corresponding application. For example and in one embodiment, if the application state is a comment on a local restaurant in a review class application, process 1100 starts a review class application with the application state of the local restaurant. In this example, the review class application will be started so that the view presented to the user is one of the local restaurants in the review class application. In one embodiment, if the application is not installed on the device, process 1100 can download the application from a remote source such as an application store, a web page or other remote server. In this embodiment, process 1100 will install the application and start the application using the selected application state.

[0083] Figure 12 is a block diagram of one embodiment of a system 1200 for indexing application state views for use in remote search indexing. Figure 12 , device 1202 is coupled to application state storage 1206 and application state index 1208. In one embodiment, device 1202 retrieves the application states stored in application state storage 1206, and for each application state, device 1202 generates a view for each of these application states. In one embodiment, a view of an application state is a representation of the user interface of the application corresponding to the application state. For example, in one embodiment, the user interface may include text, images, video, audio, animation, graphics, and / or other types of user interface components. In this example, the corresponding view is a two-dimensional representation of the user interface. In one embodiment, an application may have many different views generated based on different application states associated with the application. For example and in one embodiment, a review application that has access to the content of thousands or millions of reviews of a business or service may have a view for each of these thousands or millions of reviews. In the following Figure 13 Views are further described in .

[0084] In one embodiment, the application state stored in application state storage 1206 may be as described above in Figure 6 , which has been occupied by a user the requisite number of times as explained in . Alternatively, application state storage 1206 may also include application states that have not been indexed. In addition, application state index 1208 includes indexed application states with views generated for these application states. In this embodiment, these views can be returned along with the application states as part of a set of results for a query. Search engine 1210 includes an application state search service 1212 that receives queries from devices 1214. Application state search service 1212 receives queries from devices 1214, uses these queries to search the application state index, determines matching application states for the queries with associated views, scores the matching application states, sorts the matching application states, and returns the matching application states as a set of results to the device that sent the original query.

[0085] As mentioned above, application states can have associated views. Figure 13 is a block diagram of one embodiment of the application state view 1302. Figure 13 In the embodiment, device 1300 has an application executing in a particular application state. The application in the application state displays an application state user interface 1302. Application state user interface 1302 may include various components, such as icons, text, images, video, audio, animation, graphics, and / or other types of user interface components. For example and in one embodiment, application state user interface 1302 includes image 1304, text 1306, and icon 1308. In one embodiment, a view can be generated from application state user interface 1302. In this environment, the view is a representation of application state user interface 1302 that can be saved and indexed in an application state index along with the application state. For example and in one embodiment, the view of application state user interface 1302 is a two-dimensional image, such as a GIF, JPEG, PNG, and / or another type of two-dimensional image. In this example, the two-dimensional image of the view can be stored in the application state index along with the application state.

[0086] Figure 14 is a flow chart of one embodiment of a process 1400 for generating an application state view using application state. In one embodiment, process 1400 is performed by an application state view generator and an indexer to generate an application state view using application state, such as the one above. Figure 12 The application state view generator and indexer 1204 described in . Figure 14In the example, process 1400 begins by receiving an application state at block 1402. In one embodiment, process 1400 receives an application state from an application state storage device (such as, for example, Figure 12 The application state is received by the application state storage device 1206 described in

[0064] . At box 1404, process 1400 uses the application state to generate an application state view. In one embodiment, process 1400 generates the application state view by simulating the application using the application state. In this embodiment, the application is executed in a simulator with the application state. Process 1400 can use the simulator's private framework to capture the application user interface of the application state. Alternatively, process 1400 can load the application onto a virtual platform or the device itself and use a mechanism to generate a view of the application state. At box 1406, process 1400 adds the application state view to the application state index for the corresponding application state.

[0087] By generating these application state views, process 1400 can generate multiple views for one or more applications. These views can be used to preview the application state and generally the application. In one embodiment, these application state views can be used to preview the application state returned in a set of results for a query, or can be used to generally preview the application. In the following Figure 15 Using this view with a query is further described in [ 15 ]. In one embodiment, collecting multiple application state views for an application can be used to preview the application. For example, and in one embodiment, a review application can have dozens of application state views available for the application. For someone interested in the review application, such as someone viewing the application in an app store, these application state views can be provided to the user, allowing the user to preview the application before purchasing and / or downloading it. In this example, the user can browse forward and backward through these dozens of views to get a feel for what the application looks like.

[0088] Figure 15 is a flow chart of one embodiment of a process 1500 for receiving and presenting an application state, including an application state view, as part of a query result. In one embodiment, process 1500 is performed by a device to receive and present an application state view as part of a query result, such as described above in Figure 12 The device described in 1214. Figure 15In the embodiment, process 1500 begins by sending a query to a remote search service at block 1502. In one embodiment, the query can be a search string entered by a user in an application and sent to the remote search service. In this embodiment, the input can be input through text, spoken words, automatically generated, received from a coupled device (e.g., a smartwatch coupled to a portable device), and / or some other means of entering a search string. At block 1504, process 1500 receives a set of results, where the results include an application state. In this embodiment, the application state is as described above. Figure 6 , which has been occupied by the user the necessary number of times. In one embodiment, the order of this set of results is based on the highest-ranked results as the first choice. At box 1506, process 1500 presents the application state in the user interface. In this embodiment, the application state includes an application state title, a summary, an indication of the icon corresponding to the application for the application state, and an indication of the availability of the corresponding application view. In response to the application state view being selected, process 1500 retrieves and presents the application state view. In one embodiment, by displaying the application state view, the user can obtain a preview of the application executing in the application state. This can help the user decide whether to select the application state. In another embodiment, even if the application is installed on the device, the preview view may be faster than launching the application with the application state. For example, and in one embodiment, if the application state view is a review of a local restaurant in a review application, process 1500 retrieves and displays the application state view.

[0089] Figure 16 1 is a block diagram of one embodiment of an application state indexing module 106 that indexes application states received from multiple different applications on a device. In one embodiment, the application state indexing module 106 includes a receive application index module 1602 and an add application state module 1604. In one embodiment, the receive application index module 1602 receives multiple application states, as described above. Figure 4A The add application state module 1604 adds the application state to the index as described in block 402 of FIG. Figure 4A As described in block 404 of .

[0090] Figure 171700 is a block diagram of one embodiment of a result module 1700 that uses an application state index to determine a query result for a query. In one embodiment, the result module 1700 includes a receive query module 1702, a query result module 1704, a sort result module 1706, and a return result module 1708. In one embodiment, the receive query module 1702 receives a query, as described above. Figure 4B The query result module 1704 determines a set of results, as described in block 452 of FIG. Figure 4B The sorting result module 1706 sorts the results as described in block 454 of FIG. Figure 4B The return result module 1708 returns the result as described in the block 456 of FIG. Figure 4B as described in box 458 of .

[0091] Figure 18 is a block diagram of one embodiment of a presentation module 1800 that receives and presents application state as part of a query result. In one embodiment, the presentation module 1800 includes a send query module 1802, a receive result module 1804, a present result module 1806, and a launch application module 1808. In one embodiment, the send query module 1802 sends a query, as described above. Figure 5 The receiving result module 1804 receives the set of results, as described above. Figure 5 The result presentation module 1806 presents the result, as described above. Figure 5 The start application module 1808 starts the application, as described in the box 506 of FIG. Figure 5 As described in box 508 of .

[0092] Figure 19 is a block diagram of one embodiment of an application state module 1900 that adds application state to an application state index. In one embodiment, the application state module 1900 includes a receive application state module 1902, a take-up module 1904, a cleanup module 1906, and a send application state module 1908. In one embodiment, the receive application state module 1902 receives the application state, as described above. Figure 8 The occupation module 1904 determines whether the application state has been occupied, as described in the above block 802. Figure 8 As described in block 804 of FIG. 1906, the cleanup module cleans up the application state, as described above. Figure 8 The Send Application Status module 1908 sends the application as described above. Figure 8 As described in box 808 of .

[0093] Figure 20is a block diagram of one embodiment of an application state export module 2000 that exports application state to an application state indexing service. In one embodiment, the application state export module 2000 includes a receive application state module 2002, an increment module 2004, a comparison module 2006, a send application state request module 2008, a receive application state module 2010, and an index application state module 2012. In one embodiment, the receive application state module 2002 receives an indication of the application state, as described above. Figure 9 As described in block 902 of FIG. 2004, the increment module increases the occurrence of the application state, as described above. Figure 9 The comparison module 2006 compares the number of occurrences, as described above. Figure 9 The Send Application Status Request module 2008 sends the application status request to the device, as described above. Figure 9 The receive application status module 2010 receives the application status, as described above in block 908. Figure 9 The index application state module 2012 indexes the application state as described above in block 910. Figure 9 As described in box 912 of .

[0094] Figure 21 2 is a block diagram of one embodiment of a query module 2100 that performs a query search using an application state index. In one embodiment, the query module 2100 includes a receive query module 2102, a search module 2104, a determine result module 2106, a sort result module 2108, a combine result module 2110, and a return result module 2108. In one embodiment, the receive query module 2102 receives a query, as described above. Figure 10 The search module 2104 searches the index as described above. Figure 10 The query result module 2106 determines a set of results, as described in block 1004 of FIG. Figure 10 The sorting result module 2108 sorts the results as described in block 1006 of FIG. Figure 10 The combined results module 2110 combines results from other search domains, as described above. Figure 10 The return result module 2112 returns the result as described in the block 1010 of FIG. Figure 10 As described in box 1012.

[0095] Figure 22is a block diagram of one embodiment of another presentation module 2200 that receives and presents application state as part of a query result. In one embodiment, the presentation module 2200 includes a send query module 2202, a receive result module 2204, a present result module 2206, and a launch application module 2208. In one embodiment, the send query module 2202 sends a query, as described above. Figure 11 The receiving result module 2204 receives the set of results, as described in the above block 1102. Figure 11 The result presentation module 2206 presents the result, as described in the block 1104 of FIG. Figure 11 The start application module 2208 starts the application as described in the box 1106 of FIG. Figure 11 as described in box 1108 of .

[0096] Figure 23 2 is a block diagram of one embodiment of an application state view module 2300 that uses application state to generate an application state view. In one embodiment, the application state view module 2300 includes a receive application state module 2302, a generate application view module 2304, and an add application view module 2306. The receive application state module 2302 receives the application state, as described above. Figure 14 As described in block 1402 of FIG. 23. The Generate Application View module 2304 generates an application state view, as described above. Figure 14 The add application view module 2306 adds the application view to the index as described in block 1404 of FIG. Figure 14 as described in block 1406 of .

[0097] Figure 24 is a block diagram of one embodiment of an application state view rendering module 2400 that receives and renders application state including an application state view as part of a query result. In one embodiment, the application state view rendering module 2400 includes a send query module 2402, a receive result module 2404, a render result module 2406, and a display application view module 2408. In one embodiment, the send query module 2402 sends a query, as described above. Figure 15 The receiving result module 2404 receives the set of results, as described above. Figure 15 The present application state module 2406 presents the application state, as described above in block 1504. Figure 15 The display application view module 2408 displays the application view, as described in block 1506 of FIG. Figure 15 as described in box 1508 of .

[0098] The embodiments described above have shown that the first device receives a query input, sends a query, receives query results, and launches an application on the device. In an alternative embodiment, the first device (e.g., a smart phone or another type of portable device) is coupled to a second portable device (e.g., a wearable device such as a smart watch, or other types of wearable or portable devices) via a personal area network such as Bluetooth. In these embodiments, the second portable device can perform any of the above-mentioned actions regarding fulfilling the query search. For example, and in one embodiment, the second portable device can receive an input query from the user by various means (e.g., input text, gestures, voice, and / or another type of input), wherein the first device sends the query to the search service. Alternatively, the second device can display the query results, wherein the first device receives the query results and relays these results to the second device. In another embodiment, the display and selection of the results can launch an application on the second device. If the application is not installed on the second device, the application can be downloaded and installed on the second device before the application is launched. For example, and in one embodiment, the results can be displayed on the first or second device, and after selecting the result, the application is launched on the second device. Alternatively, the results can be displayed on the second device and the application is launched on the first device.

[0099] As yet another example and embodiment, a smartwatch is coupled to a smartphone via a Bluetooth connection. In this embodiment, the smartphone or smartwatch receives a query input by the user. If the query is received on the smartwatch, the query is sent to the smartphone. The smartphone sends the query to a search service, which can be local or remote. The smartphone receives search results, which can be displayed on the smartphone, smartwatch, or a combination thereof. Upon selecting one of the results on the smartwatch or smartphone, an application can be launched on the smartphone, smartwatch, or a combination thereof. If the application is not installed on the relevant device, it can be downloaded and installed on the device before launching the application.

[0100] This disclosure recognizes that the use of such personal information data within the present technology can be used to benefit users. For example, this personal information data can be used to deliver targeted content of particular interest to the user. Thus, the use of such personal information data enables planned control over the content delivered. Furthermore, this disclosure contemplates other uses of personal information data that can benefit users.

[0101] The present disclosure also envisions that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. For example, personal information from users should be collected for the entity's legitimate and reasonable purposes and not shared or sold outside of these legitimate uses. In addition, such collection should only be carried out after the user's informed consent. In addition, such entities should take any necessary steps to safeguard and protect access to such personal information data and ensure that others who can access personal information data comply with their privacy policies and procedures. In addition, such entities can subject themselves to third-party assessments to prove that they comply with widely accepted privacy policies and practices.

[0102] Regardless of the foregoing, the present disclosure also contemplates implementations in which users can selectively block the use or access of personal information data. That is, the present disclosure contemplates providing hardware and / or software components to prevent or block access to such personal information data. For example, a user may choose not to provide precise location information but permit the transmission of location area information.

[0103] Figure 25 An example of a data processing system 2500 that can be used with an embodiment of the present invention is shown. For example, a system including Figure 1 The system 2500 of the device 100 is shown. Note that although Figure 25 Various components of the computer system are shown, but they are not intended to represent any particular configuration or manner of interconnecting these components, and such details are not germane to the present invention. It should also be understood that network computers and other data processing systems or other consumer electronic devices having fewer or perhaps more components may also be used with the present invention.

[0104] like Figure 25As shown, the computer system 2500 in the form of a data processing system includes a bus 2503 coupled to one or more microprocessors 2505, ROM (read-only memory) 2507, volatile RAM 2509 and non-volatile memory 2511. The microprocessor 2505 may include one or more CPUs, GPUs, special-purpose processors and / or combinations thereof. The microprocessor 2505 can retrieve instructions from memory 2507, 2509, 2511 and execute the instructions to perform the above-mentioned operations. The bus 2503 is interconnected with these various components and interconnects these components 2505, 2507, 2509 and 2511 to a display controller and a display device 2513, as well as to peripheral devices such as input / output (I / O) devices, which may be a mouse, keyboard, modem, network interface, printer and other devices well known in the art. Typically, the input / output device 2515 is coupled to the system via the input / output controller 2513. The volatile RAM (Random Access Memory) 2509 is typically implemented as dynamic RAM (DRAM) which requires continuous power to refresh or retain data in the memory.

[0105] The mass storage device 2511 is typically a magnetic hard drive, a magneto-optical drive, an optical drive, a DVD RAM, a flash memory, or other type of memory system that retains data (e.g., large amounts of data) even after the system is powered off. Typically, the mass storage device 2511 may also be a random access memory, although this is not required. Although Figure 25 The mass storage device 2511 is shown as a local device directly coupled to the rest of the components in the data processing system, but it should be understood that the present invention can utilize non-volatile storage that is remote from the system, such as a network storage device coupled to the data processing system through a network interface such as a modem, an Ethernet interface, or a wireless network. The bus 2503 may include one or more buses interconnected by various bridges, controllers, and / or adapters as are well known in the art.

[0106] Figure 26 26 shows an example of another data processing system 2600 that can be used with an embodiment of the present invention. For example, system 2600 can be implemented as Figure 1 The device 100 is shown. Figure 26 The data processing system 2600 shown in FIG. 1 includes a processing system 2611, which may be one or more microprocessors or a system-on-chip integrated circuit, and a memory 2601 for storing data and programs executed by the processing system. The system 2600 may also include an audio input / output subsystem 2605, which may include a microphone and a speaker for, for example, playing back music or providing telephone functionality through the speaker and microphone.

[0107] The display controller and display device 2609 provide a visual user interface for the user; this digital interface may include a graphical user interface similar to that displayed on a Macintosh computer when running the OS X operating system software, or on an Apple iPhone or Apple Watch when running the iOS operating system, etc. The system 2600 also includes one or more wireless transceivers 2603 to communicate with another data processing system (such as a Figure 26 The wireless transceiver may be a WLAN transceiver, an infrared transceiver, a Bluetooth transceiver, and / or a wireless cellular telephone transceiver. It should be understood that in some embodiments, additional components not shown may also be part of the system 2600, and in some embodiments, additional components may also be used in the data processing system. Figure 26 The system 2600 also includes one or more communication ports 2617 to communicate with another data processing system such as Figure 15 The communication port can be a USB port, a FireWire port, a Bluetooth interface, etc.

[0108] The data processing system 2600 also includes one or more input devices 2613 that are provided to allow a user to provide input to the system. These input devices may be a keypad or keyboard or a touch panel or multi-touch panel. The data processing system 2600 also includes an optional input / output device 2615, which may be a connector for docking. It should be understood that, as is well known in the art, one or more buses (not shown) may be used to interconnect the various components. Figure 26 The data processing system shown in the figure may be a handheld computer or personal digital assistant (PDA), or a cellular phone with similar functionality to a PDA, or a handheld computer that includes a cellular phone, or a media player such as an iPod, or a device that combines aspects or functionality of these devices, such as a media player combined with a PDA and a cellular phone in one device or an embedded device or other consumer electronic device. In other embodiments, data processing system 2600 may be a network computer or an embedded processing device within another device, or may have a larger size than that of a computer. Figure 26 Other types of data processing systems may have fewer or more components than those shown.

[0109] At least some embodiments of the present invention may be part of a digital media player, such as a portable music and / or video media player, which may include a media processing system for presenting media, a storage device for storing the media, and may further include a radio frequency (RF) transceiver (e.g., an RF transceiver for a cellular phone) coupled to an antenna system and the media processing system. In certain embodiments, media stored on a remote storage device may be sent to the media player via the RF transceiver. For example, the media may be one or more of music or other audio, still images, or moving images.

[0110] Portable media players may include media selection devices such as the iPod® available from Apple Inc. (Cupertino, CA). or iPod Nano The portable media player can be used to select a media player that is stored in a storage device and / or a remote storage device. In at least some embodiments, the portable media player can include a display device that is coupled to a media processing system to display the title or other indicators of the media selected by the input device and presented by a loudspeaker or earphone or on a display device or on a display device and on a loudspeaker or earphone. The example of the portable media player is described in published U.S. Patent 7,310,671 and the U.S. published patent 2004 / 0224638, which are incorporated herein by reference.

[0111] Part of the foregoing can be realized by utilizing logic circuits such as dedicated logic circuits, or by utilizing microcontrollers or other forms of processing cores for executing program code instructions. Thus, program code such as machine executable instructions can be utilized to execute the process taught by the above discussion, and the machine executable instructions enable the machine to execute these instructions to perform certain functions. In this context, a "machine" can be a machine that converts intermediate form (or "abstract") instructions into processor-specific instructions (e.g., abstract execution environments such as "virtual machines" (e.g., Java virtual machines), interpreters, common language runtimes, high-level language virtual machines, etc.), and / or an electronic circuit that is arranged on a semiconductor chip (e.g., a "logic circuit" implemented using transistors), the electronic circuit being designed to execute instructions, the processor being such as a general-purpose processor and / or a special-purpose processor. The process taught by the above discussion can also be executed by (as a substitute for a machine or in combination with a machine) an electronic circuit that is designed to execute a process (or a part thereof) without executing program code.

[0112] The present invention also relates to an apparatus for performing the operations described herein. The apparatus may be specially constructed for the desired purpose, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored on a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memory (ROM), RAM, EPROM, EEPROM, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, and each coupled to a computer system bus.

[0113] A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, machine-readable media include read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory devices, etc.

[0114] An article of manufacture may be used to store program code. The article of manufacture storing program code may be embodied as, but not limited to, one or more memories (e.g., one or more flash memories, random access memories (static, dynamic, or other)), optical disks, CD-ROMs, DVD ROMs, EPROMs, EEPROMs, magnetic or optical cards, or other types of machine-readable media suitable for storing electronic instructions. Program code may also be downloaded from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by means of a data signal contained in a propagation medium (e.g., via a communication link (e.g., a network connection)).

[0115] The foregoing detailed description has been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are tools used by those skilled in the art of data processing, and these tools can most effectively convey the essence of their work to others skilled in the art. An algorithm, as used herein, generally refers to a self-consistent sequence of operations leading to a desired result. Operations are those that require physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and in other words, manipulated. It has proven convenient, primarily for general purposes, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, and the like.

[0116] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically stated, it will be apparent from the foregoing discussion that discussions throughout this specification using terms such as "detect," "sort," "receive," "determine," "search," "send," "modify," "execute," "screen," "add," "create," "present," "detect," "index," "generate," "link," "simulate," "capture," and the like will refer to actions and processes on a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities in the computer system's registers and memories and converts it into other data similarly represented as physical quantities in the computer system's memories or registers or other such information storage, transmission, or display devices.

[0117] The process presented herein and display are not inherently relevant to any particular computer or other device. According to the teaching content of this paper, various general-purpose systems can be used together with program, or can prove that it is convenient to construct the more special-purpose device for carrying out described operation. According to the description below, the required structure for various these systems will be apparent. In addition, the present invention is not described with reference to any specific programming language. Should be appreciated that multiple programming languages ​​can be used for realizing the teaching content of the present invention as described herein.

[0118] The foregoing discussion describes only some exemplary embodiments of the present invention. Those skilled in the art will readily recognize from this discussion, the accompanying drawings, and the appended claims that various modifications can be made without departing from the spirit and scope of the present invention.

Claims

1. A non-transitory machine-readable medium having executable instructions that cause one or more processing units to perform a method for selecting an application state for use in a multi-device search, the method comprising: detecting, on a device, that the application state has been selected as a query result of a local device-level search of locally indexed application state on the device; as well as In response to the detecting, the application state is sent to a server, where the application state is indexed with other application states from other devices.

2. The machine-readable medium of claim 1 , further comprising: Clean up the application state.

3. The machine-readable medium of claim 2, wherein the cleaning comprises: Remove private information from the application state. 4 . The machine-readable medium of claim 1 , wherein the application state comprises data representing an application at a snapshot in time of the application state.

5. The machine-readable medium of claim 4, wherein a plurality of different application states exist for the application.

6. The machine-readable medium of claim 1 , wherein said being selected comprises: It is determined that the application state has been presented to a user of the application as a result of a query.

7. The machine-readable medium of claim 1, wherein the application state is made available to another device in response to a query that matches the application state.

8. A machine-readable medium having executable instructions that cause one or more processing units to perform a method for performing a search for a first device using an application state received from a second device, the method comprising: receiving, from a plurality of devices, an indication of a plurality of application states, the plurality of application states from a plurality of applications running on the plurality of devices from which the application states originated, wherein each of the plurality of application states is sanitized by the plurality of devices from which the application states originated before the plurality of devices from which the application states originated sent the plurality of application states for indexing, wherein the sanitized application states include application states from which private information has been removed, and wherein at least one of the plurality of application states has been selected as a query result of a local device-level search of locally indexed application states on the device from which the application states originated; for each application state in the plurality of application states, if an indication of the application state is received a number of times that satisfies a threshold, adding the application state to an index, wherein the index includes the plurality of application states that are purged; receiving, from the first device, a query for searching data stored on the second device; searching the plurality of application states using the index and the search query; determining a match of one of the plurality of application states to the search query; as well as The match for matching application state is returned to the second device.

9. The machine-readable medium of claim 8, wherein the threshold is 50.

10. The machine-readable medium of claim 8, wherein adding the plurality of application states to an index further comprises: For each of the plurality of application states, If the application state is above the threshold, a request for the complete application state is sent, receiving the complete application state, and Index the application state.

11. The machine-readable medium of claim 8, wherein the indication of an application state is a hash of the application state.

12. The machine-readable medium of claim 8, wherein the cleaned application state is cleaned using a corresponding application that tagged the private information, and an originating device removes the tagged private information.

13. A method for performing a search for a first device using an application state received from a second device, the method comprising: receiving, from a plurality of devices, an indication of a plurality of application states, the plurality of application states from a plurality of applications running on the plurality of devices from which the application states originated, wherein each of the plurality of application states is sanitized by the plurality of devices from which the application states originated before the plurality of devices from which the application states originated sent the plurality of application states for indexing, wherein the sanitized application states include application states from which private information has been removed, and wherein at least one of the plurality of application states has been selected as a query result of a local device-level search of locally indexed application states on the device from which the application states originated; for each application state in the plurality of application states, if an indication of the application state is received a number of times that satisfies a threshold, adding the application state to an index, wherein the index includes the plurality of application states that are purged; receiving, from the first device, a query for searching data stored on the second device; searching the plurality of application states using the index and the search query; determining a match of one of the plurality of application states to the search query; as well as The match for matching application state is returned to the second device. The method according to claim 13 , wherein the threshold value is 50.

15. The method of claim 13, wherein adding the plurality of application states to an index further comprises: For each of the plurality of application states, If the application state is above the threshold, a request for the complete application state is sent, receiving the complete application state, and Index the application state. The method of claim 13 , wherein the indication of an application state is a hash of the application state.

17. The method of claim 13, wherein each of the plurality of application states comprises data representing a snapshot in time of an application for the application state.

18. The method of claim 13, wherein a plurality of different application states exist for at least one of the plurality of applications.

19. The method of claim 13, wherein the cleaned application state is cleaned using the corresponding application that marked the private information, and the originating device removes the marked private information.

20. A method of selecting an application state for use in a multi-device search, the method comprising: detecting, on a device, that the application state has been selected as a query result of a local device-level search of locally indexed application state on the device; as well as In response to the detecting, the application state is sent to a server, where the application state is indexed with other application states from other devices.

21. The method of claim 20, wherein the application state is made available to another device in response to a query matching the application state.

Images