Use Hybrid Extraction with Caching on the Device

Abstract

The present disclosure relates to hybrid extraction using on-device caching. The disclosed techniques relate to enabling a computing device to process voice queries and provide query responses even when the computing device and the vehicle do not have an Internet connection. According to the disclosed techniques, the computing device can detect a query via an input device of the computing device and output a determined query response based on the detected query. Aspects of the techniques of the present disclosure can enable the computing device to use a query response cache to generate query responses rather than directly querying a remote computing system. The query response cache can include predicted queries and query responses retrieved from a query response cache of a remote computing system, enabling the computing device to respond to detected queries when experiencing an unreliable Internet connection.

Description

Technical Field

[0001] The present disclosure relates to hybrid fetching using on-device caching. Background Art

[0002] Vehicles such as automobiles, motorcycles, airplanes, and boats may include one or more computing systems that perform functions and provide information, entertainment, assistance, and / or environmental control to occupants. For example, an automobile may include an entertainment system for providing entertainment content, a navigation system for providing information and navigation assistance, a temperature control system for heating or cooling the vehicle cabin, a control system for adjusting various components or features of the vehicle (e.g., a sunroof or window shades), or an "infotainment system" that performs some or all of the foregoing functions. Modern vehicles are typically equipped with an automotive head unit (AHU) having a display device (e.g., a presence-sensitive display) and a computing engine configured to execute an operating system and one or more applications.

[0003] In many cases, a vehicle operator can search for one or more queries while traveling in the vehicle, and the AHU can provide answers to the queries via a display or microphone. However, answers to the queries may not always be available during travel, especially when the vehicle is out of range of an Internet connection. Summary of the Invention

[0004] Generally, the technology disclosed herein relates to enabling a computing device to process queries (e.g., voice queries or text queries) and provide answers to the queries without requiring an active network connection. According to the disclosed technology, a computing device can detect a query via an input device of the computing device and output a determined query answer based on the detected query, without requiring the computing device to send a request to a remote computing system after receiving the query and wait for a response from the remote computing system. Various aspects of the technology disclosed herein can enable a computing device to generate query answers using a local query answer cache (i.e., a query answer cache stored locally on the computing device) rather than directly querying a remote computing system. The query answer cache can include predicted queries and query answers retrieved from the query answer cache of the remote computing system, thereby enabling the computing device to respond to detected queries when experiencing an unreliable Internet connection. The query answer cache can include predicted queries and query answers that a user is predicted to ask, thereby enhancing traditional offline query support. Alternatively or additionally, using such a query answer cache can reduce the amount of time required for the computing device to provide query answers, even when the computing device has a reliable Internet connection. In this way, the disclosed technology can reduce query answer latency and reduce or eliminate the use of network data when users provide queries, thereby potentially improving search efficiency of computing devices.

[0005] In one example, the present disclosure relates to a method that includes: receiving, by a computing device, a user input; determining, by the computing device and at least partially based on the user input, at least one text query; in response to determining the at least one text query, generating, by the computing device, one or more context parameters associated with the at least one text query, wherein the one or more context parameters include at least one of a current time, a current location, a current user of the computing device, a set of historical user queries, a set of historical user behaviors, and a predicted destination; querying, by the computing device and based on the at least one text query and the one or more context parameters, a query answer cache of the computing device to generate a query result; determining, based on the query result, whether the query answer cache includes a query answer for the at least one text query and the one or more context parameters; in response to determining that the query answer cache does not include the query answer: sending, from the computing device, an indication of the at least one text query and an indication of the one or more context parameters to a remote computing system; and receiving, from the remote computing system, an indication of a query answer, wherein the query answer is retrieved from a query answer cache of the remote computing system, wherein the query answer cache includes a plurality of predicted query answers for the computing device, wherein each predicted query answer from the plurality of predicted query answers is associated with a corresponding text query and a corresponding set of context parameters; and outputting, by the computing device, the indication of the query answer.

[0006] In another example, aspects of the technology relate to a computing device that includes: at least one processor; and at least one computer-readable storage device storing instructions that, when executed by the at least one processor, cause the at least one processor to: receive a user input; determine at least one text query at least partially based on the user input; in response to determining the at least one text query, generate one or more context parameters associated with the at least one text query, wherein the one or more context parameters include at least one of a current time, a current location, a current user of the computing device, a set of historical user queries, a set of historical user behaviors, and a predicted destination; query a query answer cache of the computing device to generate a query result based on the at least one text query and the one or more context parameters; determine, based on the query result, whether the query answer cache includes a query answer for the at least one text query and the one or more context parameters; in response to determining that the query answer cache does not include the query answer: send an indication of the at least one text query and an indication of the one or more context parameters to a remote computing system; and receive an indication of a query answer from the remote computing system, wherein the query answer is retrieved from a query answer cache of the remote computing system, wherein the query answer cache includes a plurality of predicted query answers for the computing device, wherein each predicted query answer from the plurality of predicted query answers is associated with a corresponding text query and a corresponding set of context parameters; and output the indication of the query answer.

[0007] In another example, aspects of the technology relate to a computer-readable storage device storing instructions that, when executed, cause at least one processor of a computing device to perform operations. These exemplary operations include: receiving user input; determining at least one text query based at least in part on the user input; in response to determining at least one text query, generating one or more context parameters associated with the at least one text query, wherein the one or more context parameters include at least one of a current time, a current location, a current user of the computing device, a set of historical user queries, a set of historical user behaviors, and a predicted destination; querying a query answer cache of the computing device based on the at least one text query and the one or more context parameters to generate a query result; determining based on the query result whether the query answer cache includes a query answer for the at least one text query and the one or more context parameters; in response to determining that the query answer cache does not include a query answer: sending an indication of the at least one text query and an indication of the one or more context parameters from the computing device to a remote computing system; and receiving an indication of a query answer from the remote computing system, wherein the query answer is retrieved from a query answer cache of the remote computing system, wherein the query answer cache includes a plurality of predicted query answers for the computing device, wherein each predicted query answer from the plurality of predicted query answers is associated with a corresponding text query and a corresponding set of context parameters; and outputting an indication of the query answer.

[0008] Details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a conceptual diagram showing a side view of the interior of a vehicle according to one or more aspects of the present disclosure, wherein an exemplary mobile computing device is configured to provide an indication of a query answer to an exemplary vehicle computing system.

[0010] Figure 2 is a block diagram showing additional details of an exemplary mobile computing device configured to provide an indication of a query answer to an exemplary vehicle computing system according to one or more aspects of the present disclosure.

[0011] Figure 3 is a conceptual diagram showing an exemplary vehicle computing system configured to generate an indication of a query answer according to one or more aspects of the present disclosure.

[0012] Figure 4 is a conceptual diagram showing an exemplary remote computing system configured to generate queries and query answers and communicate with an exemplary client computing device according to one or more aspects of the present disclosure.

[0013] Figure 5 is a flowchart showing exemplary operations performed by a mobile computing device in accordance with one or more aspects of the present disclosure.

[0014] Figure 6 is a flowchart showing exemplary operations performed by a remote computing system in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION

[0015] Figure 1 is a conceptual diagram showing a side view inside a vehicle 100 in accordance with one or more aspects of the present disclosure, where an exemplary mobile computing device 120 is configured to provide an indication of a query response to an exemplary vehicle computing system 112 of the vehicle 100. Figure 1 Illustrates a cross-sectional view inside a vehicle 100 having a vehicle computing system 112. The vehicle 100 can be an automobile, but aspects of the present disclosure can also apply to other types of vehicles, including trucks, motorcycles, airplanes, boats, trains, or other vehicles.

[0016] Generally, the vehicle computing system 112 can operate to assist, inform, entertain, or perform other tasks that require interaction between the user and the vehicle occupants. The vehicle computing system 112 can be referred to as an automotive head unit, a vehicle head unit, an infotainment system, or a sub-component thereof. For example, the vehicle computing system 112 can execute various applications that perform functions or process information for one or more of the vehicle's occupants. For example, the vehicle computing system 112 can provide navigation services that provide directions to a destination. The vehicle computing system 112 can also provide vehicle data about the vehicle, or multimedia such as audio or video. The vehicle computing system 112 can also provide an information retrieval service that provides information in response to a query. These are just a few examples of the functionality that the vehicle computing system 112 can provide, and the vehicle computing system 112 can provide many additional capabilities.

[0017] In some instances, the vehicle computing system 112 may have its own cellular data service and may be configured to retrieve one or more query responses based on one or more queries (e.g., wirelessly retrieve one or more query responses from an external server). The one or more queries may be from the user 150 (e.g., via text input or audio input from the user 150). In some instances, the vehicle computing system 112 may not have any cellular data service, and the user 150 may wish to use the mobile computing device 120 to establish a data connection with the vehicle computing system 112 (e.g., for a search query). The mobile computing device 120 may have a cellular connection and may be configured to provide one or more query responses to the vehicle computing system 112 via one or more data connections - such as one or more wired or wireless data connections that enable data communication with the vehicle computing system 112).

[0018] However, during driving, query responses may not always be available. For example, when the vehicle 100 and the mobile computing device 120 are outside the range of an Internet connection, query responses may not be available.

[0019] By leveraging one or more techniques of the present disclosure, the vehicle computing system 112 can provide an improved mechanism by which the vehicle computing system 112 is configured to obtain one or more query responses without requiring a fast Internet connection or even an Internet connection at all. As will be described in further detail below, the vehicle computing system 112 can receive a query (e.g., via the display device 114 or the microphone 116) and generate one or more context parameters associated with the query. The vehicle computing system 112 can also use a query response cache to generate a query result based on the received query and the one or more context parameters. The query result indicates whether the query response cache includes a query response for the received query.

[0020] The query response cache may include query responses retrieved from a query response cache of a remote computing system. The query response cache may include predicted text queries and query responses. Each predicted query response from the predicted query responses is associated with a corresponding text query and a corresponding set of context parameters. If the query result indicates that the query response cache includes a query response, the vehicle computing system 112 can output an indication of the query response retrieved from the query response cache. If the query result indicates that the query response cache does not include a query response, an indication of at least one text query and an indication of one or more context parameters may be sent to the remote computing system to retrieve an indication of the query response.

[0021] In some cases, query answer caching can be used in combination with query answering to retrieve query results and perform Internet searches. In these cases, retrieving query answers from the query answer cache helps to speed up the time taken by the vehicle computing system 112 and / or the mobile computing device 120 to perform Internet searches.

[0022] In this way, the disclosed techniques can improve the processing speed of the vehicle computing system 112 to obtain query answers for queries using the query answer cache, thereby potentially improving the search efficiency and power efficiency of the vehicle computing system 112, as well as enabling offline searches. The disclosed techniques can potentially reduce the number of processing cycles or power usage of the vehicle computing system 112 by reducing or eliminating Internet searches.

[0023] The query answer cache can be located on the vehicle computing system 112 and / or on an external device, such as the mobile computing device 120. As Figure 1 shown, the mobile computing device 120 includes a query answer cache 128. The mobile computing device 120 also includes a presence-sensitive display device 122, a communication unit 123, an application 124, and a query information memory 126. The mobile computing device 120 is configured to establish a data connection with the vehicle computing system 112 to receive user input and generate a query answer based on the user input by querying the query answer cache 128.

[0024] In some cases, user input can be detected via the display device 114 of the vehicle computing system 112. As an example, the display device 114 of the vehicle computing system 112 can include a presence-sensitive screen that can receive tactile input from a user 150 of the vehicle computing system 112. For example, the display device 114 can be used as a presence-sensitive input device using a presence-sensitive screen technology such as a resistive touch screen, a surface acoustic wave touch screen, a capacitive touch screen, a projected capacitive touch screen, a pressure-sensitive screen, an acoustic pulse recognition touch screen, or another presence-sensitive screen technology. The display device 114 can receive an indication of tactile input by detecting one or more inputs from the user 150 of the vehicle computing system 112 (e.g., the user touches or points to one or more locations on the display device 114 with a finger). For example, the display device 114 can receive a text input from the user 150 that includes "what is the weather".

[0025] In some cases, user input can be detected via the microphone 116 of the vehicle computing system 112. For example, the vehicle computing system 112 can capture oral audio data from the microphone 116. According to the techniques of the present disclosure, the vehicle computing system 112 can use the microphone 116 to capture audio only with the explicit permission from the user 150. For example, the user 150 can initialize the execution of the assistive application module, which causes the display device 114 to output a graphical user interface for the assistive application module. Without the explicit authorization of the user, the vehicle computing system 112 will not capture oral audio data from the microphone 116.

[0026] In response to detecting user input, one or more communication units 123 of the mobile computing device 120 can communicate with the vehicle computing system 112 to receive user input data. The communication unit 123 can implement wired and / or wireless communication with the vehicle computing system 112, including short-range wireless communication (e.g., Bluetooth, WIFI, or Bluetooth Low Energy (BLE) protocol). The mobile computing device 120 also includes one or more ports 121 (e.g., one or more Universal Serial Bus (USB) ports) coupled to the communication unit 123. For example, the mobile computing device 120 can receive oral audio data from the vehicle computing system 112 using a USB data pipe. The mobile computing device 120 can further determine at least one text query based at least in part on the oral audio data. For example, the vehicle computing system 112 can capture audio including "what is the weather", and can transmit data indicating the captured audio to the mobile computing device 120. Using speech-to-text conversion technology, the mobile computing device 120 can generate the text query "what is the weather" from the audio data.

[0027] In some cases, the mobile computing device 120 can generate one or more text queries based on verbal audio data and can provide the generated one or more text queries to the display device 114 for display. The display device 114 can be used as an output device using any one of one or more display devices such as a liquid crystal display (LCD), a dot matrix display, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro light emitting diode (micro-LED) display, an active matrix organic light emitting diode (AMOLED) display, electronic ink, or a similar monochromatic or color display capable of outputting visual information to a user or a vehicle occupant. For example, the vehicle computing system 112 can capture audio including "when is Emma Sherron’s birthday", and the mobile computing device 120 can generate two text queries "when is Emma Sherron’s birthday" and "when is Emily Sherron’s birthday". The mobile computing device 120 can transmit the two text queries to the vehicle computing system 112 for display. The user 150 can interact with the display device 114 to select the text query that matches the verbal query among the one or more text queries.

[0028] In response to determining the text query, the mobile computing device 120 can generate one or more context parameters associated with the determined text query. Examples of the one or more context parameters include, but are not limited to, the current time, the current location, the current user of the mobile computing device, the set of historical user queries, the set of historical user behaviors, the predicted destination, and other suitable context parameters. For example, the mobile computing device 120 can generate the text query "what is the weather" from the received audio and can generate context parameters including the current time and the current location of the mobile computing device 120. The mobile computing device 120 can also store the generated context parameters in the query information memory 126.

[0029] In various examples, the storage of context parameters—such as the current location data of the mobile computing device 120 or historical user behavior associated with the user 150—can occur only in response to receiving affirmative consent from the user 150 by the mobile computing device 120 (e.g., via the presence-sensitive display device 122). For example, the presence-sensitive display device 122 can display to the user an explicit request regarding the storage and subsequent use of such data, and the mobile computing device 120 can continue to store the data only when an affirmative response and consent are received from the user (e.g., via the presence-sensitive display device 122).

[0030] In addition to the above description, the user 150 can be provided with controls that allow the user to select whether and when the systems, programs, or features described herein can collect and / or store user information (e.g., historical user behavior data, user location data, user identification data), and / or whether and when the systems, programs, or features described herein can transfer content or communicate between devices. Additionally, certain data can be processed in one or more ways before storage or use to remove identifiable information. For example, a user's identity can be processed such that no identifiable information can be determined for the user, or the user's geographical location can be generalized where location information is obtained (such as to the city, zip code, or state level) such that the user's specific location cannot be determined. Thus, the user 150 can control what information about the user is collected, how that information is stored and / or used, and what information is provided to the user.

[0031] In response to the mobile computing device 120 generating one or more context parameters associated with a determined text query, the mobile computing device 120 can query the query answer cache 128 of the mobile computing device 120 to generate a query result. The query answer cache 128 can include query answers retrieved from the query answer cache 144 of the server system 140. The mobile computing device 120 can use the determined text query and the one or more context parameters associated with the determined text query as inputs to output a query result indicating whether the query answer cache 128 includes a query answer for the determined text query and the one or more context parameters associated with the determined text query.

[0032] By querying the query answer cache 128, the mobile computing device 120 can determine whether the query answer cache 128 includes a query answer for the determined text query and one or more context parameters associated with the determined text query. If the query result indicates that the query answer cache 128 includes the query answer, the mobile computing device 120 can output an indication of the query answer to the vehicle computing system 112. If the query result indicates that the query answer cache does not include the query answer, the mobile computing device 120 can perform an Internet search operation by sending an indication of the text query and an indication of one or more context parameters associated with the determined text query to a remote computing system (e.g., to the server system 140) to retrieve an indication of the query answer. The mobile computing device 120 can also transmit the indication of the query answer retrieved from the remote computing system to the vehicle computing system 112. The vehicle computing system 112 can output an indication of the query answer via the display device 114 and / or the microphone 116.

[0033] In some examples, the query answer can include one or more auxiliary commands associated with the query answer, and the vehicle computing system 112 can execute one or more auxiliary commands together with the received query answer indication based on the query answer. For example, if the information in the query answer cache 118 and / or the query answer cache 128 is stale (e.g., the answer is inaccurate, incorrect, or not relevant to the current time when the query is received), the vehicle computing system 112 and / or the mobile computing device 120 can identify one or more auxiliary commands included in the query answer cache and execute one or more auxiliary commands to retrieve the latest information for the query. When executing one or more auxiliary commands, the vehicle computing system 112 can send a query to a remote computing system (e.g., the server system 140) and receive a response including the query answer from the remote computing system.

[0034] In some instances, it is possible for the mobile computing device 120 to use the query answer cache 128 in combination with on-device extraction to perform an Internet search operation. In these cases, the mobile computing device 120 can initiate an Internet search operation when determining the text query. Once the query answer is received via the Internet search or on-device extraction, the mobile computing device 120 can transmit an indication of the query answer to the vehicle computing system 112. Thus, using the query answer cache 128 to retrieve the query answer helps to speed up the time it takes for the mobile computing device 120 to perform an Internet search.

[0035] In some instances, both the mobile computing device 120 and the vehicle computing system 112 may have query answer caches, such as the query answer cache 128 in the mobile computing device 120 and the query answer cache 118 in the vehicle computing system 112. In these instances, the vehicle computing system 112 may determine a text query based on user input detected via the microphone 116 and / or the display device 114. In response to determining the text query, the vehicle computing system 112 may generate one or more context parameters associated with the determined text query and query the query answer cache 118 in the vehicle computing system 112 to generate a first query result. The first query result indicates whether the query answer cache 118 includes a query answer for the determined text query and one or more context parameters associated with the determined text query. Similar to the query answer cache 118, the query answer cache 128 may include query answers retrieved from the query answer cache 144 of the server system 140. If the first query result indicates that the query answer cache 128 does not include a query answer, the vehicle computing system 112 may send an indication of the determined text query and an indication of one or more context parameters to the mobile computing device 120. The mobile computing device 120 may then query the query answer cache 118 to generate a second query result. Similar to the first query result, the second query result indicates whether the query answer cache 118 includes a query answer for the determined text query. If the second query result also indicates that the query answer cache 118 does not include a query answer, the mobile computing device 120 may then send an indication of the determined text query and an indication of one or more context parameters to the server system 140 to retrieve an indication of a query answer for the determined text query.

[0036] In various examples, the query answer cache 118 and / or the query answer cache 128 includes query answers retrieved from the query answer cache 144 of the server system 140. For example, the query answer cache 128 may be synchronized with the query answer cache 144 of the server system 140 via the network 130. In some cases, the query answer cache 118 and / or the query answer cache 128 may be synchronized with the query answer cache 144 of the server system 140 periodically, such as once a day, once a week, or once a month. In some cases, the query answer cache 118 and / or the query answer cache 128 may be synchronized with the query answer cache 144 of the server system 140 based on a predicted lack of network connectivity. The query answer cache 144 may include predicted text queries and query answers. Each query answer stored in the query answer cache 144 is associated with a corresponding text query and a corresponding set of context parameters.

[0037] In some cases, query responses retrieved from the query response cache 144 of the server system 140 include one or more global query responses generated using global query data collected from multiple computing devices. The server system 140 may be configured to receive search queries collected from multiple mobile computing devices and may determine search query samples and corresponding query responses based on the received search queries. For example, the server system 140 may be configured to rank the search queries collected from multiple computing devices based on search volume over a predefined time period and may select one or more top-ranked search queries as search query samples.

[0038] In some cases, query responses retrieved from the query response cache 144 of the server system 140 include one or more local query responses generated using local data collected from the mobile computing device 120 and / or the vehicle computing system 112. For example, the server system 140 may be configured to receive search queries collected from the mobile computing device 120 / or the vehicle computing system 112 and may generate search query samples and corresponding query responses based on the search queries collected from the mobile computing device 120 / or the vehicle computing system 112. For example, the server system 140 may be configured to rank the search queries collected from the computing device 120 / or the vehicle computing system 112 based on search volume over a predefined time period and may select one or more top-ranked search queries as search query samples. In some examples, one or more local query responses may include one or more auxiliary commands associated with the one or more local query responses.

[0039] In some cases, query responses retrieved from the query response cache 144 of the server system 140 include one or more predicted query responses generated using the prediction model 146. The prediction model is trained using local query data collected from the mobile computing device 120 / or the vehicle computing system 112. For example, the prediction model may be a machine learning model that takes as input search queries collected from the computing device 120 / or the vehicle computing system 112 and context parameters associated with the search queries and outputs one or more queries that a user 150 of the mobile computing device 120 / or the vehicle computing system 112 may be likely to search for.

[0040] In this way, the disclosed techniques may improve the processing speed of the vehicle computing system 112 to obtain query responses for queries using the query response cache, thereby potentially improving the search efficiency and power efficiency of the vehicle computing system 112 and enabling offline search. The disclosed techniques may potentially reduce the number of processing cycles or power usage of the vehicle computing system 112 by reducing or eliminating Internet searches.

[0041] Figure 2 FIG. 220 is a block diagram of an exemplary mobile computing device configured to provide an indication of a query response to an exemplary vehicle computing system in accordance with one or more aspects of the present disclosure. In some cases, the mobile computing device 220 may be Figure 1 a more detailed example of the mobile computing device 120. Figure 2 Only one specific example of the mobile computing device 220 is shown, and many other examples of the mobile computing device 220 may be used in other examples. In various cases, the mobile computing device 220 may include Figure 2 a subset of the components shown, or may include Figure 2 additional components not shown.

[0042] In Figure 2 an example, the mobile computing device 220 includes a presence-sensitive display device 222, one or more processors 230, one or more input components 242, one or more communication units 246, one or more output components 244, and one or more storage devices 248. A communication channel 249 may interconnect each of the components 221, 222, 230, 242, 244, 246, and / or 248, which is used for inter-component communication (physically, communicatively, and / or operably). In some instances, the communication channel 249 may include a system bus, a network connection, an interprocess communication data structure, or any other method for communicating data between hardware and / or software.

[0043] One or more input components 242 of the mobile computing device 220 may receive input, such as input from a user. Examples of input are touch / haptic, presence-sensitive, and audio input. Examples of input components 242 include a presence-sensitive screen, a touch-sensitive screen, a touch screen, a mouse, a keyboard, a trackpad, a voice response system, a video camera, a microphone, or any other type of device for detecting input from a human or a machine.

[0044] One or more output components 244 of the mobile computing device 220 may generate output. Examples of output are tactile, audio, and visual output. Examples of output components 244 include a presence-sensitive screen, a touch-sensitive screen, a touch screen, a sound card, a video graphics adapter, a speaker, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a micro light emitting diode (micro-LED) display, an active matrix organic light emitting diode (AMOLED) display, or any other type of device for generating output to a human or a machine.

[0045] One or more communication units 246 of the mobile computing device 220 may communicate with external devices via one or more networks by transmitting and / or receiving network signals over one or more networks (e.g., one or more wired and / or wireless networks). For example, the mobile computing device 220 may use the communication unit 246 to transmit and / or receive radio signals over a radio network such as a cellular radio network. Similarly, the communication unit 246 may transmit and / or receive satellite signals over a satellite network such as a Global Positioning System (GPS) network. Examples of the communication unit 246 include network interface cards (e.g., such as an Ethernet card), optical transceivers, radio frequency transceivers, GPS receivers, or any other type of device that can send and / or receive information. Other examples of the communication unit 246 may include shortwave radios, cellular data radios, wireless Ethernet network radios, and Universal Serial Bus (USB) controllers.

[0046] The mobile computing device 220 includes a presence-sensitive display device 222 and one or more ports 247 (e.g., USB ports) and others. Similar to the description referenced above Figure 1 the mobile computing device 220 may establish a wired communication with the vehicle computing system.

[0047] One or more storage devices 248 within the computing device 200 include a query information memory 250 and a query answer cache 252. The storage device 248 may store information for processing during the operation of the mobile computing device 220 (e.g., during the execution of one or more of the query fulfillment module 254, the wireless connection module 256, the application 258, the auxiliary application 262, the UI module 264, and / or the operating system 266). For example, the auxiliary application 262 of the mobile computing device 220 may generate one or more context parameters associated with a text query and may store one or more contexts in the information memory 250. Similar to the description referenced above Figure 1 the query answer cache 252 may include predicted queries and query answers retrieved from the query answer cache of a remote computing system.

[0048] In some examples, the storage device 248 includes a temporary memory, which means that the primary purpose of the storage device 248 is not for long-term storage. The storage device 248 on the mobile computing device 220 may be configured as a volatile memory for short-term storage of information and thus does not retain the stored content if power is lost. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0049] In some examples, the storage device 248 includes one or more computer-readable storage media. The storage device 248 can be configured to store a larger amount of information than volatile memory. The storage device 248 can also be configured as non-volatile memory space for long-term storage of information and retain information after power-on / power-off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or in the form of electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. The storage device 248 can store program instructions and / or data associated with one or more query fulfillment modules 254, wireless connection modules 256, applications 258, auxiliary applications 262, UI modules 264, and / or operating systems 266. In some examples, one or more of the query fulfillment modules 254, wireless connection modules 256, applications 258, auxiliary applications 262, UI modules 264, and / or operating systems 266. In other cases, one or more of the query fulfillment modules 254, wireless connection modules 256, applications 258, auxiliary applications 262, UI modules 264, and / or operating systems 266 can be stored remotely in the cloud (e.g., on one or more external servers).

[0050] In various cases, the query fulfillment module 254 can store data in the query information memory 250 and / or the query answer cache 252, or retrieve data therefrom. In various examples, the storage of user data - such as location information or identification information associated with the user - can occur only in response to receiving affirmative consent from the user by the mobile computing device 220 (e.g., via the UI module 264). For example, the UI module 264 can provide an explicit request to the user regarding the storage and subsequent use of such data, and the mobile computing device 220 can continue to store data locally only upon receiving an affirmative response and consent from the user.

[0051] The wireless connection module 256 can be configured to control wireless connection events and / or operations of the mobile computing device 220. In various cases, the wireless connection module 256 can establish one or more wireless connections with one or more external devices or systems. In certain cases, the wireless connection module 256 can store unique identifiers of external devices or systems with which it has established wireless connections, as well as any other information associated with these connections (e.g., connection identifiers, protocol identifiers, port identifiers, connection passwords or keys, timestamps, pairing information, etc.).

[0052] One or more processors 230 may implement functionality and / or execute instructions within mobile computing device 220. For example, a processor 230 on mobile computing device 220 may receive and execute instructions stored by storage device 248, which perform the functionality of query fulfillment module 254, wireless connection module 256, applications 258, auxiliary applications 262, UI module 264, and / or operating system 266. These instructions, when executed by processor 230, may cause mobile computing device 220 to store information within storage device 248 during program execution. Processor 230 may execute instructions of query fulfillment module 254, wireless connection module 256, applications 258, auxiliary applications 262, UI module 264, and / or operating system 266 to perform one or more operations. That is, query fulfillment module 254, wireless connection module 256, applications 258, auxiliary applications 262, UI module 264, and / or operating system 266 may be operable by processor 230 to perform the various functions described herein.

[0053] In some examples, mobile computing device 220 may only include or otherwise comprise processor 230. In such examples, one or more of input component 242, presence sensitive display device 222, communication unit 246, output component 244, and storage device 248 may be external to mobile computing device 220, but communicatively coupled to mobile computing device 220 (e.g., via communication channel 249).

[0054] In some examples, computing device 200 may predict future usage of the assistive application 262. Future usage may include predicting one or more queries to be received by the assistive application 262, and predicting a time range in which the assistive application 262 receives at least one of the one or more queries (e.g., receiving user input during a particular part of the day, such as between 8 a.m. and 5 p.m.). In various instances, computing device 200 may predict future usage based on prior usage. That is, computing device 200 may maintain an assistive application usage history and / or a query history. Such histories may be user-specific, device-specific, cross-device, or crowdsourced (e.g., including information from other users, such as those who may have profiles similar to the current user of computing device 200). Based on this history, computing device 200 may identify specific time ranges (e.g., day of the week, time of day, part of a month, part of a year, etc.) in which the user has previously used the assistive application 262 and / or requested various queries. For example, a user may regularly execute the assistive application 262 and ask "what is the weather going to be like today" around 8 a.m. on each weekday and around 9 a.m. on each weekend. For example, based on this usage history, computing device 200 may predict that the user of computing device 200 will query the weather for the day between 7 a.m. and 10 a.m. each morning, or between 7 a.m. and 9 a.m. on weekdays and between 8 a.m. and 10 a.m. on weekends. In some examples, the predicted queries and query answers may include time-sensitive information. For example, a user may regularly execute the assistive application 262 and ask "what is the weather right now" around 8 a.m. each day. Based on this usage history, computing device 200 may predict that the user of computing device 200 will query the weather around 8 a.m. each day. By predicting the queries and query answers that the user is predicted to ask, computing device 220 enhances traditional offline query support because it can provide more detailed information.

[0055] Computing device 200 may update the query answer cache 252 periodically based on the predicted future usage of the assistive application 262. For example, based on the predicted future usage of the assistive application 262, computing device 200 may update the query answer cache 252 by retrieving the predicted queries and query answers from the query answer cache of a remote computing system. In an example where computing device 200 predicts that the user will query the weather between 7 a.m. and 9 a.m. during the week, computing device 200 may update the query answer cache 252 before 7 a.m. (e.g., at 5 a.m., 6 a.m., 6:59 a.m., etc.) to retrieve the weather forecast for the day.

[0056] In some examples, computing device 200 may predict the network connection state of computing device 200 based on predicted future usage of auxiliary application 262, such as predicting that auxiliary application 262 may receive user input during a particular part of the day (e.g., between 8:00 a.m. and 9:00 a.m.). Computing device 200 may also determine whether the predicted network connection state indicates that computing device 200 has a network connection based on the time range during which auxiliary application 262 is predicted to receive user input.

[0057] When predicting future network connection states, computing device 200 with explicit user permission may maintain a network connection history that includes one or more of network connection state, the location of computing device 200 (e.g., zip code, geographical region, GPS coordinates, etc.), date information, and time information. As an example, a user may regularly go to a sandwich shop for lunch. Along the path to the sandwich shop, computing device 200 has a low-quality (e.g., weak wireless signal, reduced data transfer rate, etc.) network connection or a network connection interruption. Additionally, when going to the sandwich shop, the user may execute auxiliary application 262 and query the top news headlines for the day. Typically, the query may fail due to the low quality or interruption of the network connection. However, by using the techniques of the present disclosure, computing device 200 may predict the low quality and / or interruption of the network connection and proactively update query answer cache 252 to include information about the top news headlines. That is, based on determining that the predicted network connection state indicates that computing device 200 does not have a network connection within that time range, computing device 200 may proactively update query answer cache 252 by retrieving predicted queries and query answers from the query answer cache of the remote computing system before the predicted problematic network connection state occurs.

[0058] Figure 3 is a conceptual diagram showing an exemplary automotive head unit 312 configured to generate indications of query answers. In some cases, automotive head unit 312 may be Figure 1 a more detailed example of vehicle computing system 112. Figure 3 Only one specific example of automotive head unit 312 is shown, and many other examples of automotive head unit 312 may be used in other instances. In various cases, automotive head unit 312 may include Figure 3 a subset of the components shown, or may include Figure 3 additional components not shown in

[0059] In Figure 3 the example of, vehicle 300 includes processor 306, graphics processing unit (GPU) 308, and system memory 318. In some examples, processor 306, GPU 308, and transceiver module ( Figure 3Not shown) may be formed into an integrated circuit (IC). For example, an IC may be considered a processing chip within a chip package and may be a system-on-chip (SoC).

[0060] Examples of the processor 306 and the GPU 308 include, but are not limited to, one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. The processor 306 may represent the central processing unit (CPU) of the vehicle 300. In some examples, the GPU 308 may be dedicated hardware that includes integrated and / or discrete logic circuits that provide the GPU 308 with the ability for large-scale parallel processing suitable for graphics processing. In some instances, the GPU 308 may also include general-purpose processing capabilities and may be referred to as a general-purpose GPU (GPGPU) when implementing general-purpose processing tasks (i.e., non-graphics-related tasks). Although shown as a dedicated GPU 308, the GPU 308 may represent an integrated GPU integrated into the underlying circuit board (e.g., a so-called "motherboard") or otherwise incorporated into the processor 306.

[0061] The processor 306 may execute various types of applications. Examples of applications include a web browser, an email application, a spreadsheet, a video game, or other applications that generate visual objects for display. The system memory 318 may store instructions for the execution of one or more applications. The execution of the application by the processor 306 causes the processor 306 to generate graphic data of the image content to be displayed. The processor 306 may transmit the graphic data of the image content to the GPU 308 for further processing based on the instructions or commands transmitted by the processor 306 to the GPU 308.

[0062] The processor 306 may communicate with the GPU 308 according to an application programming interface (API). Additionally, the techniques described in this disclosure do not need to operate according to an API, and the processor 306 and the GPU 308 may use any technique to communicate with the GPU 308.

[0063] The system memory 318 may represent the memory of the vehicle 300. The system memory 318 may include a query information memory 350 and a query answer cache 352. The system memory 318 may include one or more computer-readable storage media. Examples of the system memory 318 include, but are not limited to, random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other media that can be used to carry or store the required program code in the form of instructions and / or data structures and can be accessed by a computer or a processor.

[0064] In some aspects, the system memory 318 can include instructions that cause the processor 306 to perform the functions ascribed to the processor 306 in this disclosure. Accordingly, the system memory 318 can be a non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors (e.g., processor 306) to perform various functions.

[0065] The system memory 318 is a non-transitory storage medium. The term "non-transitory" indicates that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term "non-transitory" should not be construed to mean that the system memory 318 is immovable or that its contents are static. As an example, the system memory 318 can be removed from the vehicle 300 and moved to another device. As another example, a memory substantially similar to the system memory 318 can be inserted into the autonomous vehicle 300. In a particular example, the non-transitory storage medium can store data that can change over time (e.g., in RAM).

[0066] As Figure 3 further shown in the example of, the vehicle 300 can include a display 314, a microphone 316, and a user interface 304. The display 314 can represent any type of passive reflective screen that can project an image, or an active reflective or emissive or transmissive display capable of displaying an image (such as a light-emitting diode (LED) display, an organic LED (OLED) display, a liquid crystal display (LCD), or any other type of active display). Although illustrated as including a single display 314, the vehicle 300 can include multiple displays that can be located throughout the cockpit of the vehicle 300. In some examples, a passive version of the display 314 or a particular type of active version of the display 314 (e.g., an OLED display) can be integrated into a seat, a table, a headliner, a floor, a window (or, in a vehicle with few or no windows, a wall), or other aspects of the vehicle cockpit. When the display 314 represents a passive display, the display 314 can also include a projector or other image projection device capable of projecting or otherwise re-creating an image on the passive display 314. Additionally, the display 314 can include a display integrated into the driver-side instrument panel that virtually represents a physical instrument cluster (illustrating speed, revolutions, engine temperature, etc.).

[0067] The user interface 304 can represent any type of physical or virtual interface through which a user can control various functions of the vehicle 300. The user interface 304 can include physical buttons, knobs, sliders, or other physical control tools. The user interface 304 can also include a virtual interface, whereby a passenger of the vehicle 300 interacts with virtual buttons, knobs, sliders, or other virtual interface elements via, as an example, a touch-sensitive screen or via a non-touch interface. A passenger can interface with the user interface 304 to control one or more of the climate within the vehicle 300, the audio playback of the vehicle 300, the video playback of the vehicle 300, a transmission through the vehicle 300 (such as a phone call), or any other operation that can be performed by the vehicle 300.

[0068] When acting as an extension to or in place of a display integrated into the vehicle 300, the user interface 304 can also represent an interface extended to the display 314. That is, the user interface 304 can include a virtual interface presented via a head-up display (HUD), an augmented reality computing device, a virtual reality computing device or display, a tablet computer, or any other different type of extended display listed above.

[0069] In Figure 3 the example of, the processor 306, GPU 308, system memory 318, display 314, and UI 304 can at least partially together represent what is referred to as an automotive head unit (AHU) 312 in an automotive scenario. The AHU 312 can represent any integrated or stand-alone computing device capable of interfacing with various aspects of the vehicle 300 and / or providing entertainment and / or information about the vehicle 300 to a passenger (where such a head unit can be referred to as an "infotainment unit" or "infotainment system").

[0070] As Figure 3As further shown in the example, the vehicle 300 may include a plurality of different vehicle systems 302A - 302N (“vehicle systems 302”). The vehicle systems 302 may include a heating, ventilation, and air conditioning (HVAC) system or a temperature regulation system (e.g., in addition to the HVAC system, it may also include heated and / or cooled seats) (one or both of which may also be referred to as a climate system herein), a lighting system (for providing interior and / or exterior lighting), a seat control system (for adjusting the position of the occupant seat), a mirror control system (for controlling interior and / or exterior mirrors, including rearview mirrors, side mirrors, visor mirrors, etc.), a windshield wiper control system, an entertainment system (for controlling radio playback, video playback, image display, etc.), a safety assistance system (for controlling parking assistance, reverse assistance, etc.), a driving mode system (for controlling the suspension, transmission, etc.), a sunroof control system (for controlling the sunroof), and any other type of vehicle system that can be controlled via a head unit - such as the AHU 312. Examples of the vehicle systems 302 may include an electronic control unit (ECU), which may control any of the foregoing examples of the vehicle systems 302.

[0071] In accordance with aspects of the techniques described in the present disclosure, the AHU 312 is configured to detect user input and generate a query response based on the user input by querying the query response cache 352. The system memory 318 may store information for processing during the operation of the AHU 312 (e.g., during the execution of one or more of the query fulfillment module 354, the wireless connection module 356, and the auxiliary application 362).

[0072] In operation, the AHU 312 may detect user input via the microphone 316 and / or the UI 304 and may determine a text query based on the detected user input. In response to determining the text query, the AHU 312 may generate one or more context parameters associated with the text query and store the generated context parameters in the query information memory 350.

[0073] In response to the AHU 312 generating one or more context parameters associated with the text query, the query fulfillment module 354 may obtain the text query and the one or more context parameters associated with the text query and output one or more query responses. The query fulfillment module 354 may generate one or more query responses via an Internet search and / or on - device extraction.

[0074] In some examples, the query fulfillment module 354 may send an indication of at least one text query and an indication of one or more context parameters to a remote computing system (e.g., the server system 340) to retrieve an indication of the query response.

[0075] In some examples, instead of directly retrieving a query response by querying a remote computing system, the query fulfillment module 354 may retrieve the query response from the query response cache 352. The query response cache 352 may include predicted queries and query responses retrieved from a query response cache (e.g., query response cache 344) of a remote computing system (e.g., server system 340). For example, the AHU 312 may predict that a user will query the weather for the day based on historical data of text queries and may prefetch the weather forecast for the day from the query response cache 344 of the server system 340 to update the query response cache 352.

[0076] In response to retrieving a query response from the query response cache 352, the rendering module 356 may render the retrieved query response to produce an indication of the query response for output via the display 314 and / or the microphone 316. For example, the rendering module 356 may render the retrieved query response to allow the retrieved query response to be scaled and encoded into a known format. Additionally, the rendering module 356 may further generate a two-dimensional (2D) or three-dimensional (3D) graphical view of the retrieved query response. For example, in an example where the retrieved query response indicates a storm, the rendering module 356 may generate a storm symbol for display via the display 314.

[0077] Figure 4 is a conceptual diagram illustrating an exemplary remote computing system according to one or more aspects of the present disclosure, the remote computing system being configured to generate queries and query responses and communicate the queries and query responses with an exemplary client computing device. Figure 4 includes a client device 412 in communication with the remote computing system 440. The client device 412 is Figure 1 an example of a vehicle computing system 112, Figure 1 an example of a mobile computing device 120, Figure 2 an example of a mobile computing device 220, and / or Figure 3 an example of an AHU 312. The remote computing system 440 is Figure 1 an example of an exemplary server system 140 and / or Figure 3 an example of an exemplary server system 340.

[0078] In Figure 4 an example, the remote computing system 440 includes a processing component 442, a memory component 450, and a machine learning model 446. As described below, the machine learning model 446 may be an example of any model used by the query generator 452 to generate one or more queries.

[0079] In some examples, the machine learning model 446 is trained to receive one or more types of input data and, in response, provide one or more types of output data. The input data can include search queries collected from multiple computing devices, search queries collected from the client device 412, and context parameters associated with the search queries collected from the client device 412.

[0080] The machine learning model 446 can output one or more global queries. In some examples, given search queries collected from a pooled population, the machine learning model 446 can generate one or more global queries that are most likely to be searched by the pooled population. For example, the machine learning model 446 can use search queries collected from multiple mobile computing devices as input and can output one or more global queries based on the search volume of the received search queries, e.g., output one or more top-ranked search queries as global queries.

[0081] In some examples, the machine learning model 446 can output one or more local queries. The machine learning model 446 can use search queries collected from the client device 412 as input and generate one or more local queries based on the search queries collected from the client device 412. For example, the machine learning model 446 can rank the search queries collected from the client device 412 based on the search volume within a predefined time period and can output one or more top-ranked search queries as local queries.

[0082] In some examples, the machine learning model 446 can output one or more predicted queries. For example, the machine learning model 446 can use search queries and context parameters associated with the search queries collected from the client device 412 as input and can generate one or more predicted queries that may be searched by the user of the client device 412. As an example, a user may regularly retrieve the weather forecast for a day of the week. Based on this usage history, the machine learning model 446 can generate one or more predicted queries such as "what is the weather going to be like today" and "what is the weather going to be like tomorrow".

[0083] The machine learning model 446 can be or include one or more various different types of machine learning models. In some examples, the machine learning model 446 can include one or more other forms of artificial neural networks, such as, for example, deep Boltzmann machines; deep belief networks; stacked autoencoders; and so on. Any neural network described herein can be combined (e.g., stacked) to form a more complex network. In some examples, the machine learning model 446 can perform or be subject to one or more reinforcement learning techniques, such as Markov decision processes; dynamic programming; Q - functions or Q - learning; value function methods; deep Q - networks; differentiable neural computers; asynchronous advantage actor - critic; deterministic policy gradients; and so on.

[0084] In some examples, centralized training techniques can be used to train the machine learning model 446 (e.g., based on a centrally stored dataset). This process can be repeated over time. Thus, the machine learning model 446 can become increasingly accurate and can be pushed to all client devices on which the same auxiliary application is installed. In other examples, decentralized training techniques, such as distributed training, federated learning, etc., can be used to train, update, or personalize the machine learning model 446. Decentralized training allows the machine learning model 446 to be improved based on real - world usage in a privacy - friendly manner.

[0085] The query generator 452 can provide the query generated by the machine learning model 446 to the query answer generator 454 to generate a query answer and context parameters associated with the query. The query answer generator 454 can also provide the query, the query answer, and the context parameters associated with the query to the framework 456. The framework 456 can receive the query, the query answer, and the context parameters associated with the query and store them in the query answer cache 444. For example, the framework 456 can encode the query answer and the context parameters associated with the query into a known format and store the encoded query answer and context parameters in the query answer cache 444.

[0086] Although described with respect to the remote computing system 440, the mobile device 120, the vehicle computing system 112, the mobile device 220, or the AHU 312 can actively predict queries and actively extract query answers for those predicted queries. For example, the vehicle computing system 112 or the mobile device 120 can predict the queries and query answers that the user 150 is predicted to ask based on in - vehicle data and store the predicted queries and query answers in a local query answer cache. In this way, aspects of the technology can enable the vehicle computing system 112 or the mobile device 120 to use the local query answer cache to retrieve query answers, thereby enhancing traditional offline query support.

[0087] Figure 5is a flowchart showing an exemplary operation of a computing device configured according to one or more aspects of the present disclosure. The computing device can be Figures 1-3 any of the computing devices shown. For illustrative purposes only, Figure 5 the operation of Figure 3 is described with reference to the automotive head unit (AHU) 312 shown.

[0088] As Figure 5 shown, the AHU 312 can receive user input data (e.g., via the UI 304, the display 314, and / or the microphone 316) (502). For example, the AHU 312 can receive a haptic input including "what is the weather" via the display 314 and can generate a text query "what is the weather" based on the received haptic input. The AHU 312 can determine at least one text query at least in part based on the user input data (5*4). For example, the AHU 312 can receive an audio input including "what is the weather" via the microphone 316 and can use speech-to-text conversion technology to generate a text query "what is the weather" from the audio data.

[0089] In response to determining at least one text query, the AHU 312 can generate one or more context parameters associated with the at least one text query (506). The one or more context parameters include at least one of a current time, a current location, a current user of the computing device, a set of historical user queries, a set of historical user behaviors, and a predicted destination. In the example where the AUH 312 generates the text query "what is the weather", the AUH 312 can generate the current time and the current location of the AUH 312 as context parameters.

[0090] Based on at least one text query and one or more context parameters, the AHU 312 can query the query answer cache 352 to generate a query result (508). The query answer cache 352 can include queries and query answers retrieved from the query answer cache 344 of the server system 340, enabling the AHU 312 to respond to detected queries even when the AHU 312 does not have an Internet connection. In some examples, the queries retrieved from the query answer cache 344 can include one or more global queries that are most likely to be searched by the aggregated population. In some examples, the queries retrieved from the query answer cache 344 can include one or more local queries searched by the user of the AHU 312. In some examples, the queries retrieved from the query answer cache 344 can include one or more predicted queries that may be searched by the user of the AHU 312. For example, a prediction model can be used to generate one or more predicted queries, and the prediction model is trained using local query data collected from the AHU 312.

[0091] Based on the query result, the AHU 312 can determine whether the query answer cache 352 includes a query answer for at least one text query and one or more context parameters (510). In response to determining that the query answer cache 352 does not include a query answer (the "no" branch of 510), the AHU 312 can send (512) an indication of at least one text query and an indication of one or more context parameters to the server system 340 to retrieve a query answer. The AHU 312 can receive an indication of the query answer from the server system 340 (514). The AHU 312 can also output (e.g., via the display 314 and / or the microphone 316) an indication of the query answer (516). For example, the AHU 312 can use the rendering module 356 to render the query answer to allow the query answer to be scaled and encoded into a known format. As another example, the AHU 312 can use the rendering module 356 to generate a graphical view of the query answer.

[0092] In response to determining that the query answer cache 352 includes a query answer (the "yes" branch of 510), the AHU 312 can output an indication of the query answer. In some examples, the AHU 312 can output an indication of the query answer via the display 314. In other examples, the AHU 312 can output an indication of the query answer via the microphone 316.

[0093] Figure 6 is a flowchart showing an exemplary operation of a server system configured according to one or more aspects of the present disclosure. According to one or more aspects of the present disclosure, the server system can be any server system such as Figure 1 and 3 shown or Figure 4Any of the remote computing systems shown. For illustrative purposes only, Figure 5 the operation of Figure 3 the remote computing system 440 shown is described.

[0094] As Figure 6 shown, a query generator 452 of the remote computing system 440 can generate a query (602) using a machine learning model 446. The machine learning model 446 can receive one or more inputs, such as search queries collected from multiple computing devices, search queries collected from the client device 412, and context parameters associated with the search queries collected from the client device 412. Based on the one or more inputs, the machine learning model 446 can generate one or more queries, such as one or more global queries, one or more local queries, and one or more predictive queries. In some examples, the machine learning model 446 can generate one or more global queries that are most likely to be searched by the aggregated group based on the search queries collected from multiple computing devices. In some examples, the machine learning model 446 can generate one or more local queries that are searched by the user of the client device 412 based on the search queries collected from the client device 412. In some examples, the machine learning model 446 can also generate one or more predictive queries that may be searched by the user of the client device 412 based on the search queries collected from the client device 412.

[0095] A query answer generator 454 of the remote computing system 440 can generate a query answer and context parameters (604) associated with the one or more queries based on the one or more queries. For example, the query answer generator 454 can generate one or more query answers via an Internet search.

[0096] A framework 456 of the remote computing system 440 can also store the one or more queries, query answers, and context parameters associated with the one or more queries in a query answer cache 444 (606). For example, the framework 456 can encode the queries, query answers, and context parameters associated with the queries into a known format and store the encoded queries, query answers, and context parameters in the query answer cache 444.

[0097] In one or more examples, the described functionality may be implemented in hardware, hardware and software, hardware and firmware, or any combination thereof. If implemented in software, the functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium or media corresponding to tangible media such as data storage media, or media including any medium that facilitates transfer of a computer program from one place to another, such as according to a communication protocol. In this manner, the computer-readable medium generally may correspond to: (1) a tangible computer-readable storage medium which is non-transitory; or (2) a communication medium such as a signal or carrier. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0098] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and media and data storage media do not include connections, carriers, signals, or other transient media, but rather refer to non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks generally reproduce data magnetically, while discs use lasers to optically reproduce data. Combinations of the above should also be included within the scope of computer-readable media.

[0099] The instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, as used herein, the term "processor" may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules. Moreover, the techniques may be fully implemented in one or more circuits or logic elements.

[0100] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or IC sets (e.g., chip sets). Various components, modules, or units are described in this disclosure to emphasize functional aspects configured to perform the disclosed techniques, but need not be implemented by distinct hardware units. Rather, as described above, the various units may be combined in a hardware unit or provided by a collection of interoperating hardware units including one or more of the processors described above in conjunction with suitable software and / or firmware.

[0101] Various embodiments have been described. These and other embodiments are within the scope of the appended claims.

Claims

1. A method for processing queries, comprising: Executing an auxiliary application by a computing device, wherein the auxiliary application generates one or more current context parameters and queries a query answer cache of the computing device; Predicting, by the computing device, future usage of the auxiliary application, wherein the future usage includes predicting one or more queries to be received by the auxiliary application and predicting a time range in which the auxiliary application will receive at least one of the one or more queries; Predicting, by the computing device, a network connection state of the computing device based on the predicted future usage of the auxiliary application; Determining, by the computing device, based on the time range, whether the predicted network connection state indicates that the computing device has a network connection; In response to determining that the predicted network connection state indicates that the computing device does not have a network connection within the time range: Predicting, by the computing device, a plurality of predicted queries to be received by the auxiliary application within the time range, wherein each predicted query from the plurality of predicted queries is associated with a corresponding set of context parameters, and wherein the corresponding set of context parameters includes at least one of a historical set of user queries of a user of the computing device, a historical set of user behaviors of the user of the computing device, and a first predicted destination; Pre-extracting, by the computing device, at least one predicted query answer based on the plurality of predicted queries; and Storing, in the query answer cache of the computing device, the at least one predicted query answer, an indication of the predicted query associated with the at least one predicted query answer, and an indication of the corresponding set of context parameters associated with the predicted query associated with the at least one predicted query answer; Receiving, by the computing device, user input; Determining, by the computing device and at least partially based on the user input, at least one text query; In response to determining the at least one text query, generating, by the computing device, one or more current context parameters associated with the at least one text query, wherein the one or more current context parameters include one or more of a current time, a current location, a current user of the computing device, and a second predicted destination; Querying, by the computing device and based on the at least one text query and the one or more current context parameters, the query answer cache of the computing device to generate a query result; Determining, based on the query result, whether the query answer cache includes a query answer to the at least one text query and the one or more current context parameters, wherein the query answer cache of the computing device includes the query answer to the at least one text query when the at least one text query and the one or more current context parameters match a predicted query from the plurality of predicted queries and the corresponding set of context parameters associated with the predicted query from the plurality of predicted queries; In response to determining that the query answer cache includes the query answer, determining that the query result is the query answer; Output an indication of the query response via the computing device.

2. The method according to claim 1, wherein Receiving the user input further includes: Receiving oral audio data from a head unit of a vehicle, wherein the oral audio data is detected via a microphone of the vehicle.

3. The method according to claim 2, wherein The computing device is a mobile computing device connected to the head unit of the vehicle, and wherein the oral audio data is sent from the head unit to the mobile computing device.

4. The method according to claim 1, wherein Receiving the user input further includes: Receiving a tactile input from a head unit of a vehicle, wherein the tactile input is detected via a touch-sensitive screen of the vehicle.

5. The method according to claim 1, further comprising: Periodically updating the query response cache of the computing device by the computing device and based on the predicted future use of the auxiliary application.

6. The method according to claim 1, further comprising: In response to determining that the query response cache does not include the query response: Sending an indication of the at least one text query and an indication of the one or more context parameters from the computing device to a remote computing system; And Receiving an indication of the query response from the remote computing system, wherein the query response is retrieved from a query response cache of the remote computing system, wherein the query response cache of the remote computing system includes a plurality of predicted query responses for the computing device, and wherein each predicted query response in the plurality of predicted query responses is associated with a corresponding text query and a corresponding set of context parameters.

7. The method according to claim 6, wherein, The query response retrieved from the query response cache of the remote computing system includes one or more global query responses generated using global query data, wherein the global query data is collected from a plurality of computing devices.

8. The method according to claim 6, wherein The query response retrieved from the query response cache of the remote computing system includes one or more local query responses generated using local query data, wherein the local query data is collected from the computing device.

9. The method according to claim 6, wherein The query response retrieved from the query response cache of the remote computing system includes one or more predicted query responses generated using a prediction model, wherein the prediction model is trained using local query data collected from the computing device.

10. A computing device for processing queries, comprising: At least one processor; And At least one computer-readable storage device storing instructions that, when executed by the at least one processor, cause the at least one processor to: Execute an auxiliary application, wherein the auxiliary application generates one or more context parameters and queries a query response cache of the computing device; Predict a future use of the auxiliary application, wherein the future use includes predicting one or more queries to be received by the auxiliary application and predicting a time range in which the auxiliary application will receive at least one of the one or more queries; Predict a network connection state of the computing device based on the predicted future use of the auxiliary application; Determine whether the predicted network connection state indicates that the computing device has a network connection based on the time range In response to determining that the predicted network connection status indicates that the computing device does not have a network connection within the time range: Predict, by the computing device, a plurality of predicted queries to be received by the auxiliary application within the time range, wherein each predicted query from the plurality of predicted queries is associated with a corresponding set of scenario parameters, and wherein the corresponding set of scenario parameters includes at least one of a set of historical user queries for a user of the computing device, a set of historical user behaviors of the user of the computing device, and a first predicted destination; Pre-extract, by the computing device, at least one predicted query answer based on the plurality of predicted queries; and Store, in the query answer cache of the computing device, the at least one predicted query answer, an indication of the predicted query associated with the at least one predicted query answer, and an indication of the corresponding set of scenario parameters associated with the predicted query associated with the at least one predicted query answer; Receive user input; Determine at least one text query based at least in part on the user input; In response to determining the at least one text query, generate one or more current scenario parameters associated with the at least one text query, wherein the one or more current scenario parameters include one or more of a current time, a current location, a current user of the computing device, and a second predicted destination; Query the query answer cache of the computing device to generate a query result based on the at least one text query and the one or more current scenario parameters; Determine, based on the query result, whether the query answer cache includes a query answer for the at least one text query and the one or more current scenario parameters, wherein the query answer cache of the computing device includes the query answer for the at least one text query when the at least one text query and the one or more current scenario parameters match a predicted query from the plurality of predicted queries and the corresponding set of scenario parameters associated with the predicted query from the plurality of predicted queries; In response to determining that the query answer cache includes the query answer, determine that the query result is the query answer; In response to determining that the query answer cache does not include the query answer: Send an indication of the at least one text query and an indication of the one or more scenario parameters to a remote computing system; and Receive an indication of the query answer from the remote computing system, wherein the query answer is retrieved from a query answer cache of the remote computing system, and wherein the query answer cache of the remote computing system includes a plurality of predicted query answers for the computing device, and wherein each predicted query answer from the plurality of predicted query answers is associated with a corresponding text query and a corresponding set of scenario parameters; and Output the indication of the query answer.

11. The computing device according to claim 10, wherein, The query answer retrieved from the query answer cache of the remote computing system includes one or more global query answers generated using global query data, wherein the global query data is collected from a plurality of computing devices.

12. The computing device according to claim 10, wherein, The query responses retrieved from the query response cache of the remote computing system include one or more local query responses generated using local query data, where the local query data is collected from the computing device.

13. The computing device according to claim 10, wherein, The query responses retrieved from the query response cache of the remote computing system include one or more predicted query responses generated using a prediction model, where the prediction model is trained using local query data collected from the computing device.

14. The computing device according to claim 10, wherein, The user input includes verbal audio data, and the instructions stored on the at least one computer-readable storage device further cause the at least one processor to: Receive the verbal audio data from a head unit of a vehicle, where the verbal audio data is detected via a microphone of the vehicle.

15. The computing device according to claim 14, wherein, The computing device includes a mobile computing device connected to the head unit of the vehicle, and wherein the verbal audio data is sent from the head unit to the mobile computing device.

16. The computing device according to claim 10, wherein, The user input includes tactile input, and the instructions stored on the at least one computer-readable storage device further cause the at least one processor to: Receive tactile input from a head unit of a vehicle, where the tactile input is detected via a touch-sensitive screen of the vehicle.

17. The computing device according to claim 10, wherein, The instructions stored on the at least one computer-readable storage device further cause the at least one processor to: Periodically update the query response cache of the computing device based on the predicted future use of the auxiliary application.

18. A computer-readable storage device storing instructions that, when executed, cause at least one processor of a computing device to perform the following operations: Execute an auxiliary application, where the auxiliary application generates one or more context parameters and queries the query response cache of the computing device; Predict the future use of the auxiliary application, where the future use includes predicting one or more queries to be received by the auxiliary application and predicting a time range in which the auxiliary application will receive at least one of the one or more queries; Predict the network connection state of the computing device based on the predicted future use of the auxiliary application; Determine, based on the time range, whether the predicted network connection state indicates that the computing device has a network connection; In response to determining that the predicted network connection state indicates that the computing device does not have a network connection within the time range: Predict a plurality of predicted queries to be received by the auxiliary application within the time range, where each predicted query from the plurality of predicted queries is associated with a corresponding set of context parameters, where the corresponding set of context parameters includes at least one of a set of historical user queries for a user of the computing device, a set of historical user behaviors of the user of the computing device, and a first predicted destination; Pre-extract at least one predicted query response based on the plurality of predicted queries; and Store the at least one predicted query answer, an indication of the predicted query associated with the at least one predicted query answer, and an indication of the corresponding set of context parameters associated with the predicted query associated with the at least one predicted query answer in the query answer cache of the computing device; Receive user input; Determine at least one text query at least in part based on the user input; In response to determining the at least one text query, generate one or more current context parameters associated with the at least one text query, wherein the one or more current context parameters include one or more of a current time, a current location, a current user of the computing device, and a second predicted destination; Query the query answer cache of the computing device based on the at least one text query and the one or more current context parameters to generate a query result; Determine whether the query answer cache includes a query answer for the at least one text query and the one or more current context parameters based on the query result, wherein the query answer cache of the computing device includes the query answer for the at least one text query when the at least one text query and the one or more current context parameters match a predicted query from the plurality of predicted queries and the corresponding set of context parameters associated with the predicted query from the plurality of predicted queries; In response to determining that the query answer cache includes the query answer, determine that the query result is the query answer; In response to determining that the query answer cache does not include the query answer: Send an indication of the at least one text query and an indication of the one or more context parameters to a remote computing system; and Receive an indication of the query answer from the remote computing system, wherein the query answer is retrieved from the query answer cache of the remote computing system, wherein the query answer cache of the remote computing system includes a plurality of predicted query answers for the computing device, and wherein each predicted query answer in the plurality of predicted query answers is associated with a corresponding text query and a corresponding set of context parameters; and Output the indication of the query answer.

Images