Condition preparation for automatic assistant input from user in vehicle

Abstract

The invention relates to condition preparation for automatic assistant input from a user in a vehicle. Implementations set forth herein relate to preemptively initializing an automatic assistant in a vehicle according to certain indications in order to reduce latency while also seeking to preserve computing resources. In some implementations, data for implementing one or more features of an automated assistant can be loaded into memory of a computing device based on vehicle interaction data. For example, vehicle interaction data can characterize that a user has invoked an instance of an automatic assistant from within their vehicle for a threshold period of time during which an application completes an operation. Based on the vehicle interaction data, subsequent instances of completing operations while the user is in the vehicle can cause the data to be loaded into memory to preemptively prepare the automated assistant to be utilized by the user.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 202210383133.6, with the filing date of April 13, 2022. Background Art

[0003] Humans may participate in human-computer conversations using interactive software applications referred to herein as "automated assistants" (also referred to as "digital agents," "chatbots," "interactive personal assistants," "intelligent personal assistants," "conversational agents," etc.). For example, humans (who may be referred to as "users" when they interact with automated assistants) may provide commands and / or requests using spoken natural language input (i.e., utterances) that may in some cases be converted to text and then processed and / or by providing textual (e.g., typed) natural language input.

[0004] In some instances, a user may access an automated assistant via an interface of a computing device located within a vehicle to cause the automated assistant to perform certain operations. However, due to the variability of network connections in a moving vehicle and / or due to constrained processing capabilities at the vehicle computing device, the automated assistant may exhibit latency in response to the user invoking the automated assistant from the vehicle computing device.

[0005] For example, constraints on processing power at the vehicle computing device can prevent the automated assistant from being ready to receive spoken commands at any time. For example, some or all of the processes of the local automated assistant may not be preemptively loaded in the memory before the initial use of the automated assistant and / or can be cleared from the memory and / or free resources for other processes after a period of non-use. This may cause delays in processing various user commands for the automated assistant at least due to the delays when those processes are loaded into the memory. As another example, the local automated assistant may not be continuously connected to the remote assistant server. This may be the result of efforts to save network resources and / or due to changes in network availability and / or conditions. Therefore, when the assistant server is needed to distinguish user commands for the automated assistant and a connection is not established with the assistant server, a delay may be generated while the connection is established. In some instances, because a portion of the user command is not captured due to the delay when the process is loaded into the memory and / or the delay when establishing communication with the assistant server, the delay experienced may cause the automated assistant to not understand the user command. In response to the user reissuing certain commands that were not initially understood, computing resources may then be wasted. For example, network bandwidth and memory can be consumed in instances where the vehicle computing device processes commands re-issued by the user. Additionally, user safety may be at risk when the user re-issues assistant commands while driving and / or riding in the vehicle. Summary of the invention

[0006] Implementations described herein involve preemptively loading one or more features of an automated assistant and / or preemptively establishing a network connection with an assistant server. Preemptively loading features and / or preemptively establishing a network connection can reduce latency that may occur when a user invokes their automated assistant from within a vehicle and / or can reduce latency in processing a corresponding command from the user pursuant to the invocation of the automated assistant.

[0007] Many implementations involve only selectively preemptively loading features and / or establishing network connections in order to achieve a reduction in latency, while also preventing unnecessary excessive occurrences of preemptively loading features and / or establishing network connections. Excessive occurrences of preemptively loading features may cause unnecessary utilization of local resources and / or may cause other non-automatic assistant active processes to be cleared from memory, thereby increasing latency associated with those other processes. Excessive occurrences of preemptively establishing network connections may cause excessive utilization of network resources and / or may constrain the bandwidth that can be utilized by other non-automatic assistant active processes.

[0008] Therefore, various implementations disclosed herein seek to balance the desire to achieve a reduction in latency with the computational disadvantages and / or network disadvantages that may be generated by preemptively performing actions to achieve a reduction in latency. As described herein, some of those various implementations utilize interaction data to enable a distinction between situations where preemptively performing actions to reduce latency is guaranteed (e.g., those situations where the corresponding user is sufficiently likely to utilize the automated assistant) and alternative situations where preemptively performing actions to reduce latency is not guaranteed (e.g., those situations where the corresponding user is not sufficiently likely to utilize the automated assistant and preemptively performing actions will unnecessarily consume computing resources and / or network resources). The interaction data can be based on the corresponding user's past automated assistant interactions and / or another user's past automated assistant interactions. As described herein, such interaction data can optionally reflect whether and / or to what extent an automated assistant interaction has occurred in response to one or more specific categories of events (e.g., incoming messages, ongoing call stops) and / or contexts related to those events (e.g., the navigation stage when the event occurred, the source of the incoming message, the date or time of the ongoing call stop, etc.).

[0009] In order to maintain the responsiveness of the automated assistant while in the vehicle, various characteristics of the context in which the user initializes the automated assistant from within the vehicle can be captured as interaction data that can be processed. The interaction data can be processed to determine when to preemptively initialize certain operations associated with the automated assistant. For example, data that the automated assistant relies on can be preemptively retrieved and / or loaded into the memory of the vehicle computing device from a remote server device. For example, a local automatic speech recognition (ASR) process utilized by the automated assistant can be preemptively loaded into the memory. In these and other ways, any automated assistant features that rely on the data can be accessed by the user in the vehicle without delay—even when the vehicle is traveling in an area with limited network connectivity and / or the vehicle computing device is engaged in other processes.

[0010] As an example, the interaction data associated with the user can characterize an instance in which the user listens to a corresponding podcast while in a vehicle driving to a corresponding destination. In some instances, when the corresponding podcast is completed outside the threshold amount of time to reach the corresponding destination, the user may have called their automatic assistant from the vehicle computing device of the vehicle. In other instances, when the corresponding podcast is completed within the threshold amount of time of the estimated time to reach the destination or other media stops rendering certain information at the interface, the user may have avoided calling their automatic assistant from the vehicle computing device of the vehicle. Based on this interaction data, operations can be performed in subsequent instances to prepare for the user to call the automatic assistant when the podcast ends and / or stops rendering other media during driving / riding. For example, and based on the interaction data, when the podcast ends outside the threshold amount of time for the vehicle to reach the destination, the user can usually request the automatic assistant to play the "driving playlist". Therefore, one or more operations that are initialized can include retrieving ASR-related data for responding to spoken utterances from the user and media data for buffering a certain amount of "driving playlist". However, when the podcast is estimated to end within a threshold amount of time of the user arriving at the destination, one or more operations can be bypassed and / or one or more other operations can be initiated (eg, to prepare for the user's arrival at the destination).

[0011] In some implementations, interaction data can be generated to characterize instances in which a user receives a corresponding electronic communication from another person while in their vehicle and then initializes their automated assistant to perform one or more operations. For example, in some instances, a user may have requested that the automated assistant perform a specific action (e.g., set a reminder) within a threshold amount of time of receiving an electronic communication having one or more characteristics and / or criteria. The one or more criteria and / or characteristics may be, but are not limited to, whether the communication is from a stored contact, a relative, someone in a particular location, and / or a coworker. Alternatively or additionally, the one or more characteristics may be, but are not limited to, the time of receipt, the time of sending, certain content of the electronic communication (e.g., text, video, image, URL link, etc.), the length of the electronic communication, the time of receipt relative to the arrival time of the vehicle, and / or any other characteristics of the one or more electronic communications.

[0012] The interaction data can then be processed to facilitate determining whether to initialize one or more specific operations so that the automated assistant is ready to receive input from the user within a threshold amount of time that the user receives electronic communications while in their vehicle. For example, the interaction data can indicate that the user typically invokes their automated assistant to set a reminder when the user receives a phone call from a certain subset of their stored contacts (e.g., coworkers and / or contacts with a certain domain in their corresponding email addresses). Thereafter, when the user is in their vehicle and receives another phone call from a contact of the certain subset of contacts, one or more operations can be initialized. In some implementations, the one or more operations can include loading data corresponding to a calendar application and the automated assistant into a memory of the vehicle computing device. In this way, a request to set a reminder via the automated assistant can be easily processed when the user is participating in a phone call and / or within a threshold amount of time of the end of a phone call.

[0013] In some implementations, interaction data accessible to the automated assistant can characterize an instance in which a user invokes the automated assistant after modifying a destination of a navigation application. Alternatively or additionally, the destination can be modified when the vehicle is on a route to an initial destination, but the user receives an electronic communication or other notification associated with the route. Thereafter, the interaction data can be processed to determine when to prepare the vehicle computing device to receive input to the automated assistant. For example, the vehicle can provide access to a navigation application that can be operated while the vehicle is driving toward a destination. The user can interact with the navigation application and / or another application to modify the destination of the vehicle. Based on the interaction data associated with the user, the vehicle computing device and / or another computing device can cause one or more operations to be initialized to prepare for the user to provide a request to the automated assistant within a threshold amount of time to modify the destination.

[0014] For example, the interaction data can indicate that the user typically requests the automated assistant to read store reviews to the user within a threshold amount of time of modifying the vehicle's destination so that the vehicle drives to the corresponding store. Thus, when the user modifies the vehicle's destination, one or more operations can be initialized to retrieve media data associated with store reviews (e.g., reviews of stores to which the vehicle has recently been requested to drive) and / or establish a secure connection with the automated assistant server. In some implementations, authentication data can be communicated from a portable computing device (e.g., a user's cellular phone) and / or a vehicle computing device to a server device to establish a secure connection. In this way, a secure connection for transmitting input data can be established before the user provides input to the automated assistant. This can reduce delays that may be exhibited when a user invokes the automated assistant shortly after performing an operation in another application.

[0015] The above description is provided as an overview of some implementations of the present disclosure. Further descriptions of those implementations and other implementations are described in more detail below.

[0016] Other implementations may include a non-transitory computer-readable storage medium storing instructions executable by one or more processors (e.g., a central processing unit (CPU), a graphics processing unit (GPU), and / or a tensor processing unit (TPU)) to perform a method such as one or more of the methods described above and / or elsewhere herein. Other implementations may include a system of one or more computers including one or more processors operable to execute stored instructions to perform a method such as one or more of the methods described above and / or elsewhere herein.

[0017] It should be understood that all combinations of the foregoing concepts and the additional concepts described in more detail herein are contemplated as being part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as being part of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A , Figure 1B and Figure 1C Illustrated is a view of an automated assistant that preemptively initiates certain operations to reduce latency and / or improve safety when a user invokes the automated assistant from their vehicle.

[0019] Figure 2 Illustrated is a system for providing an automated assistant capable of proactively preparing to perform actions and / or provide responses to a user based on various contextual characteristics associated with a vehicle.

[0020] Figure 3Illustrated is a method for preparing an automated assistant to receive input when certain contextual features are identified while a user is in a vehicle.

[0021] Figure 4 Illustrated is a method for preparing an automated assistant to receive input while a user is in a vehicle depending on whether the user receives an electronic communication having one or more characteristics.

[0022] Figure 5 Illustrated is a method for preparing an automated assistant to receive one or more inputs while a user is in a vehicle and a destination for the vehicle has been modified.

[0023] Figure 6 is a block diagram of an example computer system. DETAILED DESCRIPTION

[0024] Figure 1A , Figure 1B and Figure 1C The views 100, 120, and 140 of the automated assistant 110 are illustrated to preemptively initiate certain operations to reduce latency and improve safety when the user 102 invokes the automated assistant 110 from the vehicle 112. For example, the user 102 may be riding in the vehicle 112 driving toward a destination and can request the automated assistant 110 to perform a specific action. The user 102 can provide spoken utterances 108 such as "Assistant, change my destination to 'Ear-XY-Z'" to change the destination that the vehicle 112 is driving toward. The spoken utterances 108 can be received at the vehicle computing device 104 and / or a separate computing device that provides access to the automated assistant 110. For example, the vehicle computing device 104 can be integrated into the vehicle and can be connected to one or more other computing devices that the user 102 carries in and out of the vehicle 112. The vehicle computing device can include a display interface 114 that can render an application interface 116 that can correspond to an application 106 that can be controlled via the automated assistant 110. For example, the spoken utterance 108 can be processed by the automated assistant 110, which can cause one or more operations at the application 106 to undergo a state change. In some implementations, the state change of one or more operations of the application 106 can occur without explicit user input to undergo a state change. For example, when playback of an audiobook being rendered by an audiobook application of the vehicle computing device 104 ends, the end of the audiobook can correspond to a state change of the media playback operation at the audiobook application. Alternatively or additionally, the state change of the operation of the application can occur with or without corresponding user input.

[0025] Automated assistant 110 and / or another application 106 or operating system of computing device 104 can access historical interaction data 122 to identify prior instances in which user 102 modified a destination to which vehicle 112 was traveling. Identifying such instances can allow automated assistant 110 to prepare for certain actions that user 102 may request automated assistant 110 to perform. For example, historical interaction data 122 can indicate that user 102 previously modified the destination of vehicle 112 and subsequently invoked automated assistant 110 to initiate playback of an audiobook. Based on this indication, automated assistant 110 can perform certain operations to prepare for specific assistant input after the user modified the destination of vehicle 112.

[0026] For example, context data 124 can indicate that user 102 provided spoken utterance 108 to cause a destination change for vehicle 112. Context data 124 and historical interaction data 122 can be processed to determine that user 102 is likely to issue a command to automated assistant 110 to request playback of an audiobook. In order to reduce latency and conserve computing resources at computing device 104, operational data 126 can be generated to identify one or more operations that can be performed to prepare for the command from user 102 and playback at the audiobook. For example, operational data 126 can be generated at computing device 104 and / or separate computing device 128 and can be available at computing device 104 before user 102 requests playback of the audiobook. In some implementations, operational data 126 can include audio data corresponding to the audiobook, search results corresponding to a preemptively performed search, one or more web links, and / or any other data that can be used by an application to provide content and / or perform an action. In this way, automated assistant 110 can, in certain contexts, exhibit less latency in responding to user input provided within the vehicle.

[0027] For example, user 102 can provide spoken utterance 142 within a threshold amount of time to modify the destination of vehicle 112. In some implementations, the threshold amount of time can be based on interaction data 122 and / or prior instances of user 102 interacting with automated assistant 110 within a particular amount of time of a particular event (e.g., an application state change). Thus, when the threshold amount of time has passed after the particular event, memory and / or bandwidth saved for a request to the automated assistant can be utilized by another application or device.

[0028] The spoken utterance 142 may be, for example, “Assistant, continue the last audiobook I was listening to.” In response, the automated assistant can provide output 144 (e.g., “Okay.”) that acknowledges the input from the user and then renders a playback of the audiobook. The playback of the audiobook can incorporate data generated based on the operational data 126 and / or other operations performed using the operational data 126. For example, the automated assistant can initialize an audiobook application and buffer a portion of the audiobook into memory. Alternatively or additionally, features of the automated assistant can be initialized based on the operational data 126. Such features can include automatic speech recognition (ASR), which can correspond to functions and / or libraries that can be loaded into memory before the user provides the spoken utterance 142.

[0029] Figure 2The diagram shows a system 200 for providing an automated assistant 204 that can proactively prepare to perform actions and / or provide responses to a user based on various contextual characteristics associated with a vehicle. The automated assistant 204 can operate as part of an assistant application provided at one or more computing devices such as a computing device 202 and / or a server device. The user can interact with the automated assistant 204 via an assistant interface 220, which can be a microphone, a camera, a touch screen display, a user interface, and / or any other device that can provide an interface between the user and the application. For example, the user can initialize the automated assistant 204 by providing verbal input, text input, and / or graphical input to the assistant interface 220 so that the automated assistant 204 initializes one or more actions (e.g., providing data, controlling peripheral devices, accessing an agent, generating input and / or output, etc.). Alternatively, the automated assistant 204 can be initialized based on processing of vehicle data and / or contextual data 236 using one or more trained machine learning models. The context data 236 and / or vehicle data can characterize one or more features of an environment accessible to the automated assistant 204, with prior permission from the user, and / or one or more features of a user who can interact with the automated assistant 204. The computing device 202 can include a display device, which can be a display panel, including a touch interface for receiving touch input and / or gestures to allow a user to control an application 234 of the computing device 202 via the touch interface. In some implementations, the computing device 202 may lack a display device, thereby providing an audible user interface output without providing a graphical user interface output. In addition, the computing device 202 can provide a user interface, such as a microphone, for receiving spoken natural language input from a user. In some implementations, the computing device 202 can include a touch interface and can lack a camera, but can optionally include one or more other sensors.

[0030] Computing device 202 and / or other third-party client devices can communicate with the server device over a network such as the Internet. Additionally, computing device 202 and any other computing devices can communicate with each other over a local area network (LAN) such as a Wi-Fi network. Computing device 202 can offload computing tasks to a server device to save computing resources at computing device 202. For example, a server device can host automated assistant 204 and / or computing device 202 can transmit input received at one or more assistant interfaces 220 to the server device. However, in some implementations, automated assistant 204 can be hosted at computing device 202, and various processes that can be associated with automated assistant operations can be performed at computing device 202.

[0031] In various implementations, all or less than all aspects of automated assistant 204 can be implemented on computing device 202. In some of those implementations, aspects of automated assistant 204 are implemented via computing device 202 and can interface with a server device, which can implement other aspects of automated assistant 204. The server device can optionally serve multiple users and their associated assistant applications via multiple threads. In implementations in which all or less than all aspects of automated assistant 204 are implemented via computing device 202, automated assistant 204 can be an application separate from the operating system of computing device 202 (e.g., installed "on top" of the operating system)—or can alternatively be implemented directly by the operating system of computing device 202 (e.g., considered an application of the operating system, but integral to the operating system).

[0032] In some implementations, automated assistant 204 can include an input processing engine 206 that can employ a number of different modules to process input and / or output of computing device 202 and / or server devices. For example, input processing engine 206 can include speech processing engine 208 that can process audio data received at assistant interface 220 to identify text embodied in the audio data. The audio data can be transmitted from, for example, computing device 202 to a server device in order to conserve computing resources at computing device 202. Additionally or alternatively, the audio data can be processed exclusively at computing device 202.

[0033] The process for converting audio data into text can include a speech recognition algorithm, which can use a neural network and / or a statistical model to identify an audio data group corresponding to a word or phrase. The text converted from the audio data can be parsed by a data parsing engine 210 and provided to an automated assistant 204 as text data, which can be used to generate and / or identify command phrases, intentions, actions, slot values, and / or any other content specified by a user. In some implementations, the output data provided by the data parsing engine 210 can be provided to a parameter engine 212 to determine whether the user has provided an input corresponding to a specific intention, action, and / or routine that can be performed by an automated assistant 204 and / or an application or an intelligent agent that can be accessed via an automated assistant 204. For example, assistant data 238 can be stored at a server device and / or a computing device 202, and can include data defining one or more actions that can be performed by an automated assistant 204, and parameters necessary for performing the action. Parameter engine 212 can generate one or more parameters for intention, action, and / or slot values, and provide the one or more parameters to an output generation engine 214. Output generation engine 214 can use the one or more parameters to communicate with assistant interface 220 to provide output to a user and / or to communicate with one or more applications 234 to provide output to one or more applications 234 .

[0034] In some implementations, the automated assistant 204 may be an application that can be installed “on top of” the operating system of the computing device 202 and / or that can itself form part (or all) of the operating system of the computing device 202. The automated assistant application includes and / or can access on-device speech recognition, on-device natural language understanding, and on-device fulfillment. For example, on-device speech recognition can be performed using an on-device speech recognition module that processes audio data (detected by a microphone) using an end-to-end speech recognition machine learning model stored locally at the computing device 202. On-device speech recognition generates recognized text for spoken utterances (if any) present in the audio data. Additionally, for example, on-device natural language understanding (NLU) can be performed using an on-device NLU module that processes the recognized text generated using on-device speech recognition and optionally contextual data to generate NLU data.

[0035] The NLU data can include an intent corresponding to the spoken utterance and optionally parameters for the intent (e.g., slot values). The on-device fulfillment can be performed using an on-device fulfillment module that utilizes the NLU data (from the on-device NLU) and optionally other local data to determine an action to be taken to resolve the intent (and optionally parameters for the intent) of the spoken utterance. This can include determining a local and / or remote response (e.g., answer) to the spoken utterance, an interaction with a locally installed application to be performed based on the spoken utterance, a command to be transmitted to an Internet of Things (IoT) device based on the spoken utterance (directly or via a corresponding remote system), and / or other resolved actions to be performed based on the spoken utterance. The on-device fulfillment can then initiate local and / or remote execution / running of the determined action to resolve the spoken utterance.

[0036] In various implementations, remote speech processing, remote NLU, and / or remote fulfillment can be at least selectively utilized. For example, the recognition text can be at least selectively transmitted to a remote automated assistant component for remote NLU and / or remote fulfillment. For example, the recognition text can be optionally transmitted for remote execution in parallel with execution on the device, or in response to a failure of the NLU on the device and / or the fulfillment on the device. However, speech processing on the device, NLU on the device, fulfillment on the device, and / or running on the device may be prioritized at least because of the reduced latency they provide when distinguishing spoken utterances (since no client-server round trip is required to distinguish spoken utterances). In addition, the on-device functionality may be the only functionality available in the absence of a network connection or with a limited network connection.

[0037] In some implementations, computing device 202 can include one or more applications 234 that can be provided by a third-party entity that is different from the entity that provided computing device 202 and / or automated assistant 204. Automated assistant 204 and / or an application state engine of computing device 202 can access application data 230 to determine one or more actions that can be performed by one or more applications 234, as well as a state of each of one or more applications 234 and / or a state of a corresponding device associated with computing device 202. Automated assistant 204 and / or a device state engine of computing device 202 can access device data 232 to determine one or more actions that can be performed by computing device 202 and / or one or more devices associated with computing device 202. In addition, application data 230 and / or any other data (e.g., device data 232) can be accessed by automated assistant 204 to generate context data 236 that can characterize the context in which a particular application 234 and / or device is running, and / or the context in which a particular user is accessing computing device 202, accessing application 234, and / or any other device or module.

[0038] When one or more applications 234 are running at computing device 202, device data 232 can characterize the current operating state of each application 234 running at computing device 202. In addition, application data 230 can characterize one or more characteristics of running applications 234, such as the content of one or more graphical user interfaces being rendered at the direction of one or more applications 234. Alternatively or additionally, application data 230 can characterize action modes that can be updated by the respective applications and / or by automated assistant 204 based on the current operating state of the respective applications. Alternatively or additionally, one or more action modes for one or more applications 234 can remain static, but can be accessed by the application state engine to determine an appropriate action to initiate via automated assistant 204.

[0039] The computing device 202 can also include an assistant invocation engine 222 that can use one or more trained machine learning models to process the application data 230, the device data 232, the contextual data 236, and / or any other data accessible to the computing device 202. The assistant invocation engine 222 can process this data to determine whether to wait for the user to explicitly speak an invocation phrase to invoke the automated assistant 204, or to consider the data to indicate an intent by the user to invoke the automated assistant—instead of requiring the user to explicitly speak an invocation phrase. For example, one or more trained machine learning models can be trained using instances of training data based on scenarios in which a user is in an environment in which multiple devices and / or applications are exhibiting various operating states. Instances of training data can be generated to capture training data that characterizes contexts in which a user invokes an automated assistant and other contexts in which a user does not invoke an automated assistant.

[0040] When one or more trained machine learning models are trained based on these instances of training data, assistant invocation engine 222 enables automated assistant 204 to detect or limit detection of spoken invocation phrases from a user based on characteristics of the context and / or environment. Additionally or alternatively, assistant invocation engine 222 enables automated assistant 204 to detect or limit detection of one or more assistant commands from a user based on characteristics of the context and / or environment. In some implementations, assistant invocation engine 222 can be disabled or limited based on computing device 202 detecting an assistant suppression output from another computing device. In this manner, automated assistant 204 will not be invoked based on context data 236 when computing device 202 is detecting an assistant suppression output—which would otherwise cause automated assistant 204 to be invoked without detecting an assistant suppression output.

[0041] The system 200 can include an interaction engine 216 that can generate and / or process interaction data characterizing an interaction between one or more users and one or more automated assistants. In some implementations, the interaction data can be processed together with other data available at the computing device 202 and / or a separate computing device. For example, the application data 230 and the assistant data 238 can be processed by the interaction engine 216 to generate interaction data that can be used to characterize the user's enabling the automated assistant 204 to control one or more instances of the application 234. Alternatively or additionally, the device data 232 and / or the context data 236 can be processed by the interaction engine 216 to generate the interaction engine 216, which, when generating the interaction data, characterizes the state of one or more operations of the device and / or the user with the user's prior permission. For example, interaction data can be generated to characterize whether the user is riding in their vehicle and accessing a specific application in an instance in which the user invokes their automated assistant 204. This interaction data can then be used to preemptively prepare certain features of the computing device 202 (e.g., a vehicle computing device) to receive user input for the automated assistant 204.

[0042] In instances when a resource utility engine 218 of system 200 determines that a lack of memory or other computing resources may exist during an upcoming assistant interaction that is predicted to occur, preemptive preparation for user input to automated assistant 204 can be performed. For example, resource utility engine 218 can process data accessible to computing device 202 and / or a separate device to determine that a route currently being traveled by a user includes an area with limited network connectivity. During such times, automated assistant 204 may have to operate in an "offline" mode as a result of limited network connectivity. In such instances, resource utility engine 218 can access interaction data generated by interaction engine 216. Resource utility engine 218 can use the interaction data to determine if there is a chance that a user will interact with automated assistant 204 when a vehicle is traveling through an area with limited network connectivity and / or when automated assistant 204 is predicted to be "offline." Operation prediction engine 226 can be initialized when there is a chance or probability that a user will access automated assistant 204 when computing device 202 is experiencing a lack of one or more computing resources (e.g., network bandwidth, processing bandwidth, memory, etc.).

[0043] The operation prediction engine 226 can interact with the interaction engine 216 and / or the resource utility engine 218 to identify one or more operations that should be performed to prepare the automated assistant to receive user input in certain situations. The one or more operations can be selected by the operation prediction engine 226 based on the details of one or more interactions that the user may have with the automated assistant 204. In some implementations, the one or more operations can be selected based on one or more computing resources that may be limited when the user is estimated to interact with the automated assistant. For example, when it is estimated that the user requests to replay a podcast, the resource utility engine 218 can determine that memory may be limited during a specific portion of the route the user is traveling.

[0044] Based on this determination, the operation prediction engine 226 can identify an operation of preemptively downloading a specific podcast to ensure that the podcast is stored in memory before any other process that may consume memory. Alternatively or additionally, the operation prediction engine 226 can identify another operation of preemptively searching for a suitable podcast to be presented to the user to ensure that the search can be performed over a network (e.g., a cellular network) while there is a suitable network connection. Alternatively or additionally, the operation prediction engine 226 can identify an operation of preemptively running a test query marked as a non-user query. In this way, a test query (e.g., searching for podcasts or store reviews) can be executed to test the connection with the assistant server without affecting the preferences and / or search history stored in association with the user. On the contrary, with prior permission from the user, a test query can be occasionally submitted to simulate the interaction between the automatic assistant and the user without rendering the test query and / or the result of the test query at the assistant interface 220 of the computing device 202. In some implementations, the automatic assistant 204 can perform online test queries and / or offline test queries to test the online features and / or offline features of the automatic assistant, respectively. This can help preemptively prepare automated assistant 204 to receive user input, as both offline and online resources of the vehicle computing device may be limited in certain situations while the user is riding in their vehicle.

[0045] In some implementations, the interaction engine 216 can employ one or more trained machine learning models to estimate whether one or more users will interact with their corresponding automated assistants in a particular context. For example, the interaction engine 216 can employ a first model for generally predicting interactions based on a collection of multiple different users, and a second model for specifically predicting interactions based on a single user. In some implementations, each of the one or more models can be trained using training data with prior permission from each user, the training data being generated using a training data engine 224. The training data engine 224 can generate training data characterizing instances when preemptive execution of one or more operations results in a more optimal allocation of computing resources. In other words, the training data engine 224 can be used to train one or more models based on whether preemptive execution of one or more operations is beneficial or not beneficial to subsequent assistant interactions.

[0046] Figure 3 A method 300 is illustrated for preparing an automated assistant to receive input when certain contextual features are identified while a user is in a vehicle. The method 300 can be performed by one or more devices, applications, and / or any other means or modules capable of interacting with the automated assistant. The method 300 can include an operation 302 of determining whether an operational state change has been detected while the user is in the vehicle. The operational state change can correspond to an operational change (e.g., a change in network connectivity, a change in device settings, a change in location, an operating system change, etc.) of a device and / or application associated with the user and / or the vehicle. For example, the operational state change can correspond to a change in playback of media, completion of a download or upload, receipt of an electronic communication, termination of an electronic communication, a change in network bandwidth, a change in battery status, and / or any other change (or combination of changes) in operation that can be associated with any application and / or device.

[0047] When an operational state change has occurred, method 300 can proceed from operation 302 to operation 304. Otherwise, operation 302 can be performed until an operational state change is detected. Operation 304 can include determining whether the user has previously interacted with an automated assistant in the vehicle within a threshold amount of time of a prior state change. For example, during a previous instance in which the user was riding in a vehicle, the user may have a history of requesting the automated assistant to perform an action within an amount of time of a state change of the application. The state change may be, but is not limited to, for example, the end of a phone call and / or the end of a podcast. In such instances, the user may have a history of calling the automated assistant when a state change did not occur within a threshold amount of time of reaching a destination in the vehicle. In other words, when a specific amount of time is left before the user arrives at the destination, the user typically calls their automated assistant to initialize one or more different actions. Otherwise, when less than a specific amount of time is left before the user arrives at the destination, the user may typically avoid calling their automated assistant before arriving at the destination.

[0048] When the prior interaction indicates that the user has interacted with their automated assistant within a threshold amount of time of the prior state change, the method 300 can continue to operation 306. Otherwise, the method 300 can return to operation 302. Operation 306 can include determining whether the operating state change has indeed not occurred within another threshold time period of reaching the destination. When it is determined that the operating state change has not occurred within another threshold time period of reaching the destination, the method 300 can proceed to operation 308. Otherwise, the method 300 can return to operation 302 and / or another operation. Operation 308 can include initiating one or more operations at the vehicle computing device or another computing device. The one or more operations can include, for example: a search process utilized by the automated assistant to retrieve request data; an ASR process for allowing the automated assistant to respond to speech; an authentication process for identifying the user's voice and / or creating a secure connection with an assistant server and / or other remote servers; a natural language understanding (NLU) process for determining how to respond to natural language input; a trial query process for "warming up" the automated assistant to receive subsequent input; and / or any other process that can be associated with the automated assistant.

[0049] From operation 308, method 300 can proceed to optional operation 310, which can include determining whether the automated assistant has received input. In some implementations, optional operation 310 can include determining whether the automated assistant has received input that is not within a threshold amount of time to reach the destination. When input to the automated assistant is received, method 300 can proceed from operation 310 to optional operation 312. Otherwise, method 300 can return to operation 302 and / or another operation of method 300. Optional operation 312 can include causing the automated assistant to perform an action corresponding to the input and one or more operations. For example, the input can be a request for items listed on the user's "shopping list" by the automated assistant. The one or more operations performed can include authenticating a connection between the vehicle computing device and the assistant server with prior permission from the user so that the automated assistant can access the user's "shopping list". Alternatively or additionally, one or more operations can include storing data representing the shopping list and / or audio data corresponding to the voice of the automated assistant describing the shopping list in a memory of the vehicle computing device. In this manner, less latency can be exhibited between a user providing input "Assistant, what's on my grocery list" and the automated assistant detailing what is on the grocery list "You have milk, eggs, and laundry detergent on your grocery list." Additionally, providing a smoother interaction between a user in a vehicle and an automated assistant can improve user safety.

[0050] Figure 4 A method 400 is illustrated for preparing an automated assistant to receive input while a user is in a vehicle depending on whether the user receives an electronic communication having one or more characteristics. Method 400 can be performed by one or more devices, applications, and / or any other means or modules capable of interacting with an automated assistant. Method 400 can include an operation 402 of determining whether an electronic communication has been received while the user is in a vehicle. The electronic communication can be, but is not limited to, data received from an application and / or device associated with another user. For example, the electronic communication can be a text message from another user who is operating a separate computing device (e.g., a cellular phone) or a phone call to the user. When it is determined that the user has received an electronic communication while the user is in the vehicle, method 400 can proceed from operation 402 to operation 404. Otherwise, operation 402 can be performed until the user has received an electronic communication while the user is in the vehicle.

[0051] Operation 404 can include determining whether the user has previously interacted with the automated assistant within a threshold amount of time of receiving an electronic communication. For example, the user may have previously invoked their automated assistant on multiple occasions within a threshold amount of time of receiving an electronic communication. In some implementations, the threshold amount of time can be identified from interaction data characterizing prior instances in which the user invoked the automated assistant. For example, the threshold amount of time can be an average of prior time periods between the user receiving the electronic communication and the user invoking the automated assistant. When it is determined that the user has previously interacted with the automated assistant in the vehicle within a threshold amount of time of receiving prior electronic communications, method 400 proceeds to operation 406. Otherwise, method 400 can return to operation 402.

[0052] Operation 406 can include, with prior permission from the user, identifying one or more characteristics of a previous communication associated with the user. For example, the one or more characteristics can include, but are not limited to, the content of the electronic communication, the originating location of the electronic communication, the receiving location of the electronic communication, the specific time of day of receipt and / or transmission, the length of the electronic communication, the author of the electronic communication, one or more other recipients of the electronic communication, a domain name in an address of the electronic communication, whether the author of the electronic communication corresponds to contact information stored in association with the user, and / or any other characteristics that can be associated with an electronic communication.

[0053] From operation 406, method 400 can proceed to operation 408, which can include determining whether the electronic communication received by the user has one or more characteristics. For example, the electronic communication can be received from another user whose contact information is stored in association with the user. This can be similar to a prior instance in which a user received an electronic communication from another user whose contact information is stored in association with the user and the user subsequently invoked an automated system. When it is determined that the electronic communication received by the user has one or more characteristics, method 400 can proceed from operation 408 to operation 410. Otherwise, method 400 can return to operation 402 or another operation.

[0054] In some implementations, electronic communications can be received from a coworker of the user, and the coworker's contact information can include a domain name identifying the user's employer. In a prior instance where a user received electronic communications from their employer while the user was in their vehicle, the user may have a habit of invoking their automated assistant to set reminders and / or create meeting events via their calendar application. Based on this habit of the user, the automated assistant and / or another application can perform one or more operations to prepare for any calendar-related input to the automated assistant and / or another application.

[0055] Operation 410 can include causing one or more operations to be initialized at the vehicle computing device or another computing device. The one or more operations can include initializing a calendar application to receive input for creating reminders and / or events. Alternatively or additionally, the one or more operations can include downloading data from and / or uploading data to an assistant server associated with an automated assistant accessible via the vehicle computing device of the vehicle. This data can enable a reduction in latency between a user providing input to the automated system and the automated assistant performing an action based on the input. Additionally, this can save processor bandwidth that might otherwise be consumed when performing one or more operations after the user provides input.

[0056] Method 400 can proceed from operation 410 to optional operation 412, which can include determining whether input to the automated assistant has been received. Alternatively or additionally, optional operation 412 can include determining whether input to the automated assistant has been received within a threshold amount of time that another user receives an electronic communication. For example, input to the automated assistant can correspond to a request for the automated assistant to set a reminder associated with an electronic communication (e.g., "Assistant, set areminder for a 2:30 PM meeting today."). When it is determined that the automated assistant has received input, method 400 can proceed to optional operation 414. Otherwise, method 400 can return to operation 402 and / or another operation. Operation 414 can include causing the automated assistant to perform an action associated with the input and one or more operations. For example, when one or more operations include loading data corresponding to the NLU process of the automated assistant, a request from a user can be more easily understood by the automated assistant based on data that has been loaded into memory before the user provides the request.

[0057] Figure 5 A method 500 is illustrated for preparing an automated assistant to receive one or more inputs while a user is in a vehicle and a destination of the vehicle has been modified. The method 500 can be performed by one or more applications, devices, and / or any other apparatus or module capable of interacting with the automated assistant. The method 500 can include an operation 502 of determining whether a navigation operation of a vehicle has been modified while the user is in the vehicle. The navigation operation can be performed by a vehicle, a vehicle computing device, and / or a navigation application associated with the user and / or the vehicle. For example, while the user is riding in the vehicle, the user can drive the vehicle according to directions being provided by the navigation application. Alternatively, while the user is riding in the vehicle, the vehicle can autonomously drive according to a navigation application accessible via the vehicle computing device. The navigation operation can be modified by input from the user to the automated assistant, the navigation application, and / or any other application or device associated with the vehicle.

[0058] When it is determined that the user has modified the navigation operation while the user is in the vehicle, method 500 can proceed from operation 502 to operation 504. Otherwise, operation 502 can be performed to determine whether the navigation operation has been modified while the user is in the vehicle. Operation 504 can include determining whether the user has previously interacted with the automated assistant in the vehicle within a threshold amount of time in which the previous navigation operation was modified. For example, the user may have previously modified the navigation operation of the vehicle or application, and then interacted with the automated assistant within a threshold amount of time in which the navigation operation was modified. In some instances, based on the change in the navigation operation, the user can recognize that additional time can be added to the route as a result of the change in the navigation operation. In order to occupy this additional time, the user can request the automated assistant to perform a search to identify news-related information about the new destination. For example, in some instances, the user can ask the automated assistant to find the business hours of the business located at the updated destination. Therefore, based on this habit of the user, when the navigation operation of the vehicle or application is modified, one or more operations can be performed to prepare for calling the automated assistant for safety and efficiency purposes.

[0059] When it is determined that the user has previously invoked the automated assistant within a threshold amount of time to modify the navigation operation, method 500 can proceed to operation 506. Otherwise, method 500 can return to operation 502 to determine whether the navigation operation has been modified. Operation 506 can include causing one or more operations to be initialized at the vehicle computing device or another computing device. For example, based on interaction data characterizing prior instances of the user invoking the automated assistant, one or more operations can be identified to prepare for one or more specific inputs from the user. The one or more operations can include, but are not limited to, executing a test query to prepare for a search process for receiving a search request from a user; authenticating a user with an application that provides search results; downloading one or more libraries that can be accessed by one or more applications when responding to user input; and / or any other operation that can be executed to prepare for automated assistant input from a user.

[0060] From operation 506, method 500 can proceed to optional operation 508, which can include determining whether input to the automated assistant has been received and / or whether input has been received within a threshold amount of time of a modification to the navigation operation. For example, following a user modifying the destination of the vehicle, the user can provide a spoken utterance such as "Assistant, what are the operating hours of the bakery?" ("bakery" can be a new or updated destination for the vehicle). When it is determined at operation 508 that input has been received, method 500 can proceed to optional operation 510. Otherwise, method 500 can return to operation 502 and / or another operation.

[0061] Optional operation 510 can include causing the automated assistant to perform an action corresponding to the input and one or more operations. For example, the action can be submitting a search query to a website to determine the operating hours of a bakery. As a result of the one or more operations performed at operation 506, the action can be performed with less latency. For example, the operation that is preemptively performed can include submitting a search query, which can ensure that an appropriate communication channel is established between the automated assistant and the search server. In this way, when the user subsequently provides a spoken utterance regarding the bakery's hours, the automated assistant's response will be provided with less latency as a result of the preemptively established communication channel.

[0062] Figure 6 6 is a block diagram 600 of an example computer system 610. The computer system 610 typically includes at least one processor 614 that communicates with a number of peripheral devices via a bus subsystem 612. These peripheral devices may include, for example, a storage subsystem 624 including a memory 625 and a file storage subsystem 626, a user interface output device 620, a user interface input device 622, and a network interface subsystem 616. The input and output devices allow a user to interact with the computer system 610. The network interface subsystem 616 provides an interface to an external network and couples to corresponding interface devices in other computer systems.

[0063] The user interface input device 622 may include a keyboard, a pointing device such as a mouse, trackball, touchpad or graphic tablet, a scanner, a touch screen incorporated into a display, an audio input device such as a voice recognition system, a microphone, and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways to input information into the computer system 610 or onto a communication network.

[0064] The user interface output device 620 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for producing a visible image. The display subsystem may also provide a non-visual display, for example, via an audio output device. In general, the use of the term "output device" is intended to include all possible types of devices and methods for outputting information from the computer system 610 to a user or to another machine or computer system.

[0065] The storage subsystem 624 stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 624 may include logic for performing selected aspects of the method 300, the method 400, the method 500, and / or implementing one or more of the system 200, the computing device 104, the portable computing device, the vehicle computing device, the server device, and / or any other application, device, apparatus, and / or module discussed herein.

[0066] These software modules are typically run by the processor 614 alone or in combination with other processors. The memory 625 used in the storage subsystem 624 can include many memories, including a main random access memory (RAM) 630 for storing instructions and data during program execution and a read-only memory (ROM) 632 in which fixed instructions are stored. The file storage subsystem 626 can provide persistent storage for program and data files, and can include a hard drive, a floppy disk drive and related removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of certain implementations can be stored in the storage subsystem 624 by the file storage subsystem 626, or in other machines accessible by the processor 614.

[0067] The bus subsystem 612 provides a mechanism for the various components and subsystems of the computer system 610 to communicate with each other as intended. Although the bus subsystem 612 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0068] Computer system 610 may be of various types including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, Figure 6 The description of the computer system 610 depicted in FIG. 6 is intended only as a specific example for illustrating some implementations. Many other configurations of the computer system 610 may have more Figure 6 The computer system depicted in FIG.

[0069] In situations where the systems described herein collect personal information about users (or "participants" as they are often referred to herein) or can make use of personal information, users may be provided with the opportunity to control whether a program or feature collects user information (e.g., information about a user's social network, social actions or activities, occupation, a user's preferences, or a user's current geographic location) or to control whether and / or how content that may be more relevant to the user is received from a content server. In addition, certain data may be processed in one or more ways before it is stored or used so that personally identifiable information is removed. For example, the identity of a user may be processed so that personally identifiable information cannot be determined for that user, or the user's geographic location may be generalized (such as to a city, zip code, or state level) in the case where geographic location information is obtained so that the user's specific geographic location cannot be determined. Thus, users may control how signals are collected and / or used about users.

[0070] Although several implementations have been described and illustrated herein, various other means and / or structures for performing functions and / or obtaining results and / or one or more advantages described herein may be utilized, and each of such changes and / or modifications is considered to be within the scope of the implementations described herein. More generally, all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications to which the teachings are used. Those skilled in the art will recognize or be able to use no more than routine experiments to ascertain many equivalents of the specific implementations described herein. Therefore, it should be understood that the foregoing implementations are presented only as examples, and within the scope of the appended claims and their equivalents, implementations may be practiced in a manner different from that specifically described and claimed. The implementations disclosed herein are dedicated to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure when such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0071] In some implementations, a method implemented by one or more processors is described as including operations such as determining at a computing device that an operation of an application has undergone a state change. The method can further include operations for processing interaction data based on the state change, the interaction data being based on whether, for each of one or more prior instances, a corresponding user interacted with an automated assistant within a threshold time period in which one or more operational state changes occurred at the application or a separate application. The method can further include operations for determining at the computing device that a state change occurred while the user was in a vehicle being driven toward a destination, wherein the computing device or another computing device provides access to features of the automated assistant while the user is in the vehicle. The method can further include operations for processing vehicle data to facilitate determining whether a state change occurred within another threshold time period of an estimated time to arrive at the destination. The method can further include the following operations: when the interaction data satisfies one or more criteria and when it is determined that a state change has not occurred within another threshold time period of an estimated time to arrive at the destination: causing one or more operations to be initialized via the computing device or another computing device, wherein the one or more operations are associated with actions performed via the automated assistant.

[0072] In some implementations, the method can further include the following operations: when it is determined that the state change has occurred within another threshold time period of the estimated time of arrival at the destination: causing one or more other operations to be initiated via the computing device or another computing device without initializing the one or more operations, wherein the one or more other operations are associated with another action performed via the automated assistant or a separate application. In some implementations, causing the one or more operations to be initialized includes: identifying a portion of a memory of the other computing device to load data associated with the one or more operations, wherein the computing device is a portable computing device that the user has positioned inside the vehicle, and the other computing device is a vehicle computing device that is integral to the vehicle.

[0073] In some implementations, the method can further include the steps of: subsequent to causing one or more operations to be initialized: determining that the user provided a spoken utterance including a request for the automated assistant to perform an action, wherein the automated assistant accesses a portion of the memory in response to the spoken utterance, and causing the automated assistant to perform the action in response to the spoken utterance. In some implementations, processing the interaction data includes: determining based on the interaction data that the user caused the application to perform an operation that underwent a state change, and based on the user identifying one or more actions initialized via the automated assistant during one or more prior instances in which the user interacted with the automated assistant within a threshold time period in which one or more operational state changes occurred at the application, wherein the one or more operations are selected for initialization based on the one or more actions.

[0074] In some implementations, determining that an operation corresponding to an application has undergone a state change includes determining that the application has ceased rendering certain information via an interface of the computing device or another computing device. In some implementations, determining that the state change has occurred while the user is in a vehicle being driven toward a destination includes determining that a portion of a route to the destination includes an area of ​​restricted network activity, wherein the one or more operations initiated via the computing device or another computing device include a separate operation to retrieve data over a wireless network connection.

[0075] In other implementations, a method implemented by one or more processors is described as including operations such as determining at a vehicle computing device of a vehicle that an electronic communication is received while a user is in a vehicle, wherein the electronic communication is received at the vehicle computing device or another computing device paired with the vehicle computing device. The method can further include the following operations: based on processing interaction data, identifying an action that the user has previously initiated from within the vehicle via an automated assistant within a threshold amount of time of receiving a previous communication. The method can further include the following operations: based on processing interaction data, determining that the user initiated the action within a threshold amount of time of receiving the previous communication when the previous communication has one or more characteristics. The method can further include the following operations: when the electronic communication received by the user has the one or more characteristics: causing one or more operations to be initialized via the vehicle computing device or another computing device, wherein the one or more operations are associated with the action performed via the automated assistant.

[0076] In some implementations, the one or more characteristics correspond to a contact who authored the communication, the contact having contact information stored at the vehicle computing device or other computing device in association with the user. In some implementations, the one or more characteristics correspond to communications received at a time of day. In some implementations, the one or more characteristics correspond to communications including certain content. In some implementations, the method can further include the following operations: when another user who authored the electronic communication received by the user does not have the one or more characteristics: causing one or more other operations to be initiated via the vehicle computing device or other computing device, wherein the one or more other operations are performed via an automated assistant or a separate application without initiating the one or more operations.

[0077] In some implementations, causing one or more operations to be initialized includes: identifying a portion of a memory of a vehicle computing device or other computing device to load data associated with the one or more operations, wherein the data corresponds to an automatic speech recognition (ASR) feature of an automated assistant. In some implementations, causing one or more operations to be initialized includes: transmitting authentication data to a remote server device associated with the automated assistant via a wireless network connection. In some implementations, the method can further include the following operations: subsequent to causing the one or more operations to be initialized via the vehicle computing device or another computing device: determining that a user provided a spoken utterance to the automated assistant to facilitate causing the automated assistant to perform an action or another action, and causing audio data corresponding to the spoken utterance to be communicated to the remote server device via a secure connection established using the authentication data.

[0078] In some implementations, the action or another action corresponds to a request for the automated assistant to generate a reminder associated with the received communication. In some implementations, the method can further include determining an operation to which a computing resource is to be limited at the vehicle computing device within a threshold amount of time that the user receives the electronic communication, wherein one or more operations are initiated to be performed via the vehicle computing device or another computing device within the threshold amount of time that the user receives the electronic communication. In some implementations, the computing resource corresponds to an amount of memory available to one or more applications accessible via the vehicle computing device.

[0079] In yet another implementation, a method implemented by one or more processors is described as including operations such as determining at a vehicle computing device that a navigation operation of a vehicle has undergone a state change, wherein the vehicle computing device provides access to features of an automated assistant while a user is in the vehicle. The method can further include operations for processing interaction data based on determining that the navigation operation has undergone a state change, the interaction data being based on one or more prior instances in which the user interacted with the automated assistant within a threshold time period in which the vehicle computing device or another computing device experienced a separate state change during a prior navigation operation. The method can further include operations for identifying, based on the interaction data, an action initiated by the user via the automated assistant within a threshold time period in which the vehicle or another vehicle experienced a separate state change to the prior navigation operation. The method can further include operations for causing one or more operations to be initialized via the vehicle computing device based on the interaction data, wherein the one or more operations are associated with the action initialized via the automated assistant. In some implementations, causing the one or more operations to be initialized via the vehicle computing device includes causing operation data associated with the one or more operations to be copied to a memory of the vehicle computing device.

Claims

1. A system comprising: a memory for storing instructions; One or more processors operable to execute instructions for: Determine that the application's operations have undergone a state change; processing the interaction data based on the state changes, the interaction data being based on: for each of one or more prior instances, whether the corresponding user interacted with the automated assistant within a threshold time period of one or more operational state changes occurring at the application or a separate application; determining that the state change occurred while a user is in a vehicle being driven toward a destination, wherein a vehicle computing device provides access to features of the automated assistant while the user is in the vehicle; processing vehicle data to facilitate determining whether the change in state occurs within another threshold time period of an estimated time of arrival at the destination; and When the interaction data satisfies one or more criteria and when it is determined that the state change has not occurred within the further threshold time period of the estimated time of arrival at the destination: causing one or more operations to be initiated via the vehicle computing device, The one or more operations are associated with an action performed via the automated assistant.

2. The system according to claim 1, wherein: One or more of the processors are further operable to execute instructions for: When it is determined that the change in state has occurred within the further threshold time period of the estimated time of arrival at the destination: causing one or more other operations to be initiated via the vehicle computing device without initiating the one or more operations, The one or more other operations are associated with another action performed via the automated assistant or a separate application.

3. The system according to claim 1, wherein: In causing the one or more operations to be initialized, one or more of the processors are configured to: identifying a portion of a memory of the vehicle computing device to load data associated with the one or more operations, The application is an application of a portable computing device that the user has located inside the vehicle.

4. The system according to claim 3, wherein: One or more of the processors are further operable to execute instructions for: Following causing the one or more operations to be initialized: determining that the user provided a spoken utterance comprising a request for the automated assistant to perform the action, wherein the automated assistant accesses the portion of the memory in response to the spoken utterance, and The automated assistant is caused to perform the action in response to the spoken utterance.

5. The system according to claim 1, wherein: When processing the interaction data, one or more of the processors are configured to: determining, based on the interaction data, that the user caused the application to perform the operation that was subject to the state change, and one or more actions initiated via the automated assistant based on the user identification during the one or more prior instances in which the user interacted with the automated assistant within a threshold time period in which the one or more operational state changes occurred at the application by the user, The one or more operations are selected for initialization based on the one or more actions.

6. The system according to claim 1, wherein: Upon determining that the operation corresponding to the application has undergone the state change, one or more of the processors are configured to: It is determined that the application has stopped rendering certain information.

7. The system according to claim 1, wherein: Upon determining that the change in state occurred while the user was in the vehicle being driven toward the destination, one or more of the processors are configured to: determining that a portion of the route to the destination includes an area of ​​restricted network activity, Wherein the one or more operations initiated via the vehicle computing device include a separate operation of retrieving data via a wireless network connection.

8. A system comprising: a memory for storing instructions; One or more processors operable to execute instructions for: determining that an electronic communication is received while the user is in a vehicle, wherein the electronic communication is received at a vehicle computing device or another computing device paired with the vehicle computing device; identifying, based on processing the interaction data, an action that the user has previously initiated from within the vehicle via the automated assistant within a threshold amount of time of receiving a previous communication; determining, based on processing the interaction data, that the user initiated the action within a threshold amount of time of receiving the previous communication when the previous communication had one or more characteristics; and When the electronic communication received by the user has the one or more characteristics: causing one or more operations to be initiated via the vehicle computing device or the other computing device, Wherein the one or more operations are associated with the action performed via the automated assistant.

9. The system according to claim 8, wherein: The one or more characteristics correspond to a contact who authored the communication, the contact having contact information stored at the vehicle computing device or the other computing device in association with the user.

10. The system according to claim 8, wherein: The one or more characteristics correspond to the time of day at which communications are received.

11. The system according to claim 8, wherein: The one or more characteristics correspond to communications including certain content.

12. The system according to claim 8, wherein: One or more of the processors are further operable to execute instructions for: When another user who authored the electronic communication received by the user does not possess the one or more characteristics: causing one or more other operations to be initiated via the vehicle computing device or the other computing device, Wherein, the one or more other operations are performed via the automated assistant or a separate application without initiating the one or more operations.

13. The system according to claim 8, wherein: In causing the one or more operations to be initialized, one or more of the processors are configured to: identifying a portion of a memory of the vehicle computing device or another computing device to load data associated with the one or more operations, Wherein, the data corresponds to automatic speech recognition (ASR) features of the automated assistant.

14. The system according to claim 8, wherein: In causing the one or more operations to be initialized, one or more of the processors are configured to: Authentication data is transmitted via a wireless network connection to a remote server device associated with the automated assistant.

15. The system of claim 14, wherein: One or more of the processors are further operable to execute instructions for: Subsequent to causing one or more operations to be initiated via the vehicle computing device or the other computing device: determining that the user provided a spoken utterance to the automated assistant to facilitate causing the automated assistant to perform the action or another action, and Audio data corresponding to the spoken utterance is caused to be communicated to the remote server device via a secure connection established using the authentication data.

16. The system of claim 14, wherein: The action or the other action corresponds to a request for the automated assistant to generate a reminder associated with the received communication.

17. The system according to claim 8, wherein: One or more of the processors are further operable to execute instructions for: determining that computing resources will be limited at the vehicle computing device within the threshold amount of time that the user receives the electronic communication, Wherein causing the one or more operations to be initiated via the vehicle computing device or the other computing device is performed within the threshold amount of time of the user receiving the electronic communication.

18. The system of claim 17, wherein: The computing resource corresponds to an amount of memory available to one or more applications accessible via the vehicle computing device.

19. The system of claim 17, wherein: In causing the one or more operations to be initiated via the vehicle computing device, one or more of the processors are to: Operational data associated with the one or more operations is caused to be copied to a memory of the vehicle computing device.

20. A system comprising: a memory for storing instructions; One or more processors operable to execute instructions for: determining that the navigation operation of the vehicle has undergone a state change, wherein a vehicle computing device of the vehicle provides access to features of the automated assistant while the user is in the vehicle; processing interaction data based on determining that the navigation operation has undergone the state change, the interaction data being based on one or more prior instances in which the user interacted with the automated assistant within a threshold time period in which the vehicle computing device experienced a separate state change during a prior navigation operation; identifying, based on the interaction data, an action initiated by the user via the automated assistant within the threshold time period of the vehicle undergoing the individual state change to the prior navigation operation; and causing one or more operations to be initiated via the vehicle computing device based on the interaction data, Wherein the one or more operations are associated with the action initiated via the automated assistant.

Images