Restricting third party applications from accessing audio data content

Abstract

Implementations relate to limiting, upon request by an application, access of the application to audio data content captured after rendering the content to a user. An application may generate content to be rendered to a user and an additional request to receive audio data content from audio data captured immediately after rendering the content. The content may be processed using a trained machine learning model that generates as output an indication of a likelihood that audio data content is not appropriate to be provided after rendering the content from the application. Where the application uncomfortably requests audio data content, provision of the audio data content and / or subsequent audio data content to the application may be restricted.

Description

Background Art

[0001] Humans may participate in human-computer conversations using interactive software applications, which are referred to herein as "automated assistants" (also referred to as "digital agents," "interactive personal assistants," "intelligent personal assistants," "assistant applications," "conversational agents," and the like). For example, humans (who may be referred to as "users" when they interact with automated assistants) may provide commands and / or requests to automated assistants using verbal natural language input (i.e., utterances), which in some cases may be converted to text and then processed by providing textual (e.g., typed) natural language input and / or by touch and / or non-verbal physical movements (e.g., hand gestures, eye gaze, facial movements, and the like). The automated assistant responds to the request by providing responsive user interface outputs (e.g., auditory and / or visual user interface outputs), controlling one or more intelligent devices, and / or controlling one or more functions of a device that implements the automated assistant (e.g., controlling other applications of the device).

[0002] As mentioned above, many automated assistants are configured to be interacted with via spoken utterances. To maintain user privacy and / or conserve resources, the automated assistant avoids performing one or more automated assistant functions based on all spoken utterances present in audio data detected (at least in part) via a microphone of a client device implementing the automated assistant. Instead, certain processing based on spoken utterances occurs only in response to determining that certain conditions exist.

[0003] For example, many client devices including automated assistants and / or interfaces with automated assistants include hot word detection models. When the microphone of such client device is not deactivated, the client device can use the hot word detection model to continuously process the audio data detected via the microphone to generate a predicted output indicating whether there are one or more hot words (including multi-word phrases), such as "Hey Assistant", "OK Assistant" and / or "Assistant". When the predicted output indicates the presence of a hot word, any audio data following (and optionally determined to include voice activity) within a threshold amount of time can be processed by one or more devices such as a speech recognition component, a voice activity detection component, and / or a remote automated assistant component. Audio data predicted to contain hot words can also be processed by other devices and / or remote automated assistant components. Further, a natural language understanding engine can be used to process the recognized text (from the speech recognition component), and / or an action can be performed based on the natural language understanding engine output. An action can include, for example, generating and providing a response and / or controlling one or more applications and / or smart devices. Other hot words (e.g., "No", "Stop", "Cancel", "Volume Up", "Volume Down", "Next Track", "Previous Track", etc.) can be mapped to various commands, and when the predicted output indicates the presence of one of these hot words, the client device can process the mapped command. However, when the predicted output indicates the absence of the hot word, the corresponding audio data will be discarded without any further processing, thereby saving resources and protecting user privacy.

[0004] An automated assistant executing at least in part on a client device can communicate with applications installed on a computing device and / or with applications executing on other devices (e.g., cloud-based applications or skills of the automated assistant). As part of the functionality of the application, the application can selectively request access to audio data content of audio data captured by the automated assistant.

[0005] For example, an application may generate a prompt that is rendered to the user as part of a conversation between the user and the application. For example, the conversation may be mediated by an automated assistant, the prompt is provided to the automated assistant, and the automated assistant may render the prompt as part of the conversation. The application may further request access to the audio data content of the audio data captured after the prompt in the conversation with the user. For example, if the prompt from the application is "what kind of music would you like to listen to (what type of music you want to listen to)", then the application may request access to the audio data content after the prompt so that the application can determine the user's response (e.g., a response that specifies a music genre). The audio data content provided to the application may include the audio data itself, an automatic speech recognition (ASR) transcription of the audio data (e.g., generated by an ASR engine of an automated assistant) and / or a structured representation generated based on the transcription (e.g., NLU data generated by a natural language understanding (NLU) engine of an automated assistant).

[0006] However, providing unrestricted access to audio data content when it is requested by an application may bring various drawbacks. For example, transmitting the audio data content to an application when it is not really needed for the conversation may utilize bandwidth of limited communication resources. For example, when the application is operating on a remote server, transmitting the audio data content may utilize network resources. As another example, transmitting the audio data content to an application when it is not really needed for the conversation may bring security issues for content that is unintentionally captured in the audio data content. Summary of the invention

[0007] Some implementations disclosed herein involve determining whether to restrict access to audio data content of audio data captured by a microphone during a conversation with a user based on whether the application inappropriately requests access to the audio data content of the audio data captured by the microphone during a conversation with the user. Some implementations include identifying content generated by the application to be rendered to the user during a conversation turn, and a request from the application to provide it with audio data content corresponding to audio data captured after at least the conversation turn. The content of the conversation turn is processed using one or more machine learning models, which are trained to generate outputs indicating the likelihood that the application appropriately receives the audio data content (e.g., the user should expect that after this output instance, the microphone will be active and the client device will capture audio data, and the content of the audio data will be provided to the application). If the output does not exceed the appropriateness threshold, the instance of the application can be restricted from accessing the requested audio data content and / or content corresponding to subsequently captured audio data.

[0008] As an example, an application may generate content "What is the capital of France" to be rendered during a conversation turn in a conversation with a user. The application may request and / or otherwise cause the provision of content of audio data captured after the conversation turn (e.g., causing a microphone to be active, avoiding deactivation of the microphone if it is already active, requesting access to captured audio data, requesting content of captured audio data). In the case of a conversation turn with content "What is the capital of France?", the user may expect that the immediately following audio will be captured by the device and the content of the audio data (e.g., audio data, a textual representation of the audio data) will be provided to the application because the content of the conversation turn is a question directed to the user. Therefore, in this case, providing the audio data content to the application would likely be appropriate given the purpose of the conversation turn (i.e., eliciting a response).

[0009] As another example, an application may generate content "Thanks for playing. Goodbye" to be provided during a conversation turn in a conversation, and further cause content of the audio data captured after rendering the conversation turn to be provided to the application. In this case, the conversation turn includes a signoff message, and the user may not expect the audio data content to be provided to the application. Therefore, in this case, providing the audio data content to the application after rendering the conversation turn may not be appropriate.

[0010] In some implementations, a machine learning model may be used to determine whether it is appropriate to provide content of audio data captured after content rendered to a user during a conversation turn to the application based on content generated by the application. For example, a machine learning model may be trained using training instances, each of which includes content of a conversation turn rendered to a user and followed by captured audio data content provided to the application. Further, each training instance may include an indication of whether it is appropriate to provide audio data given the intent of the content of the corresponding conversation turn. The machine learning model may provide an output indicating the likelihood that providing the audio data content is appropriate. For example, a trained machine learning model may provide a value between 0.0 (e.g., completely confident that providing the audio data content after the conversation turn is rendered) and 1.0 (e.g., completely confident that providing the audio data content is inappropriate) as an output.

[0011] In some implementations, a textual representation of the content of a conversation turn that was originally provided to the user as audio can be used as an input to the machine learning model. For example, content including "What is the capital of France?" generated by an application and rendered by an automated assistant in communication with the application (e.g., an automated assistant facilitating communication between the application and the user) can be rendered during a conversation turn. The textual representation can be, for example, a transcription of the content provided to the user as audio during the conversation.

[0012] In some implementations, the vector representation of the content of the conversation turn can be used as an input to a machine learning model. For example, in some implementations, processing using one or more other machine learning models can be performed, which produces a vector representing the content of the content in the embedding space. This vector can then be provided as input to the machine learning model for further processing.

[0013] In some implementations, in addition to the content of the conversation turn, additional information may be provided as input to the machine learning model. In some implementations, the additional information may include an application type that indicates the type of application providing the conversation turn being processed. In some cases, a user may expect some applications to receive the content of the audio data, while other applications are less likely to have access to the audio data content. The application type may indicate, for example, whether the application is a quiz game, a map application, a ride-sharing application, a restaurant reservation application, and / or one or more other applications that can provide a conversation to the user and can cause activation of a microphone of the client device.

[0014] In some implementations, the additional information may include an indication of the context of the corresponding conversation turn. For example, a training instance may include an indication of whether the corresponding conversation turn is the first turn of the conversation (i.e., the first response from the application after invoking the application), the terminating turn of the conversation (i.e., the application does not provide a subsequent conversation turn), or whether the conversation turn is in the middle of the conversation. In addition, for example, the context may include information related to other conversation turns of the conversation (e.g., content from the immediately preceding, immediately succeeding conversation turn, and / or content from one or more other conversation turns of the conversation).

[0015] In some implementations, the additional information may include the source of the application that generated the content rendered during the conversation turn. For example, one or more sources of the application (e.g., application developer, application distributor) and the content may be provided as input to a machine learning model. In some cases, the source of the application may indicate whether the application is trusted (e.g., a known developer versus an unknown or suspicious developer).

[0016] In some implementations, the additional information may include one or more settings of the user related to the sensitivity of the shared audio data content. For example, the user may set one or more settings as part of the user account that indicate how sensitive the user expects the application to be when receiving the content of the audio data.

[0017] In addition to the content provided as input to the machine learning model, the training instance may also include any or all of the additional information previously described as additional input to the machine learning model. For example, the training instance may include the type of application that generates the corresponding content, the context of the content, and / or other additional information related to the content of one or more rendered conversation turns. Further, in some implementations, training instances may be generated based on other conversation sources. For example, training instances may be generated using content from one or more discussion websites such as message boards, each training instance including comments generated by a user and an indication of whether the comment is an appropriate comment to elicit a response. An indication of the appropriateness / inappropriateness of providing a response to an application that generates the content of the training instance may be determined, for example, by one or more human curators who review the content and determine whether the user expects to capture a response and provide the response to the application based on the assumption that the content originates from the application.

[0018] Once the machine learning model is trained, the machine learning model can be used to process content provided during one or more conversation turns of a target application, such as a third-party application, to determine whether the target application should have access to audio data content from audio data captured by a client device that is executing an instance of the target application. For example, the machine learning model can be used to identify and process content from multiple conversation turns that have been provided in response to a request from the target application to provide the content and followed by the application receiving the audio data content. The output of the machine learning model can be used to determine the likelihood that the target application has been appropriately provided with audio data.

[0019] As an example, 1000 requests generated by a target application to provide content and receive subsequent audio data content can be identified from a log of requests for rendering content by multiple instances of the target application. For each content in the content of the rendered conversation turn, the output of the machine learning model can be in the range of 0.0 to 1.0, as previously described. If, for example, the output from processing the content of any request in the 1000 requests indicates a possibility that the corresponding request for subsequent audio data content is inappropriate (e.g., an output that exceeds a threshold), then the target application can be marked for manual review, and / or the instance of the target application can be restricted from accessing subsequent audio data content (e.g., not providing audio data content, providing audio data content but more significantly warning the user to pay attention to the transmission), and the restriction is permanent or until further manual review is performed. Additionally, for example, if the generated output from a threshold number of requests (e.g., content from more than 10 processed requests out of 1000 processed requests) exceeds a threshold, the target application can be marked for further manual review and / or instances of the application can be restricted from accessing subsequent audio data content, including uninstalling and / or disabling the application.

[0020] In some implementations, the request may come from multiple client devices, each of which executes an instance of the target application. For example, logs from multiple devices that have executed an instance of an application can be identified, and a request for rendering content that additionally includes a request for subsequent audio data content to be provided can be used to determine whether any instance in the instance of the application inappropriately accesses subsequent audio data. In some implementations, content from one or more conversation turns can be processed when the content is rendered (or before rendering) so that an inappropriate request for the audio data content can be identified when the audio data content is provided to the application (or before the provision occurs). For example, for each conversation turn in which the application is rendered and then the application is provided with audio data content, a trained machine learning model can be used to process the content of the conversation turn to determine whether the audio data is appropriate. In addition, for example, for each request to render content as a conversation turn accompanied by a request for subsequent audio data content, the content can be processed before the content is rendered to determine whether the request for subsequent audio data content is appropriate. In either case, when the output from the machine learning model indicates that providing audio data content to an application is inappropriate, the application can be restricted from further access to the audio data and / or restricted from providing further content to the user.

[0021] Therefore, the method described herein can alleviate the need of processing this audio data content unnecessarily by preventing the provision of this audio data content for processing when providing audio data content is inappropriate. Further, by not transmitting audio data and / or audio data content when inappropriate and / or when the transmitted audio data content is subsequently not utilized, computing resources and / or network use are reduced. Still further, when the user does not expect that the audio data content is provided to one or more applications, security can be improved by reducing the transmission of this content. Given the content of the conversation turn rendered to the user, by determining that this provision will be inappropriate before providing audio data content, the personal information that can be included in this audio data content can remain safe, and third parties cannot access when not needed.

[0022] The above description is provided as an overview of only some of the implementations disclosed herein. Those implementations and other implementations are described in more detail herein.

[0023] It should be understood that all combinations of the foregoing concepts and additional concepts described in more detail herein are considered part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as part of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is an illustration of an example environment in which implementations disclosed herein may be implemented.

[0025] Figure 2 A conversation between an application and a user is shown.

[0026] Figure 3 is a flow diagram illustrating the training of a machine learning model and utilizing the trained machine learning model to process one or more conversation turns provided by an application.

[0027] Figure 4 is a flow chart illustrating an example method according to various implementations disclosed herein.

[0028] Figure 5 is a flow chart illustrating another example method according to various implementations disclosed herein.

[0029] Figure 6 is a flow chart illustrating another example method according to various implementations disclosed herein.

[0030] Figure 7 An example architecture of a computing device is shown. DETAILED DESCRIPTION

[0031] First go to Figure 1, showing an example environment in which various implementations may be performed. Figure 1 A client device 100 is included that executes an automated assistant that executes an instance of an automated assistant client 120. One or more cloud-based automated assistant components may be implemented on one or more computing systems (collectively referred to as "cloud" computing systems) that are communicatively coupled to the client device 100 via one or more local area networks and / or wide area networks (e.g., the Internet). The instance of the automated assistant client 120, optionally via interaction with one or more of the cloud-based automated assistant components, may form what appears to be a logical instance of the automated assistant from the user's perspective, with which the user may engage in a human-computer conversation.

[0032] The client device 100 may be, for example, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a vehicle computing device (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), a standalone interactive speaker, a smart appliance such as a smart TV, and / or a wearable device including a computing device (e.g., a watch with a computing device, glasses with a computing device, a virtual or augmented reality computing device).

[0033] The automated assistant 120 conducts a human-computer dialogue session with the user via the user interface input and output devices of the client device 100. In order to protect user privacy and / or save resources, in many cases, the user must usually explicitly call the automated assistant 120 before the automated assistant will fully process the spoken utterance. The explicit call to the automated assistant 120 can occur in response to a certain user interface input received at the assistant device 100. For example, the user interface input that can call the automated assistant 120 via the client device 100 can optionally include the actuation of the hardware and / or virtual buttons of the client device 100. In addition, the automated assistant client can include one or more local engines, such as a call engine 130 that is operable to detect the presence of one or more verbal general call wake-up words. The call engine 130 can call the automated assistant 120 in response to detecting one of the verbal call wake-up words. For example, the call engine 130 can call the automated assistant 100 in response to detecting a verbal call wake-up word such as "Hey Assistant", "OK Assistant" and / or "Assistant". The invocation engine 130 may continuously process a stream of audio data frames based on the output of one or more microphones 140 from the client device 100 to monitor for the occurrence of a spoken invocation phrase. While monitoring for the occurrence of a spoken invocation phrase, the invocation engine 130 discards any audio data frames that do not include a spoken invocation phrase (e.g., after temporary storage in a buffer). However, when the invocation engine 130 detects the occurrence of a spoken invocation phrase in a processed audio data frame, the invocation engine may invoke the automated assistant 120. As used herein, "invoking" the automated assistant 120 may include causing one or more previously inactive functions of the automated assistant 120 to be activated. For example, invoking the automated assistant 120 may include causing one or more local engines and / or cloud-based automated assistant components to further process the audio data frame based on which the invocation phrase was detected, and / or one or more subsequent audio data frames (whereas no further processing of the audio data frame occurred prior to the invocation). For example, the local and / or cloud-based components may process the captured audio data using an ASR model in response to the invocation of the automated assistant 120.

[0034] Figure 1 The automated assistant 120 in FIG. 1 is shown to include an automatic speech recognition (ASR) engine 122, a natural language understanding (NLU) engine 124, a text-to-speech (TTS) engine 126, and a fulfillment engine 128. In some implementations, one or more of the engines shown may be omitted (e.g., instead implemented solely by the cloud-based automated assistant component 140) and / or additional engines may be provided (e.g., the invocation engine described above).

[0035] The ASR engine 122 may process audio data capturing a spoken utterance to generate a recognition of the spoken utterance. For example, the ASR engine 122 may utilize one or more ASR machine learning models to process the audio data to generate a prediction of the recognized text corresponding to the utterance. In some of those implementations, the ASR engine 122 may generate a corresponding confidence metric for each of the one or more recognized terms indicating a confidence that the predicted term corresponds to the spoken utterance.

[0036] The TTS engine 126 can convert text to synthesized speech, and can rely on one or more speech synthesis neural network models to do so. For example, the TTS engine 126 can be used to convert a text response into audio data that includes a synthesized version of the text, and the synthesized version can be rendered in an audible manner via the hardware speaker 150 of the client device 100.

[0037] The NLU engine 124 determines the semantic meaning of the audio and / or text converted from the audio by the ASR engine, and determines assistant actions corresponding to those semantic meanings. In some implementations, the NLU engine 124 determines the assistant action as the intent and / or parameters determined based on the recognition of the ASR engine 122. In some cases, the NLU engine 124 can parse the intent and / or parameters based on a single utterance of the user, and in other cases, prompts can be generated based on unresolved intents and / or parameters, those prompts are rendered to the user, and the NLU engine 124 uses the user's response to those prompts to parse the intent and / or parameters. In those cases, the NLU engine 124 can optionally work in conjunction with the dialog manager engine 170, which determines the unresolved intent and / or parameters and / or generates corresponding prompts. The NLU engine 124 can use one or more NLU machine learning models to determine the intent and / or parameters.

[0038] Fulfillment engine 128 may cause execution of the assistant action determined by NLU engine 124. For example, if NLU engine 124 determines that the assistant action is "turning on the kitchen lights," fulfillment engine 128 may cause corresponding data to be transmitted (either directly to the lights or to a remote server associated with the manufacturer of the lights) to cause the "kitchen lights" to be "turned on." As another example, if NLU engine 124 determines that the assistant action is "provide a summary of the user's meetings for today," fulfillment engine 128 may access the user's calendar, summarize the user's meetings for the day, and cause the summary to be rendered at client device 100 in a visual and / or audible manner.

[0039] The automated assistant 120 further includes an application interface 160 that can communicate with one or more third-party applications executing on the client device 100. An application (not shown) can provide information to the automated assistant 120 via the application interface 160 to cause the automated assistant 120 to perform one or more actions, and / or the application can be given permission to directly access one or more components of the client device 100. For example, in some implementations, an application can provide text to the automated assistant 120, which can then utilize one or more components to process the text (e.g., perform ASR, NLU, TTS). In addition, for example, the application can be given permission to directly receive audio data content corresponding to audio data captured by the microphone 140 and / or provide output via the speaker 150.

[0040] In some implementations, automated assistant 120 may communicate with the user via microphone 140 and / or speaker 150 via client device 110. For example, Figure 2 , showing a conversation between a user and automated assistant 120. Before the conversation begins, invocation engine 130 can monitor audio data captured by microphone 140 to detect the presence of one or more spoken general invocation wake-up words in the audio data. For example, at conversation turn 205, the user says the phrase "OK Assistant", which can be a wake-up word indicating that the user intends to start a conversation with automated assistant 120.

[0041] Dialogue turn 205 further includes the user saying “open the Quiz Application”. Dialogue manager 170 may determine the intent based on one or more techniques described herein and perform one or more actions based on the intent. For example, dialogue manager 170 may determine that the user's intent is to interact with an application called “Quiz Application” that is executing on client device 100 and / or executing on one or more other devices. In response, fulfillment engine 128 may transmit some or all of the audio data content from the audio data captured by microphone 140 to the application. In response, the application may provide a response (e.g., text) to automated assistant 120 via application interface 160 to be provided to the user as an audio output via speaker 150. For example, the application may provide text “Welcome to Quiz Application. What would you like to play” to automated assistant 120, and automated assistant 120 may process the text using TTS engine 126 and then render dialogue turn 210 via speaker 150.

[0042] In some implementations, the application may further send an indication to the automated assistant 120 via the application interface 160 that a response to the provided content is expected and that subsequent audio is captured. In response, the automated assistant 120 may activate the microphone 140 so that subsequent audio can be captured and audio data can be generated and further processed. For example, after providing the conversation turn 210, the automated assistant 120 may cause the microphone 140 to be active and / or avoid deactivating the microphone 140. The audio data captured after the conversation turn 210 may be further processed by the automated assistant 120 (e.g., ASR, NLU), and one or more actions may be performed. For example, the content of the captured audio data, such as a textual representation of the audio data (after ASR processing), and / or the user's intent (after NLU), may be provided to the application as audio data content.

[0043] For example, at dialog turn 215, the user utters "Let's do country capitals". In some implementations, audio data including the utterance can be provided to an application for further processing. In some implementations, the automated assistant 120 can first perform ASR to generate the text "let's do country capitals", and the text can be provided to the application as audio data content for further processing. In some implementations, the automated assistant 120 can first determine the intent by performing NLU, and provide the intent (e.g., "QUIZ=COUNTRY CAPITALS") as audio data content to the application.

[0044] The conversation can continue, as Figure 2 As shown, until the user indicates to stop the conversation. For example, at conversation turn 235, the user indicates "I'm done playing" as the end of the conversation. In response, conversation manager 170 can determine that the user intends to stop the conversation and / or the conversation turn can be provided to the application as audio data content for further processing. In some implementations, the application can determine that the conversation is complete and subsequent audio data content should not be provided to the application. This can include, for example, providing an instruction to stop sending audio data content to automated assistant 120.

[0045] Thus, for some conversation turns provided to the user when interacting with the application 160, the user should expect the microphone 140 to be active and capturing audio data. For example, if the conversation turn is soliciting a response (e.g., a question), the user should expect that any subsequent audio data will be captured and provided to the application to determine whether the audio data includes a response to the question. However, for some conversation turns provided to the user, the user will not expect any subsequent audio data content to be provided to the application. Because the application may be a third-party application, the user may expect that the third party does not have access to audio data content that may include, for example, personal information that is not related to the conversation with the application.

[0046] Reference again Figure 1 The example environment further includes an analysis device 110 that can determine based on analyzing a conversation turn that includes content generated by an application and is followed by providing audio data content to the application. For example, when an application generates content that is provided during a conversation turn that is followed by providing audio data content to the application, the content can be stored in one or more databases such as a conversation turn database 112 for subsequent processing.

[0047] In some implementations, the client device 100 may provide content from a conversation turn as it occurs or in periodic batches, wherein the conversation turn is followed by providing subsequent audio data content to the application that generated the content. Figure 2 The content of the conversation turn 210 may be provided to the analysis device 110 in real time (e.g., immediately before the content is rendered), or may be stored on the client device 100 for a period of time and communicated with, for example, Figure 2 100 (and / or with turns from other conversations). In some implementations, content from conversation turns provided by client devices other than client device 100 can be stored in conversation turn database 112 along with content of conversation turns rendered by client device 100.

[0048] The conversation turn processor 114 can receive the content of the conversation turn and process the content to determine whether it is appropriate to provide audio data content after rendering the conversation turn. Figure 3 , a flowchart illustrating the processing content of one or more conversation turns is provided. As shown, the conversation turn processor 114 can receive the conversation turn content from one or more client devices 100, and / or can identify the conversation turn content from the conversation turn database 112. In some implementations, the conversation turn content can be stored and / or provided together with additional information. For example, in some implementations, the conversation turn content can be associated with an application type, which indicates the category of the application that provides the content of the conversation turn (e.g., a question and answer application, a carpooling application, a restaurant reservation application). The application type can be used to further process the conversation turn, as described in more detail herein. In addition, for example, in some implementations, the conversation turn content can be provided with context information about the conversation of which the conversation turn is a part (e.g., whether the conversation turn is rendered at the beginning, end, or middle of the conversation, information about the content of other conversation turns of the corresponding conversation).

[0049] In some implementations, the conversation turn content may be provided to the conversation turn processor 114 and / or stored in the conversation turn database 112 as a textual representation of synthesized speech provided to the user during the conversation. For example, in some implementations, the application may generate text during the conversation to provide to the user as the conversation turn content, and the automated assistant 120 may utilize the TTS engine 126 to generate the synthesized speech to provide to the user. In some implementations, the conversation turn content may include the intent of the synthesized speech provided to the user as part of the conversation. For example, one or more machine learning models may be utilized to perform natural language understanding on the textual representation of the content of the conversation turn, and the one or more machine learning models generate the intent of the content as output. In some implementations, the conversation turn content may include a vector representation of the content, such as using Word2vec and / or one or more other models to generate a vector embedded in a semantic embedding space.

[0050] In some implementations, a machine learning model (MLM) 121 may be utilized to determine whether it is appropriate to provide audio data content captured after rendering a given conversation turn. For example, the machine learning model 121 may receive as input the content of a conversation turn and optional additional information, such as the type of application that generated the content and / or the context of the conversation, as previously described. The machine learning model 121 may generate as output an indication of whether it is appropriate to provide subsequent audio data content to the application that generated the content. For example, the machine learning model 121 may generate a value between 0.0 and 1.0, indicating whether it is inappropriate to provide the audio data content, where 0.0 indicates a complete determination that it is appropriate to provide the audio data content, and 1.0 indicates a complete determination that it is inappropriate to provide the audio data content (or vice versa, depending on how the model is trained).

[0051] In some implementations, the machine learning model 121 may receive as input a conversation turn content that includes a textual representation of something provided to the user in the conversation. Figure 3, the conversation turn processor 114 may receive the text of the content of the conversation turn from the client device 100 and / or identify the text in the conversation turn database 112, and process the text by providing the text as input to the machine learning model 121. The resulting generated output 320 may be provided to the output analysis module 118 for analysis to determine whether to restrict the provision of audio data content generated from the audio captured in the subsequent interaction between the user and the application to the application that generated the content. In some implementations, the intention of the conversation turn provided to the user by the automated assistant 120 may be provided to the machine learning model 121 as the conversation turn content. In some implementations, as previously described, the client device 100 may provide a vector embedded in a semantic embedding space as the conversation turn content, the vector representing something rendered to the user. The machine learning model 121 may generate the generated output 320 as an output, which may be further analyzed by the output analysis module 118. In some implementations, the dialog turn processor 114 may receive and / or recognize dialog turn content in the form of text, first perform natural language processing on the text, and provide the output of the natural language processing as input to the machine learning model 121. Thus, in implementations where the machine learning model 121 receives vectors as input, the text may be processed using natural language processing to generate vectors by one or more other components, and / or the text may be processed by the dialog turn processor 114 prior to additional processing using the machine learning model 121.

[0052] The machine learning model 121 may be trained with one or more training examples that, when processed, refine the generated output of the machine learning model 121. Figure 3, the training instances 300 each include a conversation turn content 305, which may include content generated by the application and provided to the user via a conversation turn. Each training instance 300 further includes a supervisory indication 310 determined by the selection of the conversation turn content 305. For example, a human can review the content and determine whether it is appropriate to provide subsequent audio data content based on the content. The selector can assign a value (e.g., yes for appropriate, 0 for inappropriate, or vice versa) to indicate whether it is appropriate to provide subsequent audio data content if the conversation turn content 305 is provided in the conversation. In some implementations, the training instance 300 can be generated from content that was previously provided as part of the conversation (e.g., content from the conversation turn database 112 or other source of generated content). In some implementations, one or more other content sources can be used to determine the conversation turn content 305 for the training instance 300. For example, a messaging application and / or discussion board website can be utilized to identify content that may be provided to a user in a conversation, and one or more selectors can determine, based on the content, a supervisory indication 310 to assign to the training instance 300. Optionally, the training instance can include an application type 315, indicating the type of application that generated the conversation turn content. Additionally, optionally, the training instance 300 can include a conversation context 320 that indicates additional information about the conversation including the conversation turn content 305. For example, the conversation context 320 can include content from one or more other conversation turns of the conversation, the position of the conversation turn in the conversation, and / or other information of the conversation beyond the content rendered in the conversation turn.

[0053] The output analysis module 118 can determine whether to restrict the application's access to the audio data content generated from the captured audio data based on the generated output 320. As previously described, the generated output 320 can be, for example, a value between 0.0 (completely confident that the previous provision of audio data to the application 160 was appropriate) and 1.0 (completely confident that the provision of audio data was inappropriate). The output analysis module 118 can determine whether the application inappropriately requested access to the audio data content generated from the audio data captured at least after the rendering of the content when requesting the content to be rendered as a conversation turn based on processing the output generated by the machine learning model 121. If the output analysis module 118 determines that it is inappropriate for the application to request the audio data content, the application can be immediately restricted from subsequent access to the audio data, the application can be marked for manual review, and / or one or more other actions can be performed to prevent the application from continuing to inappropriately receive audio data in other ways.

[0054] In some implementations, the generated output 320 can be utilized to determine whether to increase scrutiny of subsequent requests for access to the audio data content of the audio data captured in response to rendering the content to the application. For example, if the generated output exceeds an inappropriateness threshold, subsequent interactions with the application can be accompanied by an indication rendered to the user indicating that the application is receiving the audio data content. The indication can be, for example, a visual indication that the microphone is active and audio data is being processed, a message provided to the user while the user is interacting with the application, an audio prompt that warns the user that the application may be processing audio data, and / or one or more other indications that more prominently indicate that the application may be processing audio data content than an indication that may be presented with other applications.

[0055] In some implementations, if the generated output 320 meets the inappropriateness threshold, the application can be restricted from accessing the audio data content and / or accessing subsequent audio data content. For example, the automated assistant 120 can first provide the rendered content (and optional additional information, as described herein) to the dialog turn processor 114 for processing before providing the audio data content to the application. If the generated output 320 meets the appropriateness threshold (or does not meet the inappropriateness threshold), the audio data content can be provided to the application. However, if the generated output 320 indicates that it is inappropriate to provide the audio data content after the rendered content, the audio data content may not be transmitted to the application.

[0056] In some implementations, content from multiple conversation turns can be identified, each of which is followed by providing audio data content to the application that generated the content, and the conversation turn processor 114 can be used to process the content from each of the identified conversation turns. For example, the conversation turn database 112 may include content from multiple conversation turns (from a single conversation or from multiple conversations, generated from a single instance of an application or from multiple instances of an application). In some implementations, if the generated output 320 from one or more of the processed content exceeds a threshold, the application can be restricted from obtaining subsequent audio data content. For example, if the generated output 320 is above an inappropriateness threshold of 0.7, the request for audio data content can be considered inappropriate and the application can be restricted. In addition, for example, the application can be restricted from obtaining subsequent audio data content in more than one way, and the generated output 320 can be checked to determine whether the generated output meets multiple thresholds, each threshold having a different restriction method. For example, an application that generates inappropriate content and generates an output 320 of 0.7 may be completely blocked from providing subsequent audio data content, while an application that generates inappropriate content and generates an output 320 of 0.4 may be limited by having a higher alert level to users associated with subsequent instances of providing audio data content. Other examples of limiting the provision of subsequent audio data content to an application may include uninstalling the application and / or instances of the application, alerting users that audio data content is being inappropriately provided, and / or marking the application for further manual review.

[0057] Figure 4 A flowchart depicts an example method for processing conversation turn content using a trained machine learning model. For convenience, the operations of the method are referred to as a system performing these operations, such as Figure 1 The system shown in the method - described. This system of the method includes one or more processors and / or other components of the client device. In addition, although the operations of the method are shown in a particular order, this is not intended to be limiting. One or more operations can be reordered, omitted, or added.

[0058] At step 405, content to be rendered and a request for subsequent audio data content are received from the application. The content and the request may be received by the application. Figure 1 The automated assistant 120 may receive a component that shares one or more characteristics with the automated assistant 120. For example, the automated assistant 120 may receive a request via the application interface 160 to render content that is also received with the request to the user in the conversation. Further, the application may request via the application interface 160 that audio data content from the audio data captured after at least rendering the content be provided to the application.

[0059] At step 410, the content is processed using the trained machine learning model to generate an output. The machine learning model may be used with Figure 1 The machine learning model 121 of the embodiment of the present invention shares one or more characteristics. In some implementations, the component that shares one or more characteristics with the conversation turn processor 114 can provide the content received at step 405 as input to the machine learning model 121. In some implementations, the content can be provided as input to the machine learning model 121 together with additional information. For example, the content can be provided to the machine learning model together with the type of category indicating the application that provides the content, the context information related to the conversation with the user (e.g., where the conversation turn occurs in the conversation), and / or other additional information. The content can be provided as, for example, text data, a vector generated via natural language processing, and / or the intent of the content. In some implementations, the automated assistant 120 can process the content before rendering it in the conversation turn. In some implementations, the automated assistant 120 can first provide the content and then process the content but before providing the subsequent audio data content to the application.

[0060] At step 415, based on the generated output from the machine learning model, it is determined that providing the output is inappropriate. In some implementations, the machine learning model can provide a value indicating the likelihood that providing subsequent audio data content is appropriate given the content generated by the application as the generated output. For example, the generated output can indicate that providing audio data content after rendering the given content is 0.7, thereby indicating that providing is inappropriate. There is a 70% likelihood. If the value exceeds the threshold, then at step 420, the application is restricted from accessing the audio data content.

[0061] Figure 5 A flowchart depicts an example method for processing conversation turn content using a trained machine learning model. For convenience, the operations of the method are referred to as a system performing these operations, such as Figure 1 The system shown in the method - described. This system of the method includes one or more processors and / or other components of the client device. In addition, although the operations of the method are shown in a particular order, this is not intended to be limiting. One or more operations can be reordered, omitted, or added.

[0062] At step 505, conversation turn content is identified that is rendered to the user and followed by providing audio data content to the application generating the content. The conversation turn content can be identified by a component that shares one or more characteristics with the conversation turn processor 114. In some implementations, the conversation turn content can be identified in a database, such as the conversation turn database 112, that stores content generated by one or more applications. In some implementations, the client device 100 can provide content from the rendered conversation turn. The conversation turn content can be provided with additional information, such as described herein, including reference to Figure 4 Step 405.

[0063] At step 510, the content is processed using a machine learning model to generate an output indicating a likelihood that providing the audio data content is appropriate. The processing of the conversation turn content may share one or more characteristics with step 410. For example, content from a conversation turn may be processed and the generated output may indicate a likelihood that providing subsequent audio data content to the application is inappropriate.

[0064] At step 515, it is determined that providing the audio data content to the application is inappropriate. Step 515 may be related to Figure 4 The step 415 of can share one or more characteristics. For example, the generated output from processing a given conversation turn can be compared to a threshold value, and if the generated output exceeds the threshold value, it can be determined that providing the audio data content is inappropriate. Thus, for 10 processed conversation turns, the 10 resulting generated outputs can be used to determine, for example, how much content, when rendered, results in inappropriate provision of audio data content. For example, if a threshold number of processed conversation turns are determined to be inappropriate, then at step 520, the application is restricted from accessing subsequent audio data content.

[0065] Figure 6 A flowchart depicts an example method for processing conversation turn content using a trained machine learning model. For convenience, the operations of the method are referred to as a system performing these operations, such as Figure 1 The system shown in the method - described. This system of the method includes one or more processors and / or other components of the client device. In addition, although the operations of the method are shown in a particular order, this is not intended to be limiting. One or more operations can be reordered, omitted, or added.

[0066] At step 605, content of a plurality of conversation turns is identified that is rendered to the user and followed by providing audio data content to the application generating the content. The content of the conversation turn can be identified by a component that shares one or more characteristics with the conversation turn processor 114. In some implementations, the conversation turn content can be identified in a database, such as the conversation turn database 112, that stores content generated by one or more applications. In some implementations, the client device 100 can provide content from a plurality of rendered conversation turns. The conversation turn content can be provided with additional information, such as described herein, including reference to Figure 4 Step 405 and Figure 5 Step 505.

[0067] At step 610, each of the content is processed using the machine learning model to generate an output for each conversation turn content. The processing of the conversation turn content may share one or more characteristics with step 410 and / or step 510. For example, content from a conversation turn may be processed, and the generated output may indicate a likelihood that providing subsequent audio data content to the application is inappropriate. Each conversation turn may be processed in this manner such that for each conversation turn content that is processed, a corresponding output is generated.

[0068] At step 615, the generated output is used to determine that a threshold value of the output indicates that it is inappropriate to provide the audio data output to the application. Step 615 may be associated with Figure 4 Step 415 and / or Figure 5 The step 515 of can share one or more characteristics. For example, the generated output from processing a given conversation turn can be compared to a threshold, and if the generated output exceeds the threshold, it can be determined that providing the audio data content is inappropriate. Thus, for 10 processed conversation turns, the 10 resulting generated outputs can be used to determine, for example, how much content, when rendered, results in inappropriate provision of audio data content. For example, if a threshold number of processed conversation turns are determined to be inappropriate, then at step 520, the application is restricted from accessing subsequent audio data content.

[0069] Figure 77 is a block diagram of an example computing device 710 that may optionally be used to perform one or more aspects of the techniques described herein. The computing device 710 generally includes at least one processor 714 that communicates with a plurality of peripheral devices via a bus subsystem 712. These peripheral devices may include a storage subsystem 724 (including, for example, a memory subsystem 725 and a file storage subsystem 726), a user interface output device 720, a user interface input device 722, and a network interface subsystem 716. The input and output devices allow a user to interact with the computing device 710. The network interface subsystem 716 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0070] The user interface input device 722 may include: a keyboard; a pointing device such as a mouse, trackball, touch pad, or graphic tablet, a scanner, a touch screen incorporated into a display; an audio input device such as a voice recognition system, a microphone; and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways of inputting information into the computing device 710 or into a communication network.

[0071] User interface output device 720 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide a non-visual display such as via an audio output device. Generally speaking, the use of the term "output device" is intended to include devices and methods for outputting information from computing device 710 to all possible types of users or another machine or computing device.

[0072] The storage subsystem 724 stores programming and data structures that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 724 may include a computer program that executes Figures 4 to 6 Selected aspects of the method and / or for implementing Figure 1 and Figure 3 The logic of the various components depicted in .

[0073] These software modules are typically executed by the processor 714 alone or in combination with other processors. The memory 725 used in the storage subsystem 724 may include multiple memories, including a main random access memory (RAM) 730 for storing instructions and data during program execution and a read-only memory (ROM) 732 in which fixed instructions are stored. The file storage subsystem 726 may provide persistent storage for program and data files, and may include a hard drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of certain implementations may be stored by the file storage subsystem 726 in the storage subsystem 724 or in other machines accessible by the processor 714.

[0074] The bus subsystem 712 provides a mechanism for the various components and subsystems of the computing device 710 to communicate with each other as intended. Although the bus subsystem 712 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0075] The computing device 710 can be of different types, including a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Figure 7 The depicted description of computing device 710 is intended only as a specific example for purposes of illustrating some implementations. Many other configurations of computing device 710 are possible, and these configurations are similar to those of FIG. Figure 7 Computing devices are depicted as having more or fewer components than those shown.

[0076] Implementations disclosed herein include a method comprising: during a conversation session occurring via a client device, between a user and an application, and mediated by an automated assistant: receiving, by the automated assistant, from the application: content to be rendered by the automated assistant on behalf of the application in a conversation turn of the conversation session, and providing to the application a request for audio data content corresponding to audio data captured by a microphone of the client device during and / or after rendering of the content in the conversation turn. The method further includes processing the content using a trained machine learning model to generate an output based on the content and indicating whether it is appropriate to provide the audio data content to the application. In response to determining that providing the audio data content is inappropriate, the method includes limiting the application's access to the audio data content.

[0077] These and other implementations of the techniques disclosed herein may include one or more of the following features.

[0078] In some implementations, processing the content is performed by one or more other computing devices in addition to the client device.

[0079] In some implementations, determining that providing the audio data is inappropriate includes determining that the output does not satisfy an appropriateness threshold. In some of those implementations, the method further includes selecting the appropriateness threshold from a plurality of candidate appropriateness thresholds based on one or more attributes of the application. In some of those implementations, the one or more attributes of the application on which the appropriateness threshold is selected include a type of application. In some of those implementations, the method further includes selecting the appropriateness threshold from a plurality of candidate appropriateness thresholds based on a data security level previously specified for the client device and / or for the user.

[0080] In some implementations, limiting the audio data includes disabling an application.

[0081] In some implementations, prior to a conversation session, the method includes training a machine learning model using supervised training instances, each supervised training instance comprising: corresponding content rendered during a corresponding previous conversation turn with a corresponding user, and a corresponding supervisory indication of whether audio data content is appropriate in response to rendering of the corresponding content.

[0082] Other implementations disclosed herein include a method, comprising: rendering during a conversation turn based on content of a conversation turn and receiving a message based on an application generating the content; arrive identifying content based on audio data content corresponding to audio data captured by a microphone of a rendering client device at least after the content is rendered in the conversation turn; processing the content of the conversation turn using a machine learning model to generate an output that is based on the content and indicates whether it is appropriate to provide the audio data content to the application; and in response to determining that the generated output indicates that it is inappropriate to provide the audio data content to the application: restricting access by one or more application instances of the target application to subsequent audio data content corresponding to the subsequent audio data.

[0083] These and other implementations of the techniques disclosed herein may include one or more of the following features.

[0084] In some implementations, processing the content of the conversation turn includes determining an application type of the target application and providing the application type as input to a machine learning model.

[0085] In some implementations, processing the content of the conversation turn includes identifying a textual representation of the content of the conversation turn; and providing the textual representation of the content of the conversation turn as input to a machine learning model.

[0086] In some implementations, processing the content of the conversation turn includes processing the content of the conversation turn to determine the intent of the content of the conversation turn, and providing the intent of the content of the conversation turn as an input to the machine learning model. In some of those implementations, processing the content of the conversation turn includes processing the content of the conversation turn using an additional machine learning model to generate a vector, and providing the vector as an input to the machine learning model.

[0087] In some implementations, the method further includes identifying a context of a conversation turn in the conversation, and providing the context of the conversation turn as an additional input to the machine learning model. In some implementations, the context indicates that the conversation turn is a terminating conversation turn in the conversation. In other of those implementations, the context indicates the content of one or more previous and / or subsequent conversation turns of the conversation.

[0088] Still other implementations disclosed herein include a method, comprising: identifying content of each of a plurality of conversation turns based on the following: the content is rendered during a corresponding conversation turn, and then an instance of an application generating the corresponding content receives audio data content captured as audio data by a microphone of a corresponding rendering client device at least after the corresponding content is rendered; processing the content of the conversation turn using a trained machine learning model to generate an output that is based on the corresponding content and indicates whether it is appropriate to provide the audio data content to the application; and in response to determining that a threshold of the generated output indicates that it is inappropriate to provide at least a portion of the audio data content to the application: causing restriction on obtaining subsequent audio data content by one or more current application instances of the application.

[0089] These and other implementations of the techniques disclosed herein may include one or more of the following features.

[0090] In some implementations, if a given generated output exceeds a threshold, then the given generated output indicates that providing the corresponding audio data content to the application is inappropriate.

[0091] In some implementations, multiple conversation turns are part of a single conversation.

[0092] In some implementations, the plurality of conversation turns are provided by a single instance of a target application among the one or more instances of a target application.

[0093] In some implementations, the method further includes identifying an application type of the target application; and determining a threshold value for the generated output based on the application type.

[0094] In some implementations, the method further includes identifying a third party associated with the application, and determining a threshold for the generated output based on the third party.

[0095] Where certain implementations discussed herein may collect or use personal information about a user (e.g., user data extracted from other electronic communications, information about a user's social network, a user's location, a user's time, a user's biometric information, and a user's activities and demographic information, relationships between users, etc.), the user is provided with one or more opportunities to control whether information is collected, whether personal information is stored, whether personal information is used, and how information about the user is collected, stored, and used. That is, the systems and methods discussed herein collect, store, and / or use user personal information only after receiving explicit authorization from the relevant user to collect, store, and / or use the user's personal information.

[0096] For example, a user is provided with control over whether a program or feature collects user information about that particular user or other users associated with the program or feature. One or more options are presented to each user whose personal information is to be collected to allow control over the collection of information associated with that user, providing permission or authorization as to whether to collect information and as to which portions of the information are to be collected. For example, one or more such control options may be provided to a user via a communications network. In addition, specific data may be processed in one or more ways before it is stored or used so that personally identifiable information is removed. As an example, the identity of a user may be processed so that no personally identifiable information can be determined. As another example, a user's geographic location may be generalized to a larger area so that the user's specific location cannot be determined.

[0097] Although several implementations have been described and shown herein, various other means and / or structures for performing functions and / or obtaining results and / or one or more advantages described herein may be used, and each of such changes and / or modifications is considered to be within the scope of the implementations described herein. More generally, all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and the actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications for which this teaching is used. Those skilled in the art will recognize or confirm many equivalents of the implementations described herein using only routine experiments. Therefore, it should be understood that the aforementioned implementations are presented only by way of example, and it should be understood that within the scope of the attached claims and their equivalents, implementations may be practiced in a manner other than specifically described and claimed. The implementations disclosed herein relate to each individual feature, system, article, material, kit, and / or method described herein. In addition, if such features, systems, articles, materials, kits, and / or methods do not contradict each other, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure.

Claims

1. A method comprising: During a conversational session occurring between a user and an application via a client device and mediated by an automated assistant: Received by the automated assistant from the application: content to be rendered by the automated assistant on behalf of the application in a conversation turn of the conversation session, and providing, to the application, a request for audio data content corresponding to audio data captured by a microphone of the client device during and / or after rendering of the content in the conversation turn; processing the content using a trained machine learning model to generate an output that is based on the content and indicates whether it is appropriate to provide the audio data content to the application; as well as In response to determining that providing the audio data content is inappropriate: Access to the audio data content by the application is restricted.

2. The method of claim 1, wherein processing the content is performed by one or more other computing devices in addition to being performed by the client device.

3. A method as claimed in any preceding claim, wherein determining that providing audio data is inappropriate comprises determining that the output does not satisfy an appropriateness threshold.

4. The method of claim 3, further comprising: The suitability threshold is selected from a plurality of candidate suitability thresholds based on one or more properties of the application.

5. The method of claim 4, wherein the one or more attributes of the application on which the suitability threshold is selected include a type of the application.

6. The method of claim 3, further comprising: The suitability threshold is selected from a plurality of candidate suitability thresholds based on a data security level previously specified for the client device and / or for the user.

7. A method as claimed in any preceding claim, wherein limiting the audio data comprises disabling the application.

8. The method of any preceding claim, further comprising, prior to the conversation session: The machine learning model is trained using supervised training instances, each of which includes: the corresponding content rendered during the corresponding previous conversation turn with the corresponding user, and A corresponding supervisory indication of whether the audio data content is appropriate is provided in response to rendering of the corresponding content.

9. A method comprising: identifying the content based on content of the conversation turn being rendered during the conversation turn and based on an application generating the content receiving audio data content corresponding to audio data captured by a microphone of a rendering client device at least after the content was rendered in the conversation turn; processing the content of the conversation turn using a machine learning model to generate an output that is based on the content and indicates whether it is appropriate to provide the audio data content to the application; as well as In response to determining that the generated output indicates that providing the audio data content to the application is inappropriate: Access by one or more application instances of the target application to subsequent audio data content corresponding to the subsequent audio data is restricted.

10. The method of claim 9, wherein processing the content of the conversation turn comprises: Determining an application type of the target application; as well as The application type is provided as input to the machine learning model.

11. The method of any one of claims 9 and 10, wherein processing the content of the conversation turn comprises: identifying a textual representation of the content of the conversational turn; as well as The textual representation of the content of the conversation turn is provided as input to the machine learning model.

12. The method of any one of claims 9 to 11, wherein processing the content of the conversation turn comprises: processing the content of the conversation turn to determine the intent of the content of the conversation turn; as well as The intent of the content of the conversation turn is provided as input to the machine learning model.

13. The method of any one of claims 9 to 12, wherein processing the content of the conversation turn comprises: Processing the content of the conversation turn using an additional machine learning model to generate a vector; as well as The vector is provided as input to the machine learning model.

14. The method of any one of claims 9 to 13, further comprising: identifying the context of the conversation turn in the conversation; as well as The context of the conversation turn is provided as additional input to the machine learning model.

15. The method of claim 14, wherein the context indicates that the conversation turn is a terminating conversation turn in the conversation.

16. The method of claim 14, wherein the context indicates the content of one or more previous and / or subsequent conversation turns of the conversation.

17. A method comprising: identifying content of a plurality of conversation turns based on: the content of each of the conversation turns being rendered during a corresponding one of the conversation turns and followed by an instance of an application generating the corresponding content receiving audio data content captured as audio data by a microphone of a corresponding rendering client device at least after the corresponding content was rendered; processing the content of the conversation turn using a trained machine learning model to generate an output that is based on the corresponding content and indicates whether it is appropriate to provide the audio data content to the application; as well as In response to determining that a threshold of the generated output indicates that providing at least a portion of the audio data content to the application is inappropriate: Causing restriction on obtaining subsequent audio data content by one or more current application instances of the application.

18. The method of claim 17, wherein if a given generated output exceeds a threshold, the given generated output indicates that providing corresponding audio data content to the application is inappropriate.

19. The method of any one of claims 17 and 18, wherein the plurality of conversation turns are parts of a single conversation.

20. The method of any one of claims 17 to 19, wherein the plurality of conversation turns are provided by a single instance of the target application among the one or more instances of the target application.

21. The method of any one of claims 17 to 20, further comprising: Identifying an application type of the target application; as well as The threshold of the generated output is determined based on the application type.

22. The method of any one of claims 17 to 21, further comprising: Identify third parties associated with the application; as well as The threshold of the generated output is determined based on the third party.

23. A computer program comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method of any preceding claim.

24. One or more computing devices configured to perform the method of any one of claims 1 to 22.

Images