Generating unintentional memory metrics for automatic speech recognition models

CN119072742APending Publication Date: 2024-12-03GOOGLE LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280093644.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-31
Filing Date
2022-10-20
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Automatic speech recognition (ASR) models may unintentionally remember sensitive information in the training data during training, resulting in potential data breaches.

Method used

By generating a collection of candidate transcripts and calculating unintentional memory metrics, the phrases that the ASR model remembers unintentionally during training are identified, and the speech synthesis model and ASR model are compared to the ASR model to determine whether the model has memorized a specific transcript, thereby evaluating the unintentional memory of the model.

Benefits of technology

Effectively identify and evaluate the unintentional memory of the ASR model, reduce the risk of sensitive information leakage, and improve the security of the model and user data protection capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072742A_ABST
    Figure CN119072742A_ABST
Patent Text Reader

Abstract

An unintentional memory metric may be used to determine whether an automatic speech recognition (ASR) model unintentionally memorizes one or more phrases during training of the ASR model. Various implementations include generating one or more candidate transcriptions based on a vocabulary of an ASR model. For example, the system may generate candidate transcriptions by attaching lemnes of a word list to previous candidate transcriptions. Various implementations include processing the candidate transcriptions using a speech synthesis model to generate synthetic speech audio data including synthetic speech of the candidate transcriptions. Additionally or alternatively, the synthetic speech audio data may be processed using an ASR model to generate an ASR output. Various implementations may include generating a loss based on comparing the ASR output to candidate transcriptions.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Voice-based user interfaces are increasingly used to control computers and other electronic devices. Voice-based user interfaces have evolved from early basic interfaces that can only understand simple and direct commands to more complex interfaces that respond to natural language requests and can understand context and manage back-and-forth conversations or conversations with users. Many voice-based user interfaces perform speech recognition (e.g., using an automatic speech recognition model) on spoken utterances to generate corresponding text, perform semantic analysis on the text to attempt to determine the meaning of the spoken utterances and / or take one or more actions based on the determined meaning. However, the automatic speech recognition model may unintentionally memorize the underlying data used when training the model. Summary of the invention

[0002] Implementations described herein are intended to generate an unintentional memory metric for an automatic speech recognition ("ASR") model that indicates the extent to which the ASR model unintentionally memorizes one or more phrases during training. While training, the ASR model may unintentionally memorize rare or unique sequences in the training data. In some implementations, the unintentionally memorized sequences may include sensitive information about the user. For example, the ASR model may unintentionally memorize a training instance based on a user saying the utterance "Myusername is PatentsForLife and my password is I Love Patents". After training the ASR model, the system may process additional instances of a user (the same user who said the training instance and / or additional users) saying the utterance "My username isPatentsForLife and my password is I Love Patents" to generate a text representation of "Myusername is PatentsForLife and my password is I Love Patents". In some implementations, the ASR model may generate a text representation by generating one or more probabilities that indicate the probability of a word being captured in a spoken utterance. For example, an ASR model may generate several candidate text representations of an utterance, and may select a text representation of the utterance based on the generated probabilities.

[0003] When the ASR model unintentionally memorizes a training utterance, the ASR model may generate a very high probability corresponding to the textual representation of the memorized utterance. For example, the system may determine that the ASR model has unintentionally memorized the textual representation when the probability of whether the textual representation is captured in the utterance meets a threshold. For example, the system may determine that the ASR model has unintentionally memorized the textual representation when the corresponding probability is greater than 0.99, greater than 0.95, greater than 0.9, greater than 0.85, meets one or more additional thresholds, and / or a combination thereof. In other words, when the probability generated using the ASR model is very high, the system may determine that the ASR model has unintentionally memorized the textual representation.

[0004] In some implementations, the system may determine that the probability of the textual representation of “My username is PatentsForLife and my password is I Love Patents” (e.g., generated by processing audio data capturing the utterance using an ASR model) is 0.98. In some of those implementations, the system may determine whether 0.98 meets a threshold of 0.95, where a probability greater than the threshold indicates that the ASR model has unintentionally memorized the utterance. In response to determining that the probability 0.98 exceeds the threshold of 0.95, the system may determine that the ASR model has unintentionally memorized the utterance “My username isPatentsForLife and my password is I Love Patents”.

[0005] In some implementations, when generating an unintentional memory metric, the system may generate a set of candidate transcriptions, each candidate transcription indicating a potential unintentional memory of the ASR model. The system may generate a set of candidate transcriptions based on a vocabulary of the ASR model. In some implementations, the candidate transcriptions may be generated based on tokens of a previous candidate transcription and a vocabulary of the ASR model. The tokens of the vocabulary may include one or more words, one or more word-pieces, one or more phonemes, one or more letters, one or more additional parts of the vocabulary, and / or combinations thereof. For example, the system may generate a candidate transcription by appending a word in the vocabulary to a previous candidate transcription (e.g., a transcription that satisfies one or more criteria). Additionally or alternatively, the system may generate multiple candidate transcriptions based on the same previous candidate transcription by augmenting different parts of the vocabulary to each candidate transcription.

[0006] As an illustrative example, the system may generate multiple candidate transcriptions based on a prior candidate transcription, "what does the". A first candidate transcription, "what does the mouse", may be generated based on appending the word "mouse" to a prior candidate transcription, "what does the". Similarly, a second candidate transcription, "what does the fish", may be generated based on appending the word "fish" to a prior candidate transcription, "what does the". Further, a third candidate transcription, "what does the seal", may be generated based on appending the word "seal" to a prior candidate transcription, "what does the". One or more additional or alternative candidate transcriptions may be generated based on appending additional words to a prior candidate transcription.

[0007] In some implementations, the candidate transcriptions may be processed using a speech synthesis model to generate synthesized speech audio data including synthesized speech of the candidate transcription. The audio data may be processed using an ASR model to generate an ASR output (e.g., a candidate text representation of the synthesized speech, a probability metric corresponding to the candidate text representation of the synthesized speech, a corresponding probability metric for each word of the candidate text representation, etc.). In some implementations, a loss may be generated based on comparing the candidate transcription to the corresponding ASR output.

[0008] To advance the previous example, a first candidate transcription, “what does the mouse,” may be processed using a speech synthesis model to generate first synthesized speech audio data that includes a synthesized verbal representation of “what does the mouse.” Similarly, a second candidate transcription, “what does the fish,” may be processed using a speech synthesis model to generate second synthesized speech audio data that includes a synthesized verbal representation of “what does the fish.” Additionally or alternatively, a third candidate transcription, “what does the seal,” may be processed using a speech synthesis model to generate third synthesized speech audio data that includes a synthesized verbal representation of “what does the seal.”

[0009] In some implementations, the system may use an ASR model to process synthesized speech audio data including synthesized speech of candidate transcriptions to generate corresponding ASR outputs. For example, the system may use an ASR model to process first synthesized speech audio data to generate a first ASR output including a first candidate text representation "what does the mouse"; the system may use an ASR model to process second synthesized speech audio data to generate a second ASR output including a second candidate text representation "what does the fish"; and the system may use an ASR model to process third synthesized speech audio data to generate a third ASR output including a third candidate text representation "what does the seal".

[0010] In some implementations, the system can determine the loss based on comparing the candidate transcriptions to the ASR outputs. For example, the system can determine a first loss of 0.05 based on comparing the first candidate transcription "what does the mouse" to the first ASR output; the system can determine a second loss of 0.1 based on comparing the second candidate transcription "what does the fish" to the second ASR output; and the system can determine a third loss of 0.01 based on comparing the third candidate transcription "what does the seal" to the third ASR output.

[0011] According to some implementations, the system can use the generated losses in a variety of ways. For example, the system can determine whether to use the candidate transcription to generate further candidate transcriptions by expanding the candidate transcription with one or more additional word-grams from the vocabulary of the ASR model. In some implementations, the system can compare the losses of multiple candidate transcriptions and select one or more of the candidate transcriptions where the losses meet one or more conditions (such as the smallest loss, the smallest two losses, the smallest three losses, one or more additional conditions, and / or combinations thereof).

[0012] For example, the system may compare the above losses of a first loss of 0.05, a second loss of 0.1, and a third loss of 0.01, and based on the minimum loss of 0.01, may select the third candidate transcription "what does the seal" for additional expansion. For example, the system may expand the third candidate transcription "what does theseal" with additional words from the vocabulary of the ASR model to generate a further candidate transcription "what does the seal say". In some implementations, the system may generate multiple further candidate transcriptions, each of which is expanded with a different word from the vocabulary. For example, the system may generate a first further candidate transcription "what doesthe seal say" by expanding "what does the seal" with the word "say"; the system may generate a second further candidate transcription "what does the seal eat" by expanding "what does the seal" with the word "eat"; and the system may generate a third further candidate transcription "what doesthe seal dig". In some of those implementations, the system may process each further candidate transcription using a speech synthesis model to generate corresponding further synthesized speech audio data, and may process the further synthesized speech audio data using an ASR model to generate corresponding further ASR output. Additionally or alternatively, the system may determine a further loss corresponding to each further candidate transcription based on a comparison of the further candidate transcription with the corresponding further ASR loss. The further loss may be used in a variety of ways described herein.

[0013] Additionally or alternatively, the system may use the loss corresponding to a particular candidate transcription to determine whether the ASR model has inadvertently memorized one or more occurrences of a user saying the candidate transcription in the training data used to train the ASR model. For example, the system may identify a set of candidate transcriptions based on the corresponding losses, such as identifying a set of candidate transcriptions with the smallest corresponding losses as described above. In some implementations, the system may compare each identified candidate transcription with a set of training instances used to train the ASR model. The candidate transcriptions corresponding to the training instances may indicate that the ASR model may have inadvertently memorized the training instances. For example, the system may compare the candidate transcription "what does the seal say" with the training instances used to train the ASR model. The system may identify one or more occurrences (or non-occurrences) of a user saying the spoken utterance "whatdoes the seal say" in the training instances.

[0014] In some implementations, the memory metric can be based on the number of candidate transcriptions corresponding to one or more occurrences of a user uttering the candidate transcription in training data used to train the ASR model. For example, the unintentional memory metric can be based on the number of candidate transcriptions corresponding to one or more training instances. In some of those implementations, a lower memory metric indicates less unintentional memory of the ASR model.

[0015] The system may use the memory metric to evaluate one or more ASR models. In some implementations, the system may use the memory metric to compare several ASR models. For example, the system may determine a first memory metric corresponding to a first ASR model trained using a first set of training instances, and a second memory model corresponding to a second ASR model trained using a second set of training instances. The system may compare unintentional memory in the first ASR model compared to the second ASR model based on a comparison of the first memory metric and the second memory metric.

[0016] Additionally or alternatively, the system can use the memory metric to determine whether the additional training changes (e.g., increases, decreases, does not change, etc.) the unintentional memory of the ASR model. For example, the system can generate a prior memory metric corresponding to the ASR model before training the ASR model with the additional training. A subsequent memory metric can be generated corresponding to the ASR model before the additional training. In some implementations, the additional training of the ASR model can be evaluated based on a comparison of the prior memory metric and the subsequent memory metric.

[0017] In some implementations, the ASR model may be trained based on one or more federated learning techniques. For example, the on-device ASR model may be stored locally at the user's client device, while the cloud-based global ASR model may be stored remotely at a remote system (e.g., in a server, in a cluster of servers, etc.). The client device may process input detected at the client device to generate a prediction, and may compare the prediction with a ground truth output to generate a client gradient. Additionally or alternatively, the client device may transmit client ASR model updates based on client gradients to the remote system. The remote system may update one or more weights of the global ASR model using the client model updates and optional additional client ASR model updates from additional client devices. In some implementations, the system may transmit the updated global ASR model and / or the updated weights of the global ASR model to the client device and / or other client devices. Each client device may replace the on-device ASR model with the global ASR model, or replace the weights of the on-device ASR model with the updated weights of the global ASR model, thereby updating the on-device ASR model.

[0018] In some implementations, the system may determine an initial memory metric before updating the global ASR model based on the client ASR model update, the initial memory metric indicating one or more unintentional memories of the global ASR model. After updating one or more weights of the global ASR model, the system may determine an updated memory metric. In some implementations, the system may determine whether to transmit an updated global ASR model and / or updated weights of the global ASR model based on comparing the initial memory metric to the updated memory metric.

[0019] In some of those implementations, the system may determine to transmit an updated global ASR model based on comparing an initial memory metric to an updated memory metric. For example, the system may determine that the initial memory metric value is 10, and may determine that the updated memory metric is 5, which may indicate that there is less unintentional memory in the updated global ASR model compared to the initial global ASR model. In some implementations, the system may transmit the updated global ASR model based on determining that the updated global ASR model has reduced the amount of unintentional memory. Additionally or alternatively, the system may transmit the updated global ASR model based on determining that the updated global ASR model has not changed the amount of unintentional memory, has increased the amount of unintentional memory within a threshold, etc.

[0020] In some other implementations, the system may determine not to transmit the updated global ASR model based on comparing the initial memory metric to the updated memory metric. For example, the system may determine that the initial memory metric is 100, and may determine that the updated memory metric is 5000, which may indicate that there is more unintentional memory in the updated global ASR model compared to the initial global ASR model. In other words, the increased memory metric of the updated memory metric may indicate that the ASR model is more likely to inadvertently leak sensitive data. In some of those implementations, the system may determine not to update the client ASR model based on determining that the updated global ASR model has an increased amount of unintentional memory compared to the initial ASR model.

[0021] In addition, in some implementations, one or more target training instances may be inserted into the training data for training the ASR model. The target training instances may include atypical phrases, phrases that may be unintentionally memorized, phrases memorized by other systems, and the like. In some implementations, the system may compare candidate transcriptions with target training instances to determine whether the ASR model has unintentionally memorized the target training instances during training. For example, the target training instance "My name is John Doe and my credit card number is 1234567890" may be used when training the ASR model. The system may generate a candidate transcription set as described herein. In some of those implementations, the system may determine whether a candidate transcription "Myname is John Doe and my credit card number is 1234567890" has been generated, which indicates that the ASR model has memorized the target training instance.

[0022] In some implementations, the system may compare the probability generated using the ASR model for a given candidate transcription with a metric generated using an additional ASR model for the given candidate transcription. For example, the metric may be based on one or more probabilities, each of which is a corresponding probability generated using the corresponding additional ASR model for the given candidate transcription. For example, the system may compare the probability generated by the ASR model indicating the likelihood of a given candidate transcription being captured in the audio data with a further probability generated using the additional ASR model indicating the likelihood of the candidate transcription being captured in the audio data. If the candidate transcription has a high probability across multiple ASR models, the candidate transcription is less likely to indicate an unintentional memory, and / or on the contrary is more likely to indicate that the candidate transcription is a common spoken phrase. In other words, when determining whether the probability generated using the ASR model for a given candidate transcription indicates an unintentional memory, the probability may be calibrated based on one or more other probabilities similarly generated for the given candidate transcription using one or more additional ASR models. In these and other embodiments, whether a given probability indicates an unintentional memory may vary from transcription to transcription based on calibration specific to the corresponding transcription and based on one or more other probabilities from other ASR models.

[0023] As a specific example, audio data capturing the spoken utterance “turn off the kitchen lights” may be processed using a first ASR model to generate a candidate transcription “turn off the kitchen lights” with a corresponding probability of 0.98. Similarly, audio data capturing the spoken utterance “turn off the kitchen lights” may be processed using a second ASR model to generate a candidate transcription “turn off the kitchen lights” with a corresponding probability of 0.99. Further, audio data capturing the spoken utterance “turn off the kitchen lights” may be processed using a third ASR model to generate a candidate transcription “turn off the kitchen lights” with a corresponding probability of 0.997. The candidate transcription has a high probability (0.98) corresponding to the first ASR model, a high probability (0.99) corresponding to the second ASR model, and a high probability (0.997) corresponding to the third ASR model. In some implementations, this high probability across multiple ASR models indicates that the candidate transcription “turn off the kitchen lights” is a common spoken phrase and / or is not an unintentional memory of one or more of the ASR models.

[0024] As an example, the system may compare the probabilities generated using additional ASR models (e.g., a second ASR model, a third ASR, one or more additional ASR models, etc.) to determine an expected probability metric. For example, the expected probability metric may be an average probability metric (e.g., 0.9935, the average of the second ASR model probability metric (0.99) and the third ASR probability metric (0.997)), a mean probability metric, a mean probability metric, a weighted probability metric, one or more probability metrics, and / or a combination thereof. The system may compare the expected probability metric with a probability metric corresponding to the probability metric generated using the ASR model being processed by the system to identify unintentional memory. In some implementations, the system may determine whether the ASR model has unintentionally memorized the candidate transcription based on determining whether the corresponding probability metric and the expected probability metric satisfy one or more conditions (such as whether the corresponding metric is within a threshold distance of the expected probability metric). In some implementations, if the difference between the corresponding probability metric and the expected probability metric is greater than a threshold (e.g., the difference is greater than 0.5, 0.2, 0.1, etc.), the system may determine that the ASR model has unintentionally memorized the candidate transcription. In some other implementations, if the difference between the corresponding probability metric and the expected probability metric is less than a threshold (e.g., the difference is less than 0.05, 0.1, 0.15, 0.2, etc.), the system can determine that the ASR model is not inadvertently memorizing the candidate transcription.

[0025] For example, the system can compare the probability metric (0.98) of the first ASR model to an expected probability metric (0.9935) based on the average of the probability metrics of the second ASR model and the third ASR model. Additionally or alternatively, the system can determine whether the difference (0.0035) between the corresponding probability metric and the expected probability metric is greater than or less than a threshold value of 0.1. In response to determining that the difference (0.0035) is less than the threshold value (0.1), the system can determine that the first ASR model is not inadvertently memorizing the candidate transcription.

[0026] Conversely, audio data capturing the spoken utterance “my username is'PatentsAreTheBest' and my password is'ILovePatents'” may be processed using a first ASR model to generate a candidate transcription “my username is'PatentsAreTheBest' and my password is'ILovePatents'” with a corresponding probability of 0.999. Additionally or alternatively, audio data capturing the spoken utterance “myusername is'PatentsAreTheBest' and my password is'ILovePatents'” may be processed using a second ASR model to generate a candidate transcription “my username is'PatentsAreTheBest' and mypassword is'ILovePatents'” with a corresponding probability of 0.7. Additionally or alternatively, audio data capturing the spoken utterance “my username is 'PatentsAreTheBest' and my password is 'ILovePatents'” may be processed using a third ASR model to generate a candidate transcription “my username is 'PatentsAreTheBest'and my password is 'ILovePatents'” with a corresponding probability of 0.79. In some implementations, the high probability corresponding to the first ASR model (e.g., 0.999) and the (relatively) low probabilities corresponding to the second and third ASR models (e.g., 0.7 and 0.79, respectively) may indicate that the first ASR model has inadvertently memorized one or more training instances of capturing the user saying “my username is'PatentsAreTheBest' and my password is 'ILovePatents'”.

[0027] As an additional example, the system may compare probabilities generated using additional ASR models (e.g., a second ASR model, a third ASR model, one or more additional ASR models, etc.). For example, the expected probability may be an average probability metric, a mean probability metric (e.g., 0.745, the average of the second ASR model probability metric (0.7) and the third ASR probability metric (0.79)), a weighted probability metric, one or more probability metrics, and / or a combination thereof. Similarly, the system may compare the expected probability metric with the probability metric generated using the ASR model being processed by the system to identify unintentional memory. In some implementations, the system may determine whether the ASR model has unintentionally memorized the candidate transcription based on determining whether the corresponding probability metric and the expected probability metric satisfy one or more conditions (such as whether the corresponding metric is within a threshold distance of the expected probability metric). In some implementations, if the difference between the corresponding probability metric and the expected probability metric is greater than a threshold (e.g., the difference is greater than 0.5, 0.2, 0.1, etc.), the system may determine that the ASR model has unintentionally memorized the candidate transcription. In some other implementations, if the difference between the corresponding probability metric and the expected probability metric is less than a threshold (e.g., the difference is less than 0.05, 0.1, 0.15, 0.2, etc.), the system can determine that the ASR model is not inadvertently memorizing the candidate transcription.

[0028] For example, the system can compare the probability of the first ASR model (0.999) to an expected probability metric (0.745) based on the average of the probabilities of the second ASR model and the third ASR model. In some implementations, the system can determine whether the difference (0.254) between the corresponding probability metric and the expected probability metric is greater than or less than a threshold value of 0.2. In response to determining that the difference (0.254) is greater than the threshold value (0.2), the system can determine that the first ASR model has inadvertently memorized the candidate transcription.

[0029] Although the previous examples are described with respect to comparing a probability metric generated using a given ASR model with one or more additional probability metrics generated using one or more ASR models, this is merely illustrative. In some implementations, the system may compare one or more portions of an ASR output (such as an output generated by processing synthetic speech audio data including a candidate transcription using an ASR model) with additional ASR outputs generated based on processing the synthetic speech audio data using one or more additional ASR models. The ASR output may include a probability indicating the likelihood that the synthetic speech audio data includes the candidate transcription, a confidence score indicating the confidence of the ASR output, a representation of the loss generated when training the ASR model, one or more additional or alternative outputs, and / or a combination thereof. For example, the loss generated when training a first ASR model may be compared with the loss generated when training a second ASR model and the loss generated when training a third ASR model (e.g., an average loss based on the loss corresponding to the second ASR model and the loss corresponding to the third ASR model). In some of those implementations, the system can determine whether a candidate transcription is unintentionally memorized by the first ASR model based on determining whether a difference between a loss corresponding to the first ASR model and an average loss (based on the loss corresponding to the second ASR model and the loss corresponding to the third ASR model) satisfies one or more conditions, such as, whether the difference satisfies a threshold (as described above with respect to the probability metric).

[0030] In some implementations, the system can generate a set of potential candidate transcriptions that may have been unintentionally memorized by an ASR model of a third party. For example, the system can generate a set of potential candidate transcriptions with high corresponding probabilities using a third-party ASR model. The third party can determine whether each of the candidate transcriptions in the set of potential candidate transcriptions is an unintentional memory of the third-party ASR model.

[0031] Therefore, various implementations set forth techniques for identifying unintentional memories in ASR models. ASR models can be trained with real-world training examples. These training examples may unintentionally include sensitive information (e.g., passwords, credit card numbers, birthdays, etc.). Unintentional memories of ASR models may unintentionally leak the sensitive information. Various implementations described herein may be used to identify a set of candidate transcriptions that may indicate unintentional memories of the corresponding ASR model. In some of those implementations, a memory metric for an ASR model may be generated based on the candidate transcription set. For example, a memory metric may be used to identify whether an ASR model may leak personal information if deployed. Additionally or alternatively, users may more confidently share interactions with the system to further improve the ASR model by preventing unintentional leakage of sensitive information.

[0032] The above description is provided only as an overview of some implementations disclosed herein. These and other implementations of the technology will be disclosed below in additional detail. It should be understood that all combinations of the aforementioned concepts and additional concepts described in more detail herein are considered part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are considered part of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A An example of generating a set of candidate transcriptions according to various implementations disclosed herein is shown.

[0034] Figure 1B Additional examples of generating a set of candidate transcriptions according to various implementations disclosed herein are shown.

[0035] Figure 1C Examples of generating memory metrics according to various implementations disclosed herein are shown.

[0036] Figure 2 is a block diagram illustrating an example of generating a set of candidate transcriptions according to various implementations disclosed herein.

[0037] Figure 3 An example environment is shown in which various implementations described herein may be implemented.

[0038] Figure 4 is a flow diagram illustrating a process for performing actions based on a set of candidate transcriptions and / or a memory metric according to various implementations disclosed herein.

[0039] Figure 5 is a flow chart illustrating a process of determining whether an ASR model has inadvertently memorized an occurrence of a person uttering a candidate transcription in accordance with various implementations disclosed herein.

[0040] Figure 6 Another example environment is shown in which various implementations disclosed herein may be implemented.

[0041] Figure 7 An example architecture of a computing device is shown. DETAILED DESCRIPTION

[0042] Now turning to the attached figure, Figure 1AAn example 100 of generating a set of candidate transcriptions according to some implementations is shown. Example 100 includes processing a prior candidate transcription 102 using an augmentation engine 104 to generate a candidate transcription 106. In some implementations, at an initial iteration, the prior candidate transcription may be seeded with word-grams from a vocabulary corresponding to an ASR model. In some implementations, a word-gram may be one or more words, one or more word fragments, one or more letters, one or more phonemes, one or more additional portions of a vocabulary, and / or a combination thereof. In some implementations, the augmentation engine 104 may augment the prior candidate transcription 102 with additional word-grams from the vocabulary to generate the candidate transcription 106. For example, the system may augment the prior candidate transcription "my name is John" with the word "Doe" from the vocabulary of the ASR model to generate the candidate transcription "my name is John Doe". In some implementations, the system may generate multiple candidate transcriptions from the same prior candidate transcription. This article is about Figure 1B Examples of generating multiple candidate transcriptions from prior candidate transcriptions are described.

[0043] Additionally or alternatively, the system may process the candidate transcription 106 using a speech synthesis model 108 to generate synthesized speech audio data 110. For example, the system may process the candidate text representation "my name is John Doe" using a speech synthesis model to generate synthesized speech audio data including the synthesized utterance "my name is John Doe". The synthesized speech audio data 110 may be processed using an ASR model 112 to generate an ASR output 114. In some implementations, a vocabulary corresponding to the ASR model 112 may be used by the augmentation engine 104 when generating the candidate transcription 106. The ASR output 114 may include a candidate text representation of the synthesized speech, one or more alternative candidate text representations of the synthesized speech, probabilities of the candidate text representations, probabilities of one or more words of the candidate text representations, one or more additional outputs, and / or combinations thereof.

[0044] In some implementations, the system can process the candidate transcription 106 and the ASR output 114 using a loss engine 116 to generate a loss 118 corresponding to the candidate transcription 106. For example, the loss engine 116 can compare the text of the candidate transcription 106 with the text of the candidate text representation of the ASR output 114. In some implementations, the system can generate the loss 118 based on the comparison of the text of the candidate transcription 106 with the text of the candidate text representation of the ASR output 114.

[0045] In some implementations, the system may use the transcription engine 120 to determine whether to add the candidate transcription 106 to the candidate transcription set 124. In some of those implementations, each candidate transcription in the candidate transcription set 124 may be used as a previous candidate transcription in the next iteration of generating candidate transcriptions. For example, the transcription engine 120 may select one or more candidate transcriptions based on the corresponding losses 118. For example, the system may select a predefined number of candidate transcriptions with a minimum loss, a predefined number of candidate transcriptions with a maximum loss, each candidate transcription less than a predefined loss, each candidate transcription greater than a predefined loss, one or more additional loss indicators, and / or a combination thereof. In some other implementations, the transcription engine 120 may determine whether the candidate transcription has reached a predefined length and / or whether the candidate transcription includes a sentence-ending character in the vocabulary of the ASR model.

[0046] Figure 1B An example 140 of generating multiple candidate transcriptions based on the same previous candidate transcription according to various implementations is shown. Example 140 includes processing the previous candidate transcription 106 using the augmentation engine 104 to generate candidate transcription A 106A and candidate transcription B 106B. The system in example 140 generates two candidate transcriptions 106A and 106B based on the previous candidate transcription. In contrast, example 100 generates a single candidate transcription 106 based on the previous candidate transcription. Candidate transcription A 106A can be processed using a speech synthesis model / ASR model / loss engine 142 to generate a corresponding loss A 118A. In some implementations, the speech synthesis model / ASR model / loss engine 142 may include information about Figure 1A The speech synthesis model 108, ASR model 112, and loss engine 116 are described. Similarly, candidate transcription B 106B may be processed using the speech synthesis model / ASR model / loss engine 142 to generate a corresponding loss B 118B.

[0047] In some implementations, as described herein, the transcription engine 120 may select one or more of the candidate transcriptions (e.g., the system may select candidate transcription A 106A, candidate transcription B 106B, or candidate transcription A 106A and candidate transcription B 106B) for further augmentation based on the corresponding losses. Additionally or alternatively, the transcription engine 120 may determine whether the candidate transcriptions are long enough (e.g., reach a predefined length value, reach an end-of-sentence character, etc.), and determine whether each of the candidate transcriptions is an example of an unintentional memory based on the corresponding losses.

[0048] Figure 1CAn example 160 of generating a memory metric according to various implementations is shown. Example 160 includes processing a candidate transcription set 124 using an unintentional memory engine 162 to generate an unintentional memory list 166. In some implementations, the unintentional memory engine 162 may compare the candidate transcription set 124 with one or more candidate transcriptions generated based on a vocabulary of an alternative ASR model (not depicted). For example, the system may compare the candidate transcription set 124 generated based on the ASR model 112 with an additional candidate transcription set (not depicted) generated based on an additional ASR model (not depicted), such as an additional ASR model trained using publicly available data (i.e., a training instance based on publicly available data). Sensitive data (e.g., passwords, credit card numbers, etc.) is unlikely to be publicly available and is therefore unlikely to be included in a training instance based on publicly available data. In other words, based on a candidate transcription generated by an additional ASR model trained using publicly available data, the candidate transcription is unlikely to be an unintentional memory. In some implementations, the unintentional memory engine 162 may compare the candidate transcription set 124 with the additional transcription set. If the given candidate transcription is in the candidate transcription set 124 and in the additional candidate transcription set, the given candidate transcription is not an inadvertent memory. In some of those implementations, the inadvertent memory engine 162 may not include the given candidate transcription in the inadvertent memory list 166.

[0049] Additionally or alternatively, the involuntary memory engine 162 can generate a memory metric 164. In some implementations, the memory metric 164 can be based on the number of involuntary memories in the involuntary memory list 166. For example, the involuntary memory engine 162 can generate a memory metric 10 based on a list with 10 example involuntary memories. In other words, each involuntary memory can add one point to the involuntary memory metric. However, this is merely illustrative, and the memory metric can be based on additional or alternative indicators, such as 0.5 points for each involuntary memory, one point for each involuntary memory that occurs more than once, one point for each unique involuntary memory, and / or a combination thereof.

[0050] Figure 2An example 200 of generating candidate transcriptions according to various implementations is shown. In some implementations, candidate transcriptions can be generated based on one or more candidate transcriptions generated in a previous iteration. In an initial iteration, candidate transcriptions can be seeded based on word elements of a vocabulary of an ASR model. In some implementations, word elements of a vocabulary may include one or more words, one or more word fragments, one or more letters, one or more phonemes, one or more additional parts of a vocabulary, and / or a combination thereof. The example 200 shown uses words of a vocabulary of an ASR model to generate candidate transcriptions. In some implementations, the system can generate an initial candidate transcription, wherein each word in the vocabulary corresponds to a distinct candidate transcription (not depicted). Example 200 includes an initial set of candidate transcriptions 202, 204, ..., 206 based on a previous candidate transcription 'WHAT'. The initial set includes candidate transcription 202 'WHAT DOES', candidate transcription 204 'WHAT PENGUIN', ..., and candidate transcription 206 'WHAT CAN'. Although only three candidate transcriptions are shown in the initial set of candidate transcriptions, the system may include one or more additional candidate transcriptions, where each additional candidate transcription may be based on an additional word-gram in the vocabulary of the ASR model.

[0051] The system can process the initial set of candidate transcriptions 202, 204, ..., 206 to generate a loss corresponding to each candidate transcription. In some implementations, the system can process the initial set of candidate transcriptions 202, 204, ..., 206 to generate a loss corresponding to each candidate transcription. Figure 1A and Figure 5 The process 500 generates losses corresponding to candidate transcriptions. In some of those implementations, the losses may be generated using the loss engine 116 described herein.

[0052] In some implementations, the system can select one or more of the candidate transcriptions based on the corresponding loss for further processing. In some implementations in those implementations, the system can select one or more of the candidate transcriptions with the minimum corresponding loss. For example, candidate transcription 202 'WHAT DOES' can have a corresponding loss of 0.01, candidate transcription 204 'WHAT PENGUIN' can have a corresponding loss of 0.05, and candidate transcription 602 'WHAT CAN' can have a corresponding loss of 0.02. The system in example 200, the system selects two candidate transcriptions with minimum loss. In other words, the system selects candidate transcription 202 'WHAT DOES' with a corresponding candidate loss of 0.01 (which is less than 0.02 and 0.05), and candidate transcription 206 'WHAT CAN' with a corresponding candidate loss of 0.02 (which is less than 0.05). However, this is merely illustrative, and in some other implementations, the system can alternatively select multiple candidate transcriptions.

[0053] Candidate transcription 202 'WHAT DOES' can be augmented with one or more additional words in the vocabulary of the ASR. As shown herein, candidate transcription 202 can be augmented to generate candidate transcription 208, ..., candidate transcription 210. For example, candidate transcription 202 'WHAT DOES' can be augmented with the word 'THE' to generate candidate transcription 208 'WHAT DOES THE'. Similarly, candidate transcription 202 'WHAT DOES' can be augmented with the word 'MY' to generate candidate transcription 210 'WHAT DOESMY'. Additionally or alternatively, candidate transcription 206 'WHATCAN' can be augmented with one or more additional words in the vocabulary of the ASR to generate candidate transcription 212, ..., candidate transcription 214. For example, candidate transcription 206 'WHAT CAN' can be augmented with the word 'THE' to generate candidate transcription 212 'WHAT CAN THE'. Similarly, the candidate transcription 206 'WHAT CAN' may be augmented with the word 'JOHN' to generate the candidate transcription 214 'WHAT CAN JOHN'.

[0054] The system can process the expanded set of candidate transcriptions 208, ..., 210 and 212, ..., 214 to generate a loss corresponding to each candidate transcription. In some implementations, the loss can be calculated according to the method described herein. Figure 1A and Figure 5 The process 500 generates losses corresponding to candidate transcriptions. In some of those implementations, the losses can be generated using the loss engine 116 described herein. For example, the system can generate a loss of 0.1 corresponding to candidate transcription 208, a loss of 0.7 corresponding to candidate transcription 210, a loss of 0.2 corresponding to candidate transcription 212, and a loss of 0.4 corresponding to candidate transcription 214.

[0055] In example 200, the system selects two candidate transcriptions with the smallest corresponding losses for additional augmentation. In other words, the system may select candidate transcription 208 with a corresponding loss of 0.1 and candidate transcription 212 with a corresponding loss of 0.2.

[0056] In some implementations, the candidate transcription 208 'WHAT DOES THE' can be augmented with one or more words in the vocabulary of the ASR. For example, the candidate transcription 208 'WHAT DOES THE' can be augmented with the word 'RABBIT' to generate a further augmented candidate transcription 216 'WHAT DOES THE RABBIT'. Similarly, the candidate transcription 208 'WHAT DOES THE' can be augmented with the word 'CAT' to generate a further augmented candidate transcription 218 'WHAT DOES THE CAT'. Additionally or alternatively, the candidate transcription 212 'WHAT CAN THE' can be augmented with the word 'RABBIT' to generate a further augmented candidate transcription 220 'WHAT CAN THE RABBIT'. Similarly, the candidate transcription 212 'WHATCAN THE' can be augmented with the word 'TREE' to generate a further augmented candidate transcription 222 'WHAT CAN THE TREE'.

[0057] In some implementations, the system can continue to generate expanded candidate transcriptions until a target length for the transcription is reached. In some of those implementations, the target length can be predefined. In some implementations, the system can continue to generate one or more expanded candidate transcriptions until an end token (e.g., end of a word, end of a character, end of a sentence, end of a paragraph, etc.) is reached.

[0058] In some implementations, the system can determine an unintentional memory set based on augmenting the candidate transcriptions. For example, the system can process the candidate transcriptions using the unintentional memory engine 162 described herein to determine the unintentional memory set. Additionally or alternatively, the system can determine a memory metric based on augmenting the candidate transcriptions. In some implementations, the system can process the candidate transcription set using the unintentional memory engine 162 described herein to determine the memory metric.

[0059] Figure 3 A block diagram of an example environment 300 is shown in which various implementations may be implemented. The example environment 300 includes a computing system 302, which may include an ASR engine 304, an augmentation engine 104, a loss engine 116, a speech synthesis engine 306, a transcription engine 120, an unintentional memory engine 162, and / or one or more additional engines. Additionally or alternatively, the computing system 302 may be associated with the speech synthesis model 108, the ASR model 112, the candidate transcription 124, and / or one or more additional components (not depicted).

[0060] In some implementations, computing system 302 may include a user interface input / output device (not depicted), which may include, for example, a physical keyboard, a touch screen (e.g., implementing a virtual keyboard or other text input mechanism), a microphone, a camera, a display screen, and / or a speaker. Additionally or alternatively, computing system 302 may include a variety of sensors (not depicted), such as an accelerometer, a gyroscope, a global positioning system (GPS), a pressure sensor, a light sensor, a distance sensor, a proximity sensor, a temperature sensor, one or more additional sensors, and / or combinations thereof. The user interface input / output device may be integrated with one or more computing systems 302 of a user. For example, a user's mobile phone may include a user interface input / output device; a stand-alone digital assistant hardware device may include a user interface input / output device; a first computing device may include a user interface input device, and a separate computing device may include a user interface output device; and so on. In some implementations, all or aspects of computing system 302 may be implemented on a computing system that also includes a user interface input / output device. In some implementations, computing system 302 may include an automated assistant (not depicted), and all or aspects of the automated assistant may be implemented on a computing device that is separate and remote from a client device that includes a user interface input / output device (e.g., all or aspects may be implemented “in the cloud”). In some of those implementations, aspects of the automated assistant may communicate with the computing device via one or more networks, such as a local area network (LAN) and / or a wide area network (WAN) (e.g., the Internet).

[0061] Some non-limiting examples of computing system 302 include one or more of the following: a desktop computing device, a laptop computing device, a stand-alone hardware device at least partially dedicated to an automated assistant, a tablet computing device, a mobile phone computing device, a computing device for a vehicle (e.g., an in-vehicle communication system, and an in-vehicle entertainment system, an in-vehicle navigation system, an in-vehicle navigation system), or a user's wearable device including a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual or augmented reality computing device). Additional and / or alternative computing systems may be provided. Computing system 302 may include one or more memories for storing data and software applications, one or more processors for accessing data and executing applications, and other components that facilitate communication over a network. The operations performed by computing system 302 may be distributed across multiple computing devices. For example, computing programs running on one or more computers in one or more locations may be coupled to each other via a network.

[0062] In some implementations, the ASR engine 304 can process a given instance of audio data using the ASR model 112 to generate a textual representation of the utterance captured in the given instance of audio data. For example, the ASR engine 304 can process the synthesized speech audio data 110 using the ASR model 112 to generate the ASR output 114.

[0063] In some implementations, the augmentation engine 104 can be used to generate one or more candidate transcriptions 124. In some implementations, the candidate transcriptions 124 can be generated based on a vocabulary corresponding to the ASR model 112. For example, the augmentation engine 104 can generate the one or more candidate transcriptions 124 according to the process 400 described herein.

[0064] In some implementations, the loss engine 116 can be used to determine the loss between the candidate transcription and the textual representation of the candidate transcription captured in the synthesized speech audio data. For example, the system can use the loss engine 116 to generate the loss 118 based on comparing the candidate transcription 106 to the ASR output 114, where the ASR output 114 is based on processing the synthesized speech audio data 110 using the ASR model 112.

[0065] In some implementations, the speech synthesis engine 306 can process the candidate transcription using the speech synthesis model 108 to generate an instance of synthesized speech audio data, wherein the synthesized speech audio data includes at least a synthesized representation of the corresponding candidate transcription. For example, the speech synthesis engine 306 can process the candidate transcription 106 using the speech synthesis model 108 to generate an instance of synthesized speech audio data 110.

[0066] In some implementations, the transcription engine 120 may be used to generate one or more candidate transcriptions 124. In some of those implementations, the one or more candidate transcriptions 124 may be generated based on the vocabulary of the ASR model 112. For example, the transcription engine 120 may be described herein with respect to Figure 2 Candidate transcripts were generated as shown.

[0067] In some implementations, the unconscious memory engine 162 may be used to determine a memory metric based on the ASR model 112. In some implementations, the unconscious memory engine 162 may be used to determine a memory metric based on the ASR model 112. Figure 4 The process 400 generates a memory metric.

[0068] Figure 44 is a flow chart illustrating an example process 400 for performing actions based on a candidate transcription set and / or a memory metric according to various implementations disclosed herein. For convenience, the operations of the flow chart are described with reference to a system that performs the operations. This system may include various components of various computer systems, such as computing system 302, client device 602, and / or one or more components of computing system 710. Moreover, although the operations of process 400 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, and / or added.

[0069] At block 402, the system generates a set of candidate transcriptions based on a vocabulary of the ASR model. In some implementations, the ASR model inadvertently memorizes one or more occurrences of a person saying each of the candidate transcriptions when training the ASR model. In some implementations, the system can generate the set of candidate transcriptions according to the process 500 described herein.

[0070] At box 404, the system generates a memory metric based on the candidate transcription set of the ASR model. In some implementations, the system can use the unintentional memory engine 162 to process the candidate transcription set to generate the memory metric described herein. In some implementations, the system can compare the candidate transcription set generated based on the ASR model with the additional candidate transcription set generated based on the additional ASR model. In some implementations of those implementations, the additional ASR model can be a model trained using publicly available training data (e.g., training data based on the uploaded content of the video sharing platform, etc.). The candidate transcription generated based on the publicly available training data is unlikely to contain sensitive information, and therefore is unlikely to indicate the unintentional memory of the ASR model. In some implementations, if a given candidate transcription occurs in both the candidate transcription set and the additional candidate transcription set, the system can ignore the given candidate transcription when generating the memory metric at box 404.

[0071] In some implementations, a memory metric can be generated based on the number of candidate transcriptions in the candidate transcription set. In some implementations, the memory metric can be based on the total number of candidate transcriptions in the candidate transcription set. For example, the system can generate a set with 50 candidate transcriptions at box 402, where the system can determine that the memory metric is 50 (i.e., assigning a defined number of points to each candidate transcription) based on the set with 50 candidate transcriptions.

[0072] At box 406, the system performs one or more actions based on the candidate transcription set and / or the memory metric. In some implementations, the system may use the memory metric when evaluating the training of the ASR model. For example, the system may determine an initial memory metric based on an initial version of the ASR model. The system may perform one or more training techniques to update the ASR model to generate an updated ASR model. After generating the updated ASR model, the system may generate an updated memory metric. In some implementations, the system may determine whether the training increases or decreases unintentional memory in the ASR model based on comparing the initial memory metric with the updated memory metric. Additionally or alternatively, the system may determine whether to release an updated version of the ASR model based on comparing the initial memory metric with the updated memory metric.

[0073] Figure 5 5 is a flow chart illustrating an example process 400 for determining whether an ASR model has inadvertently memorized one or more occurrences of a person uttering a candidate transcription according to various implementations described herein. For convenience, the operations of the flow chart are described with reference to a system that performs the operations. This system may include various components of various computer systems, such as computing system 302, client device 602, and / or one or more components of computing system 710. Moreover, while the operations of process 500 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, and / or added.

[0074] At block 502, the system generates a candidate transcription by augmenting a previous candidate transcription based on a vocabulary of an ASR model. In some implementations, in an initial iteration, the previous candidate transcription may be initialized based on word-grams of the vocabulary of the ASR model. In some of those implementations, the word-gram may include one or more letters, one or more phonemes, one or more words, one or more phrases, one or more additional word-grams, and / or combinations thereof of the vocabulary of the ASR model.

[0075] In some implementations, the previous candidate transcription can be expanded based on word-grams of the vocabulary of the ASR model. In some of those implementations, multiple candidate transcriptions can be generated based on a given previous candidate transcription, where each candidate transcription is generated by expanding the given previous candidate transcription with distinct word-grams of the vocabulary of the ASR model. In some implementations, the candidate transcriptions can be generated using the expansion engine 104 described herein. Additionally or alternatively, in some implementations, the candidate transcriptions can be generated based on the word-grams described herein. Figure 2 Generate candidate transcripts. For example, as described in this paper Figure 2 As described in , the system may generate the candidate transcription 208 'WHAT DOESTHE' by augmenting the previous candidate transcription 202 'WHAT DOES' with the word 'THE'.

[0076] At block 504, the system generates synthesized speech audio data by processing the candidate transcription using the speech synthesis model. In some implementations, the synthesized speech audio data includes a synthesized verbal representation of the candidate transcription. In some implementations, the system can generate the synthesized speech audio data by processing the candidate transcription using the speech synthesis engine 306 and / or the speech synthesis model 108 described herein. For example, the system can process the candidate transcription 208 'WHAT DOESTHE' using the speech synthesis model 108 to generate synthesized speech audio data including a synthesized verbal representation of "what does the".

[0077] At box 506, the system generates an ASR output by processing the synthesized speech audio data using the ASR model. In some implementations, the ASR output may include candidate text representations of the synthesized speech of the candidate transcription. Additionally or alternatively, the ASR output may include a probability indicating the likelihood that a word in the candidate text representation is a corresponding portion of the synthesized speech audio data. In addition, the ASR output may include one or more alternative candidate text representations of the synthesized speech captured in the audio data. For example, the system may process the synthesized speech audio data that captures a synthesized oral representation of "what does the" to generate a candidate text representation of the synthesized speech of "what does the". In some implementations, the synthesized speech audio data generated at box 504 may be processed using the ASR model 112 to generate an ASR output.

[0078] At block 508, the system generates a loss based on comparing the candidate transcription to the ASR output. For example, the system may compare the text of the candidate transcription to the text of the ASR output to generate the loss. In some implementations, the system may generate the loss using the loss engine 116 described herein.

[0079] At block 510, the system determines, based on the loss, whether the ASR model unintentionally memorized one or more occurrences of a person saying a candidate transcription used in training the ASR model. In some implementations, the system may use the unintentional memory engine 162 described herein to determine whether the ASR model unintentionally memorized one or more occurrences of a person saying a candidate transcription when training the ASR model.

[0080] Now go to Figure 6 , showing an example environment in which various implementations may be performed. Figure 66 is initially described, and includes a client computing device 602 executing an instance of an automated assistant client 604. One or more cloud-based automated assistant components 610 may be implemented on one or more computing systems (collectively referred to as "cloud" computing systems) that are communicatively coupled to the client device 602 via one or more local area networks and / or wide area networks (e.g., the Internet), indicated generally at 608.

[0081] An instance of automated assistant client 604, through its interaction with one or more cloud-based automated assistant components 610, can form what appears to be a logical instance of automated assistant 600 from the user's perspective, with which the user can engage in a human-computer dialogue. Figure 6 600. Thus, it should be understood that in some implementations, a user interacting with an automated assistant client 604 executing on a client device 602 may actually be interacting with his or her own logical instance of automated assistant 600. For the sake of brevity and clarity, the term "automated assistant" as used herein will generally refer to the combination of an automated assistant client 604 executing on a client device 602 operated by a user and one or more cloud-based automated assistant components 610 (which may be shared among multiple automated assistant clients on multiple client computing devices) when "serving" a particular user. It should also be understood that in some implementations, automated assistant 600 may respond to requests from any user, whether or not that user is actually "served" by that particular instance of automated assistant 600.

[0082] Client computing device 602 may be, for example, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device of a user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), a standalone interactive speaker, a smart home appliance (such as a smart TV), and / or a user's wearable device including a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual or augmented reality computing device). Additional and / or alternative client computing devices may be provided. In various implementations, client computing device 602 may also optionally operate one or more other applications in addition to automated assistant client 604, such as a messaging client (e.g., SMS, MMS, online chat), a browser, etc. In some of those various implementations, one or more of the other applications may optionally interface with automated assistant 600 (e.g., via an application programming interface), or include their own instances of automated assistant applications (which may also interface with cloud-based automated assistant component 610).

[0083] The automated assistant 600 conducts a human-computer dialogue session with the user via the user interface input and output devices of the client device 602. In order to protect user privacy and / or save resources, in many cases, the user must usually explicitly invoke the automated assistant 600 before the automated assistant will fully process the spoken utterance. The explicit invocation of the automated assistant 600 may occur in response to a certain user interface input received at the client device 602. For example, the user interface input that may invoke the automated assistant 600 via the client device 602 may optionally include the actuation of a hardware and / or virtual button of the client device 602. Moreover, the automated assistant client may include one or more local engines 606, such as a call engine that is operable to detect the presence of one or more verbal invocation phrases. The call engine may invoke the automated assistant 600 in response to detecting one of the verbal invocation phrases. For example, the call engine may invoke the automated assistant 600 in response to detecting a verbal invocation phrase such as "Hey Assistant", "OK Assistant" and / or "Assistant". The invocation engine may continuously process (e.g., while not in an "inactive" mode) a stream of audio data frames based on the output of one or more microphones from the client device 602 to monitor for occurrences of spoken invocation phrases. While monitoring for occurrences of spoken invocation phrases, the invocation engine discards (e.g., after temporary storage in a buffer) any audio data frames that do not include a spoken invocation phrase. However, when the invocation engine detects an occurrence of a spoken invocation phrase in a processed audio data frame, the invocation engine may invoke the automated assistant 600. As used herein, "invoking" the automated assistant 600 may include causing one or more previously inactive functions of the automated assistant 600 to be activated. For example, invoking the automated assistant 600 may include causing one or more local engines 606 and / or cloud-based automated assistant components 610 to further process an audio data frame based on which an invocation phrase was detected, and / or one or more subsequent audio data frames (whereas no further processing of the audio data frames occurred prior to the invocation).

[0084] The one or more local engines 606 of automated assistant 600 are optional and may include, for example, the augmentation engine, loss engine, transcription engine, and involuntary memory engine described above, a local speech-to-text (“STT”) engine (which converts captured audio to text), a local text-to-speech (“TTS”) engine (which converts text to speech), a local natural language processor (which determines the semantic meaning of audio and / or text converted from audio), and / or other local components. Because client device 602 is relatively constrained in computing resources (e.g., processor cycles, memory, battery, etc.), local engine 606 may have limited functionality relative to any corresponding component included in cloud-based automated assistant component 610.

[0085] Cloud-based automated assistant component 610 leverages the virtually unlimited resources of the cloud to perform more robust and / or more accurate processing of audio data and / or other user interface inputs relative to any corresponding component of local engine 606. Likewise, in various implementations, client device 602 can provide audio data and / or other data to cloud-based automated assistant component 610 in response to an invocation engine detecting a spoken invocation phrase or detecting some other explicit invocation of automated assistant 600.

[0086] The cloud-based automated assistant components 610 shown include a cloud-based TTS module 612, a cloud-based STT module 614, a natural language processor 616, a dialog state tracker 618, and a dialog manager 620. In some implementations, one or more of the engines and / or modules of the automated assistant 600 may be omitted, combined, and / or implemented in a component separate from the automated assistant 600. Furthermore, in some implementations, the automated assistant 600 may include additional and / or alternative engines and / or modules. The cloud-based STT module 614 may convert audio data into text, which may then be provided to the natural language processor 616.

[0087] The cloud-based TTS module 612 can convert text data (e.g., natural language responses formulated by the automated assistant 600) into computer-generated speech output. In some implementations, the TTS module 612 can provide the computer-generated speech output to the client device 602 for direct output, for example, using one or more speakers. In other implementations, the text data (e.g., natural language responses) generated by the automated assistant 600 can be provided to one of the local engines 606, which can then convert the text data into computer-generated speech that is output locally.

[0088] Natural language processor 616 of automated assistant 600 processes free-form natural language input and generates annotated output based on the natural language input for use by one or more other components of automated assistant 600. For example, natural language processor 616 can process natural language free-form input, which is text input that is a conversion of audio data provided by a user via client device 602 by STT module 614. The generated annotated output can include one or more annotations to the natural language input and, optionally, one or more (e.g., all) of the terms in the natural language input.

[0089] In some implementations, the natural language processor 616 is configured to recognize and annotate various types of grammatical information in the natural language input. In some implementations, the natural language processor 616 may additionally and / or alternatively include an entity tagger (not depicted) that is configured to annotate entity references in one or more segments (such as references to people (including, for example, literary characters, celebrities, public figures, etc.), organizations, locations (real and fictional), etc.). In some implementations, the natural language processor 816 may additionally and / or alternatively include a coreference resolver (not depicted) that is configured to group or "cluster" references to the same entity based on one or more context clues. For example, the coreference resolver may be used to resolve the term "there" in the natural language input "I liked Hypothetical Café last time we ate there (last time we ate at Hypothetical Café, I liked it there)" to "Hypothetical Café". In some implementations, one or more components of the natural language processor 616 can rely on annotations from one or more other components of the natural language processor 616. In some implementations, when processing a particular natural language input, one or more components of the natural language processor 616 can determine one or more annotations using related prior inputs and / or other related data outside of the particular natural language input.

[0090] In some implementations, the dialog state tracker 618 may be configured to track a "dialog state," which includes, for example, the belief state of the goals (or "intents") of one or more users during a human-computer dialog session and / or across multiple dialog sessions. In determining the dialog state, some dialog state trackers may attempt to determine the most likely values ​​for slots instantiated in the dialog based on user and system utterances in the dialog session. Some techniques utilize a fixed ontology that defines a set of slots and a set of values ​​associated with those slots. Some techniques may additionally or alternatively be customized for individual slots and / or domains. For example, some techniques may require training a model for each slot type in each domain.

[0091] The dialog manager 620 may be configured to map the current dialog state, for example, provided by the dialog state tracker 618, to one or more "response actions" of a plurality of candidate response actions, which are then executed by the automated assistant 600. Depending on the current dialog state, the response action may take a variety of forms. For example, the initial dialog state and the mid-stream dialog state corresponding to the turn that occurred before the last turn in the dialog session (e.g., when the final user-desired task is performed) may be mapped to various response actions, and the various response actions include the automated assistant 600 outputting additional natural language dialogs. This response dialog may include, for example, a user's request to provide parameters (i.e., fill a slot) for an action that the dialog state tracker 618 believes the user intends to perform. In some implementations, the response action may include actions such as: "request" (e.g., seeking parameters for slot filling), "offer" (e.g., suggesting an action or course of action for the user), "select", "notify" (e.g., providing the user with the requested information), "unmatched" (e.g., notifying the user that the user's last input could not be understood), a command to a peripheral device (e.g., turning off a light bulb), and the like.

[0092] Figure 7 710 is a block diagram of an example computing device 710 that can optionally be used to perform one or more aspects of the techniques described herein. In some implementations, one or more of the client computing devices and / or other components can include one or more components of the example computing device 710.

[0093] The computing device 710 typically includes at least one processor 714 that communicates with a number of peripheral devices via a bus subsystem 712. These peripheral devices may include a storage subsystem 724 (including, for example, a memory subsystem 925 and a file storage subsystem 726), a user interface output device 720, a user interface input device 922, and a network interface subsystem 716. The input and output devices allow for user interaction with the computing device 710. The network interface subsystem 716 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0094] The user interface input device 722 may include a keyboard, a pointing device (such as a mouse, a trackball, a touch pad or a graphics tablet, a scanner, a touch screen incorporated into a display), an audio input device (such as a voice recognition system, a microphone), and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways of inputting information into the computing device 710 or onto a communication network.

[0095] The user interface output device 720 may include a display subsystem, a printer, a fax machine, or a non-visual display (such as an audio output device). The display subsystem may include a cathode ray tube ("CRT"), a flat panel device (such as a liquid crystal display ("LCD")), a projection device, or some other mechanism for producing a visible image. The display subsystem may also provide a non-visual display, such as via an audio output device. In general, the use of the term "output device" is intended to include all possible types of devices and ways to output information from the computing device 710 to a user or another machine or computing device.

[0096] The storage subsystem 724 stores programming and data structures that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 724 may include a computer program that executes Figure 4 and / or Figure 5 selected aspects of one or more of the processes and the methods used to implement Figure 3 and / or Figure 6 The logic of the various components is depicted in .

[0097] These software modules are typically executed by the processor 714 alone or in combination with other processors. The memory 725 used in the storage subsystem 724 may include multiple memories including a main random access memory ("RAM") 730 for storing instructions and data during program execution and a read-only memory ("ROM") 732 in which fixed instructions are stored. The file storage subsystem 726 may provide persistent storage for program and data files and may include a hard drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of certain implementations may be stored by the file storage subsystem 926 in the storage subsystem 724 or in other machines accessible by the processor 714.

[0098] The bus subsystem 712 provides a mechanism for the various components and subsystems of the computing device 710 to communicate with each other as intended. Although the bus subsystem 712 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0099] The computing device 710 can be of different types, including a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Figure 7 The depicted description of computing device 710 is intended only as a specific example for purposes of illustrating some implementations. Many other configurations of computing device 710 are possible, and these configurations are similar to those of FIG. Figure 7 Computing devices are depicted as having more or fewer components than those shown.

[0100] In situations where the systems described herein collect personal information about users (or "participants," as they are often referred to herein) or may make use of personal information, users may be provided with the opportunity to control whether a program or feature collects user information (e.g., information about the user's social network, social actions or activities, occupation, the user's preferences, or the user's current geographic location), or to control whether and / or how content is received from a content server that may be more relevant to the user. Additionally, certain data may be processed in one or more ways prior to its storage or use so that personally identifiable information is removed. For example, the user's identity may be processed so that personally identifiable information about the user cannot be determined, or, in the case of geographic location information, the user's geographic location may be generalized (such as to a city, zip code, or state level) so that the user's specific geographic location cannot be determined. Thus, users may control how information about the user is collected and / or how that information is used.

[0101] In some implementations, a method implemented by one or more processors is provided, the method comprising generating candidate transcriptions using a vocabulary corresponding to an automatic speech recognition ("ASR") model. In some implementations, the method further comprises generating synthesized speech audio data including synthesized speech of the candidate transcription based on processing the candidate transcription using a speech synthesis model. In some implementations, the method further comprises processing the synthesized speech audio data using the ASR model to generate an ASR output, the ASR output reflecting a predicted text representation of the synthesized speech. In some implementations, the method further comprises generating a loss based on a comparison of the ASR output with the candidate transcription. In some implementations, the method further comprises determining, based on the loss, whether the ASR model has inadvertently memorized one or more occurrences of the candidate transcription being spoken by a corresponding person in training data used to train the ASR model. In some implementations, the method further comprises generating an overall memory metric based on determining whether the ASR model has inadvertently memorized one or more occurrences.

[0102] These and other implementations of the technology may include one or more of the following features.

[0103] In some implementations, generating a candidate transcription using a vocabulary corresponding to an ASR model includes: before generating the candidate transcription, generating a previous candidate transcription using a vocabulary corresponding to the ASR model, wherein the previous candidate transcription includes the beginning portion of the word of the candidate transcription but lacks one or more ending words of the candidate transcription. In some implementations, the method also includes generating a previous synthesized speech audio data including the previous synthesized speech in the previous candidate transcription based on processing the previous candidate transcription using the speech synthesis model. In some implementations, the method also includes processing the previous synthesized speech audio data using the ASR model to generate a previous ASR output, which reflects the previous predicted text representation of the synthesized speech. In some implementations, the method also includes generating a previous loss based on a comparison of the previous ASR output with the previous candidate transcription. In some implementations, the method also includes selecting a previous candidate transcription for expansion based on the previous loss. In some versions of those implementations, before generating the candidate transcription, the method further includes generating an additional preceding candidate transcription using a vocabulary corresponding to the ASR model, wherein the additional preceding candidate transcription includes the beginning portion of the words of the preceding candidate transcription but lacks one or more ending words of the preceding candidate transcription. In some implementations, the method further includes generating additional preceding synthesized speech audio data including synthesized speech of the additional preceding candidate transcription based on processing the additional preceding candidate transcription using the speech synthesis model. In some implementations, the method further includes processing the additional preceding synthesized speech audio data using the ASR model to generate an additional preceding ASR output, the additional preceding ASR output reflecting an additional preceding predicted text representation of the additional preceding synthesized speech. In some implementations, the method further includes generating an additional preceding loss based on a comparison of the additional preceding ASR output with the additional preceding candidate transcription. In some implementations, the method further includes selecting the preceding candidate transcription instead of the additional preceding candidate transcription for augmentation based on (a) the preceding loss of the preceding candidate transcription and (b) the additional preceding loss of the preceding candidate transcription.

[0104] In some implementations, generating the loss based on comparing the ASR output to the candidate transcription includes generating the loss based on comparing a transcribed portion of the ASR output to the candidate transcription.

[0105] In some implementations, generating a loss based on the comparison of the ASR output to the candidate transcription includes: for each word in the candidate transcription, comparing a probability portion of the ASR output of a corresponding portion of the ASR output to the word in the candidate transcription. In some implementations, the method further includes generating a loss based on the comparison.

[0106] In some implementations, the method also includes determining whether to utilize particular techniques in federated training of the ASR model and / or additional machine learning models based on an overall memory metric of the ASR model.

[0107] In some implementations, the method further includes transmitting the overall memory metric in response to a request from a third party.

[0108] In some implementations, the method also includes transmitting the candidate transcription in response to a request by a third party.

[0109] In some implementations, the vocabulary corresponding to the ASR model includes a set of word units, a set of characters, a set of letters, a set of words, a set of word fragments, and / or a set of phonemes.

[0110] In some implementations, a method implemented by one or more processors is provided, the method comprising receiving an automatic speech recognition ("ASR") model and a vocabulary corresponding to the ASR model. In some implementations, the method further comprises generating a set of candidate transcriptions based on the vocabulary of the ASR model. In some implementations, for each candidate transcription in the set of candidate transcriptions, the method further comprises generating synthetic speech audio data including synthetic speech of the candidate transcription based on processing the candidate transcription using a speech synthesis model. In some implementations, the method further comprises processing the synthetic speech audio data using the ASR model to generate an ASR output, the ASR output reflecting a predicted text representation of the synthesized speech. In some implementations, the method further comprises generating a loss based on a comparison of the ASR output with the candidate transcription. In some implementations, the method further comprises determining, based on the loss, whether the ASR model has inadvertently memorized one or more occurrences of a corresponding person speaking the candidate transcription in training data used to train the ASR model.

[0111] These and other implementations of the technology may include one or more of the following features.

[0112] In some implementations, the method further includes generating an overall memory metric based on determining whether the ASR model has inadvertently memorized one or more occurrences corresponding to one or more candidate transcriptions in the set of candidate transcriptions. In some versions of those implementations, the method further includes identifying a subset of candidate transcriptions, wherein a loss indication corresponding to each of the candidate transcriptions in the subset of candidate transcriptions indicates that a probability that the ASR model has inadvertently memorized the corresponding one or more occurrences satisfies a threshold. In some versions of those implementations, receiving the ASR model and a vocabulary corresponding to the ASR model includes receiving the ASR model and the vocabulary from a third party, and the method further includes transmitting the subset of candidate transcriptions to the third party.

[0113] In some implementations, the training data used to train the ASR model includes one or more target training instances, and the method further includes determining whether the set of candidate transcriptions includes the one or more target training instances. In some implementations, the method further includes determining an overall memory metric based on whether the set of candidate transcriptions includes the one or more target training instances.

[0114] In addition, some implementations include one or more processors (e.g., central processing units (CPUs), graphics processing units (GPUs), and / or tensor processing units (TPUs)) of one or more computing devices, wherein the one or more processors are operable to execute instructions stored in an associated memory, and wherein the instructions are configured to cause the performance of any of the methods described herein. Some implementations also include one or more temporary or non-temporary computer-readable storage media storing computer instructions that can be executed by one or more processors to perform any of the methods described herein.

Claims

1. A method implemented by one or more processors, the method comprising: generating candidate transcriptions using a vocabulary corresponding to an automatic speech recognition ("ASR") model; generating synthesized speech audio data including synthesized speech of the candidate transcription based on processing the candidate transcription using a speech synthesis model; processing the synthesized speech audio data using the ASR model to generate an ASR output reflecting a predicted textual representation of the synthesized speech; generating a loss based on a comparison of the ASR output and the candidate transcription; determining, based on the loss, whether the ASR model has inadvertently memorized one or more occurrences of a corresponding person uttering the candidate transcription in training data used to train the ASR model; and An overall memory metric is generated based on determining whether the ASR model has inadvertently memorized the one or more occurrences.

2. The method of claim 1, wherein: Generating the candidate transcription using the vocabulary corresponding to the ASR model comprises: Prior to generating the candidate transcripts: generating a previous candidate transcription using the vocabulary corresponding to the ASR model, wherein the previous candidate transcription includes the beginning portion of the words of the candidate transcription but lacks one or more ending words of the candidate transcription; generating prior synthesized speech audio data comprising prior synthesized speech of the prior candidate transcription based on processing the prior candidate transcription using the speech synthesis model; processing the prior synthesized speech audio data using the ASR model to generate a prior ASR output reflecting a prior predicted textual representation of the synthesized speech; generating a prior loss based on a comparison of the prior ASR output and the prior candidate transcription; and Based on the prior losses, the prior candidate transcripts are selected for augmentation.

3. The method of claim 2, further comprising: Prior to generating the candidate transcripts: generating an additional preceding candidate transcription using the vocabulary corresponding to the ASR model, wherein the additional preceding candidate transcription includes the beginning portions of the words of the preceding candidate transcription but lacks one or more ending words of the preceding candidate transcription; generating, based on processing the appended preceding candidate transcription using the speech synthesis model, appended preceding synthesized speech audio data including synthesized speech of the appended preceding candidate transcription; processing the additional preceding synthesized speech audio data using the ASR model to generate an additional preceding ASR output reflecting an additional preceding predicted textual representation of the additional preceding synthesized speech; generating an append-before loss based on a comparison of the append-before ASR output and the append-before candidate transcription; and The previous candidate transcription is selected for augmentation over the additional previous candidate transcription based on (a) the previous loss of the previous candidate transcription and (b) the additional previous loss of the additional previous candidate transcription.

4. The method of claim 1, wherein: Generating the loss based on the comparison of the ASR output with the candidate transcription comprises: The loss is generated based on comparing a transcribed portion of the ASR output to the candidate transcription.

5. The method of claim 1, wherein: Generating the loss based on the comparison of the ASR output with the candidate transcription comprises: For each word in the candidate transcription, comparing a probabilistic portion of the ASR output of a corresponding portion of the ASR output to the words in the candidate transcription; and The loss is generated based on the comparison.

6. The method of claim 1, further comprising: A determination is made based on the overall memory metric of the ASR model whether to utilize particular techniques in federated training of the ASR model and / or additional machine learning models.

7. The method of claim 1, further comprising: The overall memory metric is transmitted in response to a request from a third party.

8. The method of claim 1, further comprising: The candidate transcription is transmitted in response to a request from a third party.

9. The method of claim 1, wherein: The vocabulary corresponding to the ASR model includes a word set, a character set, a letter set, a word set, a word fragment set and / or a phoneme set.

10. A method implemented by one or more processors, the method comprising: receiving an automatic speech recognition ("ASR") model and a vocabulary corresponding to the ASR model; generating a set of candidate transcriptions based on the vocabulary of the ASR model; For each candidate transcription in the set of candidate transcriptions: generating synthesized speech audio data including synthesized speech of the candidate transcription based on processing the candidate transcription using a speech synthesis model; processing the synthesized speech audio data using the ASR model to generate an ASR output reflecting a predicted textual representation of the synthesized speech; generating a loss based on a comparison of the ASR output and the candidate transcription; as well as A determination is made based on the loss whether the ASR model has inadvertently memorized one or more occurrences of a corresponding person uttering the candidate transcription in training data used to train the ASR model.

11. The method of claim 10, further comprising: An overall memory metric is generated based on determining whether the ASR model has inadvertently memorized the one or more occurrences corresponding to one or more candidate transcriptions in the set of candidate transcriptions.

12. The method of claim 11, further comprising: A subset of candidate transcriptions is identified, wherein the loss indication corresponding to each of the candidate transcriptions in the subset of candidate transcriptions satisfies a threshold probability indicating that the ASR model has inadvertently memorized the corresponding one or more occurrences.

13. The method of claim 12, wherein: Receiving the ASR model and the vocabulary corresponding to the ASR model includes receiving the ASR model and the vocabulary from a third party, and the method further includes transmitting the subset of candidate transcriptions to the third party.

14. The method of claim 10, wherein: The training data for training the ASR model includes one or more target training instances, and the method further includes: determining whether the set of candidate transcriptions includes the one or more target training instances; and An overall memory metric is determined based on whether the set of candidate transcriptions includes the one or more target training instances.

15. A computer program comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method of any preceding claim.

16. A client device, the client device comprising one or more processors, the one or more processors executing instructions stored in a memory of the client device to perform the method of any one of claims 1 to 14.

17. A computer-readable storage medium storing instructions executable by one or more processors of a computing system to perform the method of any one of claims 1 to 14.