Generating unintended memory measures for automatic speech recognition models.

Abstract

The unintentional memory measure can be used to determine whether an automatic speech recognition (ASR) model unintentionally memorized one or more phrases during training. Various implementations include generating one or more candidate transcripts based on the vocabulary of the ASR model. For example, the system can generate the candidate transcript by adding tokens of the vocabulary to a prior candidate transcript. Various implementations include processing the candidate transcript using a speech synthesis model to generate synthetic speech audio data including a synthetic speech of the candidate transcript. Additionally or alternatively, the synthetic speech audio data can be processed using the ASR model to generate an ASR output. Various implementations may include comparing the ASR output to the candidate transcript to generate a loss.

Description

[Background technology]

[0001] Voice-based user interfaces are increasingly being used to control computers and other electronic devices. Voice-based user interfaces have evolved from early basic interfaces that could only understand simple, direct commands to more advanced interfaces that can respond to natural language requests, understand context, and manage interactions or conversations with users. Many voice-based user interfaces perform speech recognition (e.g., using an automatic speech recognition model) on an utterance to generate corresponding text, perform a semantic analysis of the text to identify the meaning of the utterance, and / or perform one or more actions based on the identified meaning. However, automatic speech recognition models can unintentionally store the underlying data used to train the model. Summary of the Invention

[0002] Implementations described herein are directed to generating an unintentional memorization measure for an automatic speech recognition ("ASR") model that indicates the extent to which one or more phrases were unintentionally memorized by the ASR model during training. During training, the ASR model may unintentionally memorize rare or unique sequences in the training data. In some implementations, the unintentionally memorized sequences may include sensitive information about the user. For example, the ASR model may unintentionally memorize a training instance based on a user's utterance of "My username is PatentsForLife and my password is I Love Patents." After training the ASR model, the system may process an utterance of "My username is PatentsForLife and my password is I Love Patents" by a user (the same user as the speaker of the training instance and / or an additional user) as an additional instance and generate a textual representation of "My username is PatentsForLife and my password is I Love Patents." In some implementations, the ASR model can generate the text representation by generating one or more probabilities indicating the probability that a probability word(s) is captured in the utterance. For example, the ASR model can generate several candidate text representations of the utterance and select the text representation of the utterance based on the generated probabilities.

[0003] If the ASR model unintentionally stores a training utterance, the ASR model may generate a very high probability corresponding to a text representation of the stored utterance. For example, the system may determine that the ASR model unintentionally stores a text representation when the probability of the text representation being included in the utterance meets a threshold. For example, the system may determine that the ASR model unintentionally stores a text representation when the corresponding probability is greater than 0.99, greater than 0.95, greater than 0.9, greater than 0.85, meets one or more additional thresholds, and / or meets a combination thereof. In other words, the system may determine that the ASR model unintentionally stores a text representation when the probability generated using the ASR model is very high.

[0004] In some implementations, the system may determine that the probability of a text representation of "My username is PatentsForLife and my password is I LovePatents" (e.g., generated by processing audio data capturing the speech with the ASR model) is 0.98. In some of those implementations, the system may determine whether 0.98 meets a threshold of 0.95, where a probability greater than the threshold indicates that the ASR model has unintentionally stored the utterance. In response to determining that the probability 0.98 exceeds the threshold of 0.95, the system may determine that the ASR model has unintentionally stored the utterance "My username is PatentsForLife and my password is I Love Patents."

[0005] In some implementations, in generating the unintended memory measure, the system can generate a set of candidate transcripts, each candidate transcript indicating a likelihood of unintended memory by the ASR model. The system can generate the set of candidate transcripts based on a vocabulary of the ASR model. In some implementations, the candidate transcripts can be generated based on a prior candidate transcript and tokens of the vocabulary of the ASR model. The tokens of the vocabulary can include one or more words, one or more word fragments, one or more phonemes, one or more characters, one or more additional portions of the vocabulary, and / or combinations thereof. For example, the system can generate candidate transcripts by appending words in the vocabulary to a prior candidate transcript (e.g., one that meets one or more criteria). Additionally or alternatively, the system can generate multiple candidate transcripts based on the same prior candidate transcript by appending different portions of the vocabulary to each candidate transcript.

[0006] As a specific example, the system may generate multiple candidate transcripts based on a leading candidate transcript of "what does the." Adding the word "mouse" to the leading candidate transcript of "what" may generate a first candidate transcript of "what does the mouse." Similarly, adding the word "fish" to the leading candidate transcript of "what" may generate a second candidate transcript of "what does the fish." Further, adding the word "seal" to the leading candidate transcript of "what" may generate a third candidate transcript of "What does the seal." Adding additional word(s) to the leading candidate transcript may generate one or more additional or alternative candidate transcripts.

[0007] In some implementations, the candidate transcript can be processed using a speech synthesis model to generate synthetic speech audio data including a synthetic speech of the candidate transcript. The audio data can be processed using an ASR model to generate an ASR output (e.g., a candidate text representation of the synthetic speech, a probability measure corresponding to the candidate text representation of the synthetic speech, and a probability measure corresponding to each word of the candidate text representation). In some implementations, a loss can be generated by comparing the candidate transcript to the corresponding ASR output.

[0008] Taking the above example further, a first candidate transcript, "What are mice?", may be processed through a speech synthesis model to generate a first synthetic speech audio data including a synthesized speech representation of "What are mice?". Similarly, a second candidate transcript, "What are fish?", may be processed through a speech synthesis model to generate a second synthetic speech audio data including a synthesized speech representation of "What are fish?". Additionally or alternatively, a third candidate transcript, "What are seals?", may be processed through a speech synthesis model to generate a third synthetic speech audio data including a synthesized speech representation of "What are seals?".

[0009] In some implementations, the system can process synthetic speech audio data including synthetic speech of a candidate transcript with an ASR model to generate a corresponding ASR output. For example, the system can process first synthetic speech audio data with the ASR model to generate a first ASR output including a first candidate text representation of "what is a mouse," the system can process second synthetic speech audio data with the ASR model to generate a second ASR output including a second candidate text representation of "what is a fish," and the system can process third synthetic speech audio data with the ASR model to generate a third ASR output including a third candidate text representation of "what is a seal."

[0010] In some implementations, the system can determine the losses based on comparing the candidate transcripts to the ASR output. For example, the system can determine a first loss as 0.05 based on comparing a first candidate transcript "what is a mouse" to the first ASR output, the system can determine a second loss as 0.1 based on comparing a second candidate transcript "what is a fish" to the second ASR output, and the system can determine a third loss as 0.01 based on comparing a third candidate transcript "what is a seal" to the third ASR output.

[0011] According to some implementations, the system can use the generated losses in various ways. For example, the system can determine whether to use a candidate transcript to generate further candidate transcripts by adding one or more additional tokens from the ASR model's vocabulary to the candidate transcript. In some implementations, the system can compare losses of multiple candidate transcripts and select one or more of the candidate transcripts whose losses meet one or more of the following criteria: least loss, least two losses, least three losses, one or more additional criteria, and / or combinations thereof.

[0012] For example, the system may compare the above losses of a first loss of 0.05, a second loss of 0.1, and a third loss of 0.01, and based on the smallest loss of 0.01, select a third candidate transcript, "what does the seal say", to further extend it. For example, the system may extend the third candidate transcript, "what does the seal say", with additional words from the vocabulary of the ASR model to generate a further candidate transcript, "what does the seal say". In some implementations, the system may generate multiple further candidate transcripts, each further transcript extended with a different word from the vocabulary. For example, the system may extend "what does the seal say" with the word "say" to generate a first further candidate transcript, "what does the seal say". The system may extend "what does the seal say" with the word "eat" to generate a second further candidate transcript, "what does the seal eat". The system can also generate a third further candidate transcript of "what does the seal dig." In some of these implementations, the system can process each further candidate transcript with a speech synthesis model to generate corresponding further synthesized speech audio data, and can process the further synthesized speech audio data with an ASR model to generate corresponding further ASR output. Additionally or alternatively, the system can determine a further loss corresponding to each further candidate transcript based on comparing the further candidate transcript to a corresponding different ASR loss. The further loss can be used according to various methods described herein.

[0013] Additionally or alternatively, the system can use the loss corresponding to a particular candidate transcript to determine whether the ASR model unintentionally memorized one or more occurrences of a user speaking the candidate transcript in the training data used to train the ASR model. For example, the system can identify a set of candidate transcripts based on the corresponding losses, e.g., identify a set of candidate transcripts with minimal corresponding losses as described above. In some implementations, the system can compare each identified candidate transcript with a set of training instances used to train the ASR model. A candidate transcript corresponding to a training instance can indicate that the ASR model may have unintentionally memorized the training instance. For example, the system can compare a candidate transcript of "what does the seal say" with the training instances used to train the ASR model. The system can identify one or more occurrences (or none) of a user speaking "what does the seal say" in the training instances.

[0014] In some implementations, the memorization measure can be based on the number of candidate transcripts in the training data used to train the ASR model that correspond to one or more occurrences of a user speaking the candidate transcript. For example, the unintentional memorization measure can be based on the number of candidate transcripts that correspond to one or more training instances. In some of these implementations, a lower memorization measure indicates less unintentional memorization by the ASR model.

[0015] The system can use the memory measure in evaluating one or more ASR models. In some implementations, the system can use the memory measure to compare several ASR models. For example, the system can determine a first memory measure corresponding to a first ASR model trained using a first set of training instances, and determine a second memory model corresponding to a second ASR model trained using a second set of training instances. The system can compare the unintended memory in the first ASR model to the second ASR model based on a comparison of the first memory measure and the second memory measure.

[0016] Additionally or alternatively, the system can use the memory measure to determine whether additional training resulted in a change (e.g., an increase, a decrease, no change, etc.) in the unintended memory of the ASR model. For example, the system can generate a leading memory measure corresponding to the ASR model prior to additional training. A subsequent memory measure corresponding to the ASR model prior to the additional training can be generated. In some implementations, additional training of the ASR model can be evaluated based on a comparison of the leading and subsequent memory measures.

[0017] In some implementations, the ASR model may be trained based on one or more federated learning techniques. For example, an on-device ASR model may be stored locally on a user's client device, and a cloud-based global ASR model may be stored remotely on a remote system (e.g., a server, server cluster, etc.). The client device may process inputs detected at the client device to generate predictions and compare the predictions with ground truth to generate client gradients. Additionally or alternatively, the client device may send updates to the remote system for the client ASR model based on the client gradients. The remote system may use the client model updates and any additional client ASR model updates from the additional client devices to update one or more weights of the global ASR model. In some implementations, the system may send the updated global ASR model and / or updated weights of the global ASR model to the client device and / or other client devices. Each client device may replace the on-device ASR model with the global ASR model or replace the weights of the on-device ASR model with the updated weights of the global ASR model, thereby updating the on-device ASR model.

[0018] In some implementations, the system can determine an initial memory measure indicative of one or more unintended memories by the global ASR model before updating the global ASR model based on the update(s) of the client ASR model. After updating one or more weights of the global ASR model, the system can determine an updated memory measure. In some implementations, the system can determine whether to send an updated global ASR model and / or updated weight(s) of the global ASR model based on comparing the initial memory measure and the updated memory measure.

[0019] In some of those implementations, the system can determine to send an updated global ASR model based on comparing the initial memory measure and the updated memory measure. For example, the system can determine that the initial memory measure is 10 and the updated memory measure is 5, which can indicate that the updated global ASR model has less unintended memory compared to the initial global ASR model. In some implementations, the system can send the updated global ASR model based on a determination that the updated global ASR model reduces the amount of unintended memory. Additionally or alternatively, the system can send the updated global ASR model based on a determination that the updated global ASR model does not change the amount of unintended memory, increases the amount of unintended memory within a threshold, etc.

[0020] In some other implementations, the system may determine not to send an updated global ASR model based on comparing the initial memory measure to the updated memory measure. For example, the system may determine that the initial memory measure is 100 and the updated memory measure is 5000, which may indicate that the updated global ASR model has more unintended memory compared to the initial global ASR model. In other words, an increase in the updated memory measure may indicate that the ASR model is more likely to unintentionally leak sensitive data. In some of those implementations, the system may determine not to update the client ASR model based on determining that the updated global ASR model has an increased amount of unintended memory compared to the initial ASR model.

[0021] Further, in some implementations, one or more target training instances may be inserted into the training data used to train the ASR model. The target training instances may include atypical phrases, phrases likely to be unintentionally memorized, phrases memorized by other systems, etc. In some implementations, the system may compare the candidate transcripts to the target training instances to determine whether the ASR model unintentionally memorized the target training instance during training. For example, a target training instance of "My name is John Doe and my credit card number is 1234567890" may be used to train the ASR model. The system may generate a set of candidate transcripts as described herein. In some of these implementations, the system may determine whether a candidate transcript is generated that indicates that the ASR model memorized the target training instance of "My name is John Doe and my credit card number is 1234567890."

[0022] In some implementations, the system can compare the probability of a particular candidate transcript generated using the ASR model with a measure(s) for the particular candidate transcript generated using the additional ASR model(s). For example, the measure(s) can be based on one or more probabilities, each probability being a corresponding probability of a particular candidate transcript generated using a corresponding additional ASR model. For example, the system can compare the probability generated by the ASR model indicating the likelihood of a particular candidate transcript being included in the audio data with further probabilities generated using the additional ASR models indicating the likelihood of the candidate transcript being included in the audio data. If a candidate transcript has a high probability across multiple ASR models, the candidate transcript is less likely to indicate unintentional memory and / or is more likely to be a commonly spoken phrase. In other words, in determining whether the probability of a particular candidate transcript generated using the ASR model indicates unintentional memory, the probability can be adjusted based on one or more other probabilities similarly generated using one or more additional ASR models for the particular candidate transcript. In these and other methods, whether a particular probability indicates an unintended memory is specific to the corresponding transcript and may vary from transcript to transcript depending on adjustments based on one or more other probabilities from other ASR model(s).

[0023] As a specific example, audio data capturing the utterance "Turn off the kitchen light" may be processed through a first ASR model to generate a candidate transcript "Turn off the kitchen light" with a corresponding probability of 0.98. Similarly, audio data capturing the utterance "Turn off the kitchen light" may be processed through a second ASR model to generate a candidate transcript "Turn off the kitchen light" with a corresponding probability of 0.99. Furthermore, audio data capturing the utterance "Turn off the kitchen light" may be processed through a third ASR model to generate a candidate transcript "Turn off the kitchen light" with a corresponding probability of 0.997. The candidate transcript has high probabilities corresponding to the first ASR model (0.98), the second ASR model (0.99), and the third ASR model (0.997). In some implementations, this high probability across several ASR models indicates that the candidate transcript of "Turn off the kitchen light" is a commonly spoken phrase and / or is not an unintentional memory by one or more of the ASR models.

[0024] As an example, the system can compare the probabilities generated using additional ASR model(s) (e.g., a second ASR model, a third ASR model, one or more additional ASR models, etc.) to determine an expected probability measure. For example, the expected probability measure can be an average probability measure (e.g., the average 0.9935 of the probability measure of the second ASR model (0.99) and the probability measure of the third ASR model (0.997)), a mean probability measure, a weighted probability measure, one or more probability measures, and / or combinations thereof. The system can compare the expected probability measure with a probability measure generated using the ASR model the system is processing and a corresponding probability measure to identify unintended storage. In some implementations, the system can determine whether the ASR model unintendedly stored the candidate transcript based on determining whether the corresponding probability measure and the expected probability measure satisfy one or more conditions, e.g., whether the corresponding measure is within a threshold distance of the expected probability measure. In some implementations, if the difference between the corresponding probability measure and the expected probability measure exceeds a threshold (e.g., the difference is greater than 0.5, 0.2, 0.1), the system can determine that the ASR model has unintentionally stored the candidate transcript. In other implementations, if the difference between the corresponding probability measure and the expected probability measure falls below a threshold (e.g., the difference is less than 0.05, 0.1, 0.15, 0.2), the system can determine that the ASR model has not unintentionally stored the candidate transcript.

[0025] For example, the system can compare the probability measure of the first ASR model (0.98) to an expected probability measure (0.9935) based on an average of the probability measures of the second and third ASR models. Additionally or alternatively, the system can determine whether the difference between the corresponding probability measure and the expected probability measure (0.0035) is above or below a threshold of 0.1. If the system determines that the difference (0.0035) is below the threshold (0.1), it can determine that the first ASR model has not unintentionally stored the candidate transcript.

[0026] Conversely, audio data capturing the utterance "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'" can be processed through a first ASR model to generate a candidate transcript "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'" with a corresponding probability of 0.999. Additionally or alternatively, audio data capturing the utterance "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'" can be processed through a second ASR model to generate a candidate transcript "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'" with a corresponding probability of 0.7. Further, audio data capturing the utterance "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'" may be processed by a third ASR model to generate a candidate transcript "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'" with a corresponding probability of 0.79. In some implementations, a high probability (e.g., 0.999) corresponding to the first ASR model and a (relatively) low probability (e.g., 0.7 and 0.79, respectively) corresponding to the second and third ASR models may indicate that the first ASR model unintentionally memorized one or more training instances capturing the user's utterance "My username is 'PatentsAreTheBest' and my password is 'ILovePatents'".

[0027] As an additional example, the system can compare probabilities generated using additional ASR model(s) (e.g., a second ASR model, a third ASR model, one or more additional ASR models, etc.). For example, the expected probability can be an average probability measure, a mean probability measure (e.g., the average 0.745 of the probability measure (0.7) of the second ASR model and the probability measure (0.79) of the third ASR model), a weighted probability measure, one or more probability measures, and / or a combination thereof. Similarly, the system can compare the expected probability measure to a probability measure generated using the ASR model the system is processing to identify unintentionally stored content. In some implementations, the system can determine whether the ASR model unintentionally stored the candidate transcript based on determining whether the corresponding probability measure and the expected probability measure meet one or more conditions, e.g., whether the corresponding measure is within a threshold distance of the expected probability measure. In some implementations, if the difference between the corresponding probability measure and the expected probability measure exceeds a threshold (e.g., the difference is greater than 0.5, 0.2, 0.1), the system can determine that the ASR model has unintentionally stored the candidate transcript. In other implementations, if the difference between the corresponding probability measure and the expected probability measure is below a threshold (e.g., the difference is less than 0.05, 0.1, 0.15, 0.2), the system can determine that the ASR model has not unintentionally stored the candidate transcript.

[0028] For example, the system can compare the probability of the first ASR model (0.999) to an expected probability measure (0.745) based on an average of the probabilities of the second and third ASR models. In some implementations, the system can determine whether the difference (0.254) between the corresponding probability measure and the expected probability measure exceeds or falls below a threshold of 0.2. In response to determining that the difference (0.254) exceeds the threshold (0.2), the system can determine that the first ASR model unintentionally stored the candidate transcript.

[0029] Although the above examples are described with respect to comparing a probability measure generated using a particular ASR model with one or more additional probability measures generated using one or more ASR models, this is merely an example. In some implementations, the system can compare one or more portions of an ASR output, e.g., an output generated by processing synthetic speech audio data including synthetic speech of a candidate transcript using an ASR model, with an additional ASR output generated based on processing the synthetic speech audio data using one or more additional ASR models. The ASR output can include a probability indicating the likelihood that the candidate transcript is included in the synthetic speech audio data, a confidence score indicating the confidence of the ASR output, a representation of losses generated during training of the ASR model, one or more additional or alternative outputs, and / or combinations thereof. For example, losses generated during training of a first ASR model can be compared with losses generated during training of a second ASR model and losses generated during training of a third ASR model (e.g., an average loss based on losses corresponding to the second ASR model and losses corresponding to the third ASR model). In some implementations, the system can determine whether a candidate transcript was unintentionally stored by a first ASR model based on whether a difference between a loss corresponding to the first ASR model and an average loss (based on the loss corresponding to the second ASR model and the loss corresponding to the third ASR model) satisfies one or more conditions, e.g., whether the difference meets a threshold (discussed above with respect to the probability measure).

[0030] In some implementations, the system can generate a set of candidate transcripts that may have been unintentionally stored by a third-party ASR model. For example, the system can use a third-party ASR model to generate a set of candidate transcripts that have a high correspondence probability. The third party can determine whether each candidate transcript in the set of candidate transcripts was unintentionally stored by the third-party ASR model.

[0031] Thus, various implementations illustrate techniques for identifying unintentional memory in an ASR model. The ASR model can be trained on real-world training instances. These training instances may include unintentionally sensitive information (e.g., passwords, credit card numbers, birthdates, etc.). Sensitive information unintentionally stored by the ASR model may be unintentionally leaked. Some implementations described herein can be used to identify a set of candidate transcripts that may be unintentionally stored by a corresponding ASR model. In some of those implementations, a memory measure for the ASR model can be generated based on the set of candidate transcripts. The memory measure can be used, for example, to identify whether personal information is likely to be leaked if the ASR model is deployed. Additionally or alternatively, by preventing unintentional leaking of sensitive information, users can feel more comfortable sharing interactions with the system and further improve the ASR model.

[0032] The above description is intended only to provide a summary of some implementations of the present disclosure. These and other implementations of the present technology are disclosed in further detail below. It should be understood that all combinations of the above concepts and additional concepts described in further detail below are contemplated as part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter described at the end of this disclosure are contemplated as part of the subject matter disclosed herein. [Brief description of the drawings]

[0033] [Figure 1A] 1 illustrates an example of generating a set of candidate transcripts according to various implementations disclosed herein. [Figure 1B] 13A-13C illustrate additional examples of generating a set of candidate transcripts according to various implementations disclosed herein. [Figure 1C] 1 illustrates an example of generating a memory measure according to various implementations disclosed herein. [Diagram 2] FIG. 1 is a block diagram illustrating an example of generating a set of candidate transcripts according to various implementations disclosed herein. [Diagram 3] 1 illustrates an example environment in which various implementations disclosed herein can be practiced. [Figure 4] 1 is a flowchart illustrating a process of performing an action based on a set of candidate transcripts and / or a memory measure, according to various implementations disclosed herein. [Diagram 5] 1 is a flowchart illustrating a process for determining whether an ASR model unintentionally remembers that a human spoke a candidate transcript, according to various implementations disclosed herein. [Figure 6] 1 illustrates another example environment in which various implementations disclosed herein may be implemented. [Figure 7] 1 illustrates an example of an architecture for a computing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Referring to the figures, FIG. 1A illustrates an example 100 for generating a set of candidate transcripts according to various implementations. The example 100 includes using an augmentation engine 104 to process a leading candidate transcript 102 and generate a candidate transcript 106. In some implementations, in a first iteration, the leading candidate transcript may be seeded with tokens from a vocabulary corresponding to the ASR model. In some implementations, the tokens may be one or more words, one or more word fragments, one or more characters, one or more phonemes, one or more additional portions of the vocabulary, and / or combinations thereof. In some implementations, the augmentation engine 104 may augment the leading candidate transcript 102 with additional tokens from the vocabulary to generate the candidate transcript 106. For example, the system may augment a leading candidate transcript of "my name is John" with the word "Doe" from the vocabulary of the ASR model to generate a candidate transcript of "my name is John Doe." In some implementations, the system can generate multiple candidate transcripts from the same leading candidate transcript. An example of generating multiple candidate transcripts from a leading candidate transcript is described herein in connection with FIG. 1B.

[0035] Additionally or alternatively, the system may process the candidate transcript 106 using a speech synthesis model 108 to generate synthetic speech audio data 110. For example, the system may process a candidate text representation of "My name is John Doe" using a speech synthesis model to generate synthetic speech audio data including a synthesized utterance of "My name is John Doe." The synthetic speech audio data 110 may be processed using an ASR model 112 to generate an ASR output 114. In some implementations, a vocabulary corresponding to the ASR model 112 may be used by the augmentation engine 104 in generating the candidate transcript 106. The ASR output 114 may include a candidate text representation of synthetic speech, one or more alternative candidate text representations of synthetic speech, a probability of the candidate text representation, a probability of one or more words of the candidate text representation, one or more additional outputs, and / or combinations thereof.

[0036] In some implementations, the system can process the candidate transcript 106 and the ASR output 114 using a loss engine 116 to generate a loss 118 that corresponds to the candidate transcript 106. For example, the loss engine 116 can compare the text of the candidate transcript 106 with the text of the candidate text representation in the ASR output 114. In some implementations, the system can generate the loss 118 based on a comparison of the text of the candidate transcript 106 with the text of the candidate text representation in the ASR output 114.

[0037] In some implementations, the system can use the transcript engine 120 to determine whether to add the candidate transcript 106 to a set of candidate transcripts 124. In some of those implementations, each candidate transcript in the set of candidate transcripts 124 can be used as a preceding candidate transcript in the next iteration of generating the candidate transcript(s). For example, the transcript engine 120 can select one or more candidate transcripts based on the corresponding loss 118. For example, the system can select a predetermined number of candidate transcripts with the smallest loss, a predetermined number of candidate transcripts with the largest loss, each candidate transcript less than a predetermined loss, each candidate transcript greater than a predetermined loss, one or more additional loss measures, and / or combinations thereof. In some other implementations, the transcript engine 120 can determine whether the candidate transcript reaches a predetermined length and / or whether the candidate transcript includes a sentence-ending character within the vocabulary of the ASR model.

[0038] FIG. 1B illustrates an example 140 for generating multiple candidate transcripts based on the same prior candidate transcript, according to various implementations. The example 140 includes processing the prior candidate transcript 106 using the augmentation engine 104 to generate candidate transcript A 106A and candidate transcript B 106B. The system of the example 140 generates two candidate transcripts 106A and 106B based on the prior candidate transcript, whereas the example 100 generates a single candidate transcript 106 based on the prior candidate transcript. The candidate transcript A 106A can be processed using a speech synthesis model / ASR model / loss engine 142 to generate a corresponding loss A 118A. In some implementations, the speech synthesis model / ASR model / loss engine 142 can include the speech synthesis model 108, the ASR model 112, and the loss engine 116 described in connection with FIG. 1A. Similarly, candidate transcript B 106B may be processed using the speech synthesis model / ASR model / loss engine 142 to generate a corresponding loss B 118B.

[0039] In some implementations, the transcription engine 120 can select one or more of the candidate transcripts for further enhancement based on the corresponding losses as described herein (e.g., the system can select candidate transcript A 106A, candidate transcript B 106B, or candidate transcript A 106A and candidate transcript B 106B). Additionally or alternatively, the transcription engine 120 can determine whether the candidate transcripts are of sufficient length (e.g., reaching a predetermined length value, reaching end-of-sentence characters, etc.) and determine whether each of the candidate transcripts is an example of unintended memory based on the corresponding losses.

[0040] FIG. 1C illustrates an example 160 of generating a memory measure according to various implementations. The example 160 includes processing the set of candidate transcripts 124 using an unintentional memory engine 162 to generate an unintentional memory list 166. In some implementations, the unintentional memory engine 162 may compare the set of candidate transcripts 124 with one or more candidate transcripts generated based on the vocabulary of an alternative ASR model (not shown). For example, the system may compare the set of candidate transcripts 124 generated based on the ASR model 112 with a set of additional candidate transcripts (not shown) generated based on an additional ASR model (not shown), e.g., an additional ASR model trained using publicly available data (i.e., training instances based on public data). Sensitive data (e.g., passwords, credit card numbers, etc.) is less likely to be publicly available and therefore less likely to be included in training instances based on public data. In other words, a candidate transcript generated based on an additional ASR model trained using public data, the candidate transcript is less likely to be an unintentional memory. In some implementations, the unintended storage engine 162 can compare the set of candidate transcripts 124 to the additional set of transcripts. If a candidate transcript is included in the set of candidate transcripts 124 and is also included in the additional set of candidate transcripts, the candidate transcript is not unintended stored. In some of these implementations, the unintended storage engine 162 cannot include the candidate transcript in the unintended storage list 166.

[0041] Additionally or alternatively, the unintended memory engine 162 can generate a memory measure 164. In some implementations, the memory measure 164 can be based on the number of unintended memories in the unintended memory list 166. For example, the unintended memory engine 162 can generate 10 memory measures based on a list of 10 examples of unintended memories. In other words, each unintended memory can add 1 point to the unintended memory measure. However, this is merely exemplary, and the memory measure can be based on additional or alternative metrics, such as 0.5 points for each unintended memory, 1 point for each unintended memory that occurs multiple times, 1 point for each unique unintended memory, and / or combinations thereof.

[0042] FIG. 2 illustrates an example 200 of generating candidate transcripts according to various implementations. In some implementations, the candidate transcripts may be generated based on one or more candidate transcripts generated in a previous iteration. In a first iteration, the candidate transcripts may be seeded based on a token(s) of a vocabulary of an ASR model. In some implementations, the tokens of the vocabulary may include one or more words, one or more word fragments, one or more characters, one or more phonemes, one or more additional portions of the vocabulary, and / or combinations thereof. In the illustrated example 200, words of the vocabulary of the ASR model are used in generating the candidate transcripts. In some implementations, the system may generate an initial candidate transcript where each word in the vocabulary corresponds to a different candidate transcript (not shown). The example 200 includes an initial set of candidate transcripts 202, 204, . . . , 206 based on a prior candidate transcript of "WHAT." The initial set includes a candidate transcript for "WHAT DOES" 202, a candidate transcript for "WHAT PENGUIN" 204, and a candidate transcript for "WHAT CAN" 206. Although only three candidate transcripts are illustrated in the initial set of candidate transcripts, the system may include one or more additional candidate transcripts, each of which may be based on an additional token in the vocabulary of the ASR model.

[0043] The system can process an initial set of candidate transcripts 202, 204,..., 206 and generate losses corresponding to each candidate transcript. In some implementations, the losses corresponding to the candidate transcripts can be generated according to the process 500 of Figures 1A and 5 described herein. In some of those implementations, the losses can be generated using the loss engine 116 described herein.

[0044] In some implementations, the system can select one or more of the candidate transcripts for further processing based on the corresponding loss. In some of those implementations, the system can select one or more of the candidate transcripts with the smallest corresponding loss. For example, the candidate transcript 202 "WHAT DOES" can have a corresponding loss of 0.01, the candidate transcript 204 "WHAT PENGUIN" can have a corresponding loss of 0.05, and the candidate transcript 602 "WHAT CAN" can have a corresponding loss of 0.02. The system of the example 200 selects the two candidate transcripts with the smallest losses. In other words, the system selects the candidate transcript 202 "WHAT DOES" with a corresponding candidate loss of 0.01 (less than 0.02 and 0.05) and the candidate transcript 206 "WHAT CAN" with a corresponding candidate loss of 0.02 (less than 0.05). However, this is merely an example, and in some other implementations, the system can select a different number of candidate transcripts.

[0045] Candidate transcript 202, "WHAT DOES," may be extended with one or more additional words in the ASR vocabulary. As illustrated herein, candidate transcript 202 may be extended to generate candidate transcript 208, ..., candidate transcript 210. For example, candidate transcript 202, "WHAT DOES," may be extended with the word "THE" to generate candidate transcript 208, "WHAT DOES THE." Similarly, candidate transcript 202, "WHAT DOES," may be extended with the word "MY" to generate candidate transcript 210, "WHAT DOES MY." Additionally or alternatively, candidate transcript 206, "WHAT CAN," may be extended with one or more additional words in the ASR vocabulary to generate candidate transcript 212, ..., candidate transcript 214. For example, candidate transcript 206, "WHAT CAN," may be extended with the word "THE" to generate candidate transcript 212, "WHAT CAN THE." Similarly, the candidate transcript 206 "WHAT CAN" may be expanded with the word "JOHN" to generate the candidate transcript 214 "WHAT CAN JOHN."

[0046] The system can process the expanded set of candidate transcripts 208,...,210 and 212,...,214 to generate losses corresponding to each candidate transcript. In some implementations, the losses corresponding to the candidate transcripts can be generated according to the process 500 of FIG. 1A and FIG. 5 described herein. In some of those implementations, the losses can be generated using the loss engine 116 described herein. For example, the system can generate a loss of 0.1 corresponding to candidate transcript 208, a loss of 0.7 corresponding to candidate transcript 210, a loss of 0.2 corresponding to candidate transcript 212, and a loss of 0.4 corresponding to candidate transcript 214.

[0047] In example 200, the system selects the two candidate transcripts with the smallest corresponding losses for further expansion. In other words, the system may select candidate transcript 208 with a corresponding loss of 0.1 and candidate transcript 212 with a corresponding loss of 0.2.

[0048] In some implementations, the candidate transcript 208 "WHAT DOES THE" can be expanded with one or more words in the ASR vocabulary. For example, the candidate transcript 208 "WHAT DOES THE" can be expanded with the word "RABBIT" to generate a further expanded candidate transcript 216 "WHAT DOES THE RABBIT". Similarly, the candidate transcript 208 "WHAT DOES THE" can be expanded with the word "CAT" to generate a further expanded candidate transcript 218 "WHAT DOES THE CAT". Additionally or alternatively, the candidate transcript 212 "WHAT CAN THE" can be expanded with the word "RABBIT" to generate a further expanded candidate transcript 220 "WHAT CAN THE RABBIT". Similarly, the candidate transcript 212 "WHAT CAN THE" can be expanded with the word "TREE" to generate a further expanded candidate transcript 222 "WHAT CAN THE TREE".

[0049] In some implementations, the system can continue to generate expanded candidate transcripts until the transcript reaches a target length. In some of those implementations, the target length may be predefined. In some implementations, the system can continue to generate one or more expanded candidate transcripts until an end token (e.g., an end of a word, an end of a character, an end of a sentence, an end of a paragraph, etc.) is reached.

[0050] In some implementations, the system can determine a set of unintended memories based on the expanded candidate transcripts. For example, the system can process the candidate transcripts using the unintended memory engine 162 described herein to determine a set of unintended memories. Additionally or alternatively, the system can determine a memory measure based on the expanded candidate transcripts. In some implementations, the system can process the set of candidate transcripts using the unintended memory engine 162 described herein to determine a memory measure.

[0051] 3 illustrates a block diagram of an example environment 300 in which various implementations may be practiced. The example environment 300 includes a computing system 302 that may include an ASR engine 304, an augmentation engine 104, a lossy engine 116, a speech synthesis engine 306, a transcript engine 120, an unintended storage engine 162, and / or one or more additional engines. Additionally or alternatively, the computing system 302 may be associated with a speech synthesis model 108, an ASR model 112, a candidate transcript 124, and / or one or more additional components (not shown).

[0052] In some implementations, the computing system 302 may include user interface input / output devices (not shown), such as, for example, a physical keyboard, a touch screen (e.g., implementing a virtual keyboard or other text input mechanism), a microphone, a camera, a display screen, and / or speaker(s). Additionally or alternatively, the computing system 302 may include various sensors (not shown), such as an accelerometer, a gyroscope, a global positioning system (GPS), a pressure sensor, a light sensor, a distance sensor, a proximity sensor, a temperature sensor, one or more additional sensors, and / or combinations thereof. The user interface input / output devices may be incorporated into one or more computing systems 302 of a user. For example, a user's mobile phone may include a user interface input / output device, a standalone digital assistant hardware device may include a user interface input / output device, a first computing device may include a user interface input device(s), another computing device may include a user interface output device(s), etc. In some implementations, all or some aspects of the computing system 302 may be implemented in a computing system that also includes a user interface input / output device. In some implementations, the computing system 302 may include an automated assistant (not shown), all or some aspects of which may be implemented on a computing device(s) separate and remote from a client device with a user interface input / output device (e.g., all or some may be implemented in the "cloud"). In some of these implementations, aspects of the automated assistant may communicate with the computing device over one or more networks, such as a local area network (LAN) and / or a wide area network (WAN) (e.g., the Internet).

[0053] Non-limiting examples of computing system 302 include desktop computing devices, laptop computing devices, standalone hardware devices at least partially dedicated to an automated assistant, tablet computing devices, mobile phone computing devices, computing devices in vehicles (e.g., vehicle-mounted communication systems, vehicle-mounted entertainment systems, vehicle-mounted navigation systems, vehicle-mounted navigation systems), user wearable devices including computing devices (e.g., user's wristwatch with computing device, user's glasses with computing device, virtual reality or augmented reality computing devices). Additional and / or alternative computing systems may be provided. Computing system 302 may include one or more memories for storage of data and software applications, one or more processors for accessing data and executing applications, and other components that facilitate communication over a network. Operations performed by computing system 302 may be distributed across multiple computing devices. For example, computing programs running on one or more computers at one or more locations may be connected to each other over a network.

[0054] In some implementations, the ASR engine 304 can process a particular audio data instance using the ASR model 112 to generate a textual representation of speech contained in the particular audio data instance. For example, the ASR engine 304 can process the synthesized speech audio data 110 using the ASR model 112 to generate the ASR output 114.

[0055] In some implementations, the augmentation engine 104 can be used to generate one or more candidate transcripts 124. In some implementations, the candidate transcripts 124 can be generated based on a vocabulary corresponding to the ASR model 112. For example, the augmentation engine 104 can generate the one or more candidate transcripts 124 according to a process 400 described herein.

[0056] In some implementations, a loss engine 116 can be used to determine a loss between the candidate transcript and a textual representation of the candidate transcript captured in the synthetic speech audio data. For example, the system can use the loss engine 116 to generate a loss 118 based on comparing the candidate transcript 106 to an ASR output 114, where the ASR output 114 is based on processing the synthetic speech audio data 110 using the ASR model 112.

[0057] In some implementations, the speech synthesis engine 306 can process the candidate transcripts 106 using the speech synthesis model 108 to generate instances of synthetic speech audio data, where the synthetic speech audio data includes at least a synthetic representation of the corresponding candidate transcript. For example, the speech synthesis engine 306 can process the candidate transcripts 106 using the speech synthesis model 108 to generate instances of synthetic speech audio data 110.

[0058] In some implementations, the transcript engine 120 can be used to generate one or more candidate transcripts 124. In some of those implementations, the one or more candidate transcripts 124 may be generated based on the vocabulary of the ASR model 112. For example, the transcript engine 120 can generate the candidate transcripts as illustrated herein in connection with FIG.

[0059] In some implementations, the unintended memory engine 162 can be used to determine the memory measure based on the ASR model 112. In some implementations, the unintended memory engine 162 can generate the memory measure according to the process 400 of FIG. 4 described herein.

[0060] 4 is a flowchart illustrating an example process 400 for performing an action(s) based on a set of candidate transcripts and / or storage measures, according to various implementations described herein. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components of various computer systems, such as one or more components of computing system 302, client device 602, and / or computing system 710. Furthermore, while the operations of process 400 are shown in a particular order, this is not meant to be limiting. One or more operations may be rearranged, omitted, and / or added.

[0061] In block 402, the system generates a set of candidate transcripts based on the vocabulary of the ASR model. In some implementations, the ASR model unintentionally memorized one or more occurrences of a human speaking each candidate transcript while training the ASR model. In some implementations, the system can generate the set of candidate transcripts according to process 500 described herein.

[0062] In block 404, the system generates a memory measure based on the set of candidate transcripts of the ASR model. In some implementations, the system can use the unintentional memory engine 162 to process the set of candidate transcripts and generate the memory measure described herein. In some implementations, the system can compare the set of candidate transcripts generated based on the ASR model with a set of additional candidate transcripts generated based on an additional ASR model. In some of those implementations, the additional ASR model can be a model trained using publicly available training data (e.g., training data based on uploads to a video sharing platform, etc.). Candidate transcripts generated based on publicly available training data are less likely to include sensitive information and therefore less likely to indicate unintentional memory by the ASR model. In some implementations, the system can ignore a particular candidate transcript when generating the memory measure in block 404 if the particular candidate transcript is included in both the set of candidate transcripts and the set of additional candidate transcripts.

[0063] In some implementations, the memory measure may be generated based on the number of candidate transcripts in the set of candidate transcripts. In some implementations, the memory measure may be based on the total number of candidate transcripts in the set of candidate transcripts. For example, the system may generate a set of 50 candidate transcripts in block 402 and determine a memory measure of 50 based on the set of 50 candidate transcripts (i.e., assign a predetermined number of points to each candidate transcript).

[0064] In block 406, the system performs one or more actions based on the set of candidate transcripts and / or the memory measure. In some implementations, the system can use the memory measure in evaluating the training of the ASR model. For example, the system can determine an initial memory measure based on an initial version of the ASR model. The system can perform one or more training techniques to update the ASR model to generate an updated ASR model. After generating the updated ASR model, the system can generate an updated memory measure. In some implementations, the system can determine whether training has increased or decreased the unintended memory of the ASR model based on comparing the initial memory measure and the updated memory measure. Additionally or alternatively, the system can determine whether to release an updated version of the ASR model based on comparing the initial memory measure and the updated memory measure.

[0065] 5 is a flowchart illustrating an example process 400 for determining whether an ASR model unintentionally stored one or more occurrences of a human speaking a candidate transcript, according to various implementations described herein. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components of various computer systems, such as one or more components of computing system 302, client device 602, and / or computing system 710. Additionally, while the operations of process 500 are shown in a particular order, this is not intended to be limiting. One or more operations may be rearranged, omitted, and / or added.

[0066] In block 502, the system generates a candidate transcript by expanding the leading candidate transcript based on a vocabulary of the ASR model. In some implementations, in a first iteration, the leading candidate transcript may be initialized based on tokens of the ASR model's vocabulary. In some of these implementations, the tokens may include one or more characters, one or more phonemes, one or more words, one or more phrases, one or more additional tokens, and / or combinations thereof of the ASR model's vocabulary.

[0067] In some implementations, the leading candidate transcript may be expanded based on a token of the ASR model's vocabulary. In some of those implementations, multiple candidate transcripts may be generated based on a particular leading candidate transcript, and each candidate transcript may be generated by expanding the particular leading candidate transcript with a different token of the ASR model's vocabulary. In some implementations, the candidate transcript may be generated using the expansion engine 104 described herein. Additionally or alternatively, in some implementations, the candidate transcript may be generated according to FIG. 2 described herein. For example, the system may expand the leading candidate transcript 202 of "WHAT DOES" with the word "THE" to generate the candidate transcript 208 of "WHAT DOES THE" as described herein in connection with FIG. 2.

[0068] At block 504, the system generates synthetic speech audio data by processing the candidate transcript using a speech synthesis model. In some implementations, the synthetic speech audio data includes a synthesized spoken representation of the candidate transcript. In some implementations, the system can generate synthetic speech audio data by processing the candidate transcript using a speech synthesis engine 306 and / or a speech synthesis model 108 described herein. For example, the system can process the candidate transcript 208 of "WHAT DOES THE" using the speech synthesis model 108 to generate synthetic speech audio data including a synthesized spoken representation of "what does the."

[0069] In block 506, the system generates an ASR output by processing the synthetic speech audio data using the ASR model. In some implementations, the ASR output may include a candidate text representation of the synthetic speech of the candidate transcript. Additionally or alternatively, the ASR output may include a probability indicating that a token in the candidate text representation is likely to be a corresponding portion of the synthetic speech audio data. Furthermore, the ASR output may include one or more alternative candidate text representations of the synthetic speech included in the audio data. For example, the system may process synthetic speech audio data including a synthesized speech representation of "what does the" to generate a candidate text representation of the synthetic speech of "what does the". In some implementations, the synthetic speech audio data generated in block 504 may be processed using the ASR model 112 to generate the ASR output.

[0070] At block 508, the system generates a loss based on comparing the candidate transcript and the ASR output. For example, the system can compare the text of the candidate transcript and the text of the ASR output to generate the loss. In some implementations, the system can generate the loss using the loss engine 116 described herein.

[0071] In block 510, the system determines, based on the loss, whether the ASR model unintentionally memorized one or more occurrences of a human speaking the candidate transcript used in training the ASR model. In some implementations, the system can use the unintentional memorization engine 162 described herein to determine whether the ASR model unintentionally memorized one or more occurrences of a human speaking the candidate transcript in training the ASR model.

[0072] Referring now to Figure 6, an exemplary environment in which various implementations may be implemented is shown. Figure 6 is first described and includes a client computing device 602, which executes an instance of an automated assistant client 604. One or more cloud-based automated assistant components 610 may be implemented in one or more computing systems (collectively referred to as "cloud" computing systems) communicatively connected to the client device 602 via one or more local and / or wide area networks (e.g., the Internet), generally indicated at 608.

[0073] An instance of an automated assistant client 604, through interaction with one or more cloud-based automated assistant components 610, can form, from the user's perspective, an instance of a logical automated assistant 600 in which the user can engage in human-computer interaction. Such an instance of an automated assistant 600 is depicted in FIG. 6. Thus, in some implementations, it should be understood that a user engaging with an automated assistant client 604 running on a client device 602 is in effect engaging with his or her own instance of a logical automated assistant 600. For the sake of brevity and simplicity, the term "automated assistant" as used herein as "serving" a particular user often refers to the combination of an automated assistant client 604 running on a client device 602 operated by the user and one or more cloud-based automated assistant components 610 (which may be shared among multiple automated assistant clients on multiple client devices). It should also be understood that in some implementations, the automated assistant 600 can respond to requests from any user, regardless of whether a particular automated assistant 600 instance is actually "serving" that user.

[0074] The client computing device 602 may be, for example, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in a user's vehicle (e.g., a vehicle-mounted communication system, a vehicle-mounted entertainment system, a vehicle-mounted navigation system), a standalone interactive speaker, a smart appliance such as a smart television, and / or an apparatus including a computing device worn by a user (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual reality or augmented reality computing device). Additional and / or alternative client computing devices may be provided. In various implementations, the client computing device 602 may optionally operate one or more other applications, such as a messaging client (e.g., SMS, MMS, online chat), a browser, etc., in addition to the automated assistant client 604. In some of these various implementations, the one or more other applications may optionally interface (e.g., via an application programming interface) with the automated assistant 600 or may include their own instance of an automated assistant application (which may also interface with the cloud-based automated assistant component(s) 610).

[0075] The automated assistant 600 engages in a human-computer interaction session with the user via the user interface input and output devices of the client device 602. To protect the user's privacy and / or to conserve resources, in many situations, the automated assistant 600 will process the utterance entirely after the user explicitly invokes the automated assistant 600. The explicit invocation of the automated assistant 600 occurs in response to a specific user interface input received at the client device 602. For example, the user interface input that may invoke the automated assistant 600 via the client device 602 may optionally include the operation of hardware and / or virtual buttons on the client device 602. Additionally, the automated device client may include one or more local engines 606, such as an invocation engine operable to detect the presence of one or more uttered invocation phrases. The invocation engine may invoke the automated assistant 600 in response to detecting one of the spoken invocation phrases. For example, the invocation engine may invoke the automated assistant 600 in response to detecting a spoken invocation phrase such as "Hey, assistant," "Okay, assistant," and / or "Assistant." The paging engine may continuously process (e.g., when not in “inactive” mode) a stream of audio data frames based on output from one or more microphones of the client device 602 to monitor for the occurrence of a spoken invoke phrase. While monitoring for the occurrence of the invoke phrase, the paging engine discards (e.g., after temporarily storing in a buffer) audio data frames that do not include the spoken invoke phrase. However, if the paging engine detects the spoken invoke phrase in a processed audio data frame, the paging engine may invoke the automated assistant 600. As used herein, the term “invoking” may include enabling one or more previously disabled features of the automated assistant 600.For example, invoking the automated assistant 600 may include operating one or more local engines 606 and / or cloud-based automated assistant component 610 to further process the audio data frame in which the invocation phrase was detected and / or one or more subsequent audio data frames (where prior to the invocation, no further processing of the audio data frames had occurred).

[0076] The one or more local engine(s) 606 of the automated assistant 600 are optional and may include, for example, the enhancement engines described above, lossy engines, transcript engines, unintentional storage engines, local speech-to-text ("STT") engines (to convert captured speech to text), local text-to-speech ("TTS") engines (to convert text to speech), local natural language processors (to determine the meaning of speech and / or text converted from speech), and / or other local components. Because the client device 602 is relatively constrained in terms of computing resources (e.g., processor cycles, memory, batteries, etc.), the local engines 606 may be more limited in functionality than the corresponding engines included in the cloud-based automated assistant component 610.

[0077] The cloud-based automated assistant component 610 leverages the virtually unlimited resources of the cloud to provide more robust and accurate processing of audio data and / or other user interface inputs compared to corresponding engines in the local engine(s) 606. Again, in various implementations, the client device 602 can provide audio data and / or other data to the cloud-based automated assistant component 610 in response to the invocation engine detecting a spoken invocation phrase or other explicit invocation of the automated assistant 600.

[0078] The illustrated cloud-based automated assistant component 610 includes a cloud-based TTS module 612, a cloud-based STT module 614, a natural language processor 616, a dialogue state tracker 618, and a dialogue manager 620. In some implementations, one or more of the engines and / or modules of the automated assistant 600 may be omitted, integrated, and / or implemented in a component separate from the automated assistant 600. Additionally, in some implementations, the automated assistant 600 may include additional and / or alternative engines and / or modules. The cloud-based STT module 614 may convert the audio data to text, which may be provided to the natural language processor 616.

[0079] The cloud-based TTS module 612 can convert text data (e.g., natural language responses generated by the automated assistant 600) into computer-generated voice output. In some implementations, the TTS module 612 can provide the computer-generated voice output to the client device 602, for example, directly output using one or more speakers. In other implementations, the text data (e.g., natural language responses) generated by the automated assistant 600 can be provided to one of the local engine(s) 606, which can convert the text data into computer-generated speech that is output locally.

[0080] The natural language processor 616 of the automated assistant 600 processes the free-form natural language input and generates annotated output based on the natural language input for use by one or more other components of the automated assistant 600. For example, the natural language processor 616 can process natural language free-form input that is a text input into which audio data provided by a user via the client device 602 has been converted by the STT module 614. The generated annotated output may include one or more annotations of the natural language input and, optionally, one or more (e.g., all) of the terms of the natural language input.

[0081] In some implementations, the natural language processor 616 is configured to identify and annotate various types of grammatical information in the natural language input. In some implementations, the natural language processor 616 may additionally and / or alternatively include an entity tagger (not shown) that is configured to annotate one or more segments with entity references, such as people (e.g., literary characters, famous people, public figures, etc.), organizations, places (real or fictional), etc. In some implementations, the natural language processor 816 may additionally and / or alternatively include a coreference resolver function (not shown) that is configured to group or "cluster" references to the same entity based on one or more contextual clues. For example, the coreference resolver function may be used to resolve "there" to "virtual cafe" in the natural language input "The last time I ate at the virtual cafe, I liked it there." In some implementations, one or more components of the natural language processor 616 may rely on annotations from one or more other components of the natural language processor 616. In some implementations, when processing a particular natural language input, one or more components of the natural language processor 616 may use associated input prior and / or other relevant data other than the particular natural language input to determine one or more annotations.

[0082] In some implementations, the dialogue state tracker 618 may be configured to track a "dialogue state" including, for example, the belief state of one or more users' goals (or "intent") over the course of a human-computer dialogue session and / or across multiple dialogue sessions. In determining the dialogue state, some dialogue state trackers may attempt to determine the most likely value(s) of the slot(s) instantiated in the dialogue based on user and system utterances during the dialogue session. Some techniques utilize a fixed ontology that defines a set of slots and a set of values ​​associated with those slots. Some techniques may additionally and / or alternatively specialize for individual slots and / or domains. For example, some techniques may require training a model for each slot type in each domain.

[0083] The dialogue manager 620 is configured to map the current dialogue state, for example provided by the dialogue state tracker 618, to one or more "response actions" from a number of candidate response actions, which may then be executed by the automated assistant 600. The response actions may take a variety of forms depending on the current dialogue state. For example, initial and intermediate dialogue states corresponding to turns in a dialogue session prior to a final turn (e.g., when the user's final desired task is performed) may be mapped to various response actions including the output of additional natural language dialogue by the automated assistant 600. This response dialogue may include, for example, a request to the user to provide parameters (i.e., fill a slot) for some action that the dialogue state tracker 618 determines the user is about to perform. In some implementations, the response actions may include a "request" (e.g., asking for parameters to fill a slot), an "offer" (e.g., making a suggestion for an action or behavior to the user), a "selection", a "notify" (e.g., providing the user with the requested information), a "no match" (e.g., informing the user that the user's last input was not understood), a command to a peripheral device (e.g., turn off a light bulb), etc.

[0084] 7 is a block diagram of an example computing device 710 that may optionally be utilized to perform one or more aspects of the techniques described herein. In some implementations, one or more of the client computing devices and / or other component(s) may include one or more components of the example computing device 710.

[0085] The computing device 710 typically includes at least one processor 714 that communicates with a number of peripherals via a bus subsystem 712. These peripherals may include a storage subsystem 724 (e.g., a memory subsystem 925 and a file storage subsystem 726), a user interface output device 720, a user interface input device 922, and a network interface subsystem 716. The input and output devices enable user interaction with the computing device 710. The network interface subsystem 716 provides an interface to external networks and is connected to corresponding interface devices in other computing devices.

[0086] The user interface input devices 722 may include a keyboard, a pointing device such as a mouse, a trackball, a touchpad, or a graphics tablet, a scanner, a touch screen integrated into a display, a voice input device such as a voice recognition system or a microphone, and / or other types of input devices. In general, use of the term "input device" is intended to include any type of device or method for inputting information into the computing device 710 or a communications network.

[0087] The user interface output devices 720 may include a display subsystem, a printer, a facsimile, or a non-visual display such as an audio output device. The display subsystem may include a flat panel device such as a cathode ray tube ("CRT"), a liquid crystal display ("LCD"), a projection device, or other mechanism for creating a visible image. The display subsystem may provide a non-visual indication, such as via an audio output device. In general, use of the term "output device" is intended to include any type of device or method for outputting information from the computing device 710 to a user or to another machine or computing device.

[0088] Storage subsystem 724 stores programs and data structures that provide the functionality of some or all of the modules described herein. For example, storage subsystem 724 may include logic for performing one or more selected aspects of the processes of Figures 4 and / or 5, and may also include logic for implementing the various components depicted in Figures 3 and / or 6.

[0089] These software modules typically execute on the processor 714 alone or in combination with other processors. The memory 725 used by the storage subsystem 724 includes multiple memories, such as a main random access memory ("RAM") 730 for storing instructions and data during program execution, and a read-only memory ("ROM") 732 in which fixed instructions are stored. The file storage subsystem 726 provides persistent storage of program files and data files, and may include a hard disk drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of a particular implementation may be stored in a file storage subsystem 926 within the storage subsystem 724 or on another machine accessible to the processor(s) 714.

[0090] Bus subsystem 712 provides a mechanism that allows the various components and subsystems of computing device 710 to communicate with each other in an intended manner. Although bus subsystem 712 is illustrated as a single bus, other implementations of the bus subsystem may use multiple buses.

[0091] Computing device 710 may be various types of devices, such as a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of computing device 710 shown in Figure 7 is intended only as a specific example to illustrate some implementations. Various other configurations of computing device 710 are possible having more or fewer components than the computing device shown in Figure 7.

[0092] In situations where the systems described herein collect or may utilize personal information about users (also referred to as "participants"), users may be provided with the opportunity to control whether a program or feature collects user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current geographic location) and whether or how the user receives content from content servers that is more relevant to the user. Also, certain data may be processed in one or more ways before being stored or used such that any personally identifiable information is removed. For example, a user's identity may be processed such that no personally identifiable information about the user is revealed, or if a user's geographic location (e.g., to a city, zip code, or state level) is obtained, the geographic location information may be generalized such that the user's specific geographic location cannot be determined. Thus, users may be able to control how information about them is collected and / or utilized.

[0093] In some implementations, a method is provided that is implemented by one or more processors, the method including generating a candidate transcript using a vocabulary corresponding to an automatic speech recognition ("ASR") model. In some implementations, the method further includes generating synthetic speech audio data including a synthetic speech of the candidate transcript based on processing the candidate transcript using a speech synthesis model. In some implementations, the method further includes processing the synthetic speech audio data using the ASR model to generate an ASR output that reflects a predicted textual representation of the synthetic speech. In some implementations, the method further includes generating a loss based on comparing the ASR output to the candidate transcript. In some implementations, the method further includes determining whether the ASR model unintentionally memorized one or more occurrences of a human speaking the corresponding candidate transcript in training data used to train the ASR model based on the loss. In some implementations, the method further includes generating an overall memory measure in response to determining whether the ASR model unintentionally memorized one or more occurrences.

[0094] The techniques of these and other implementations may include one or more of the following features.

[0095] In some implementations, generating a candidate transcript using a vocabulary corresponding to the ASR model includes generating a prior candidate transcript using a vocabulary corresponding to the ASR model prior to generating the candidate transcript, the prior candidate transcript including words at the beginning of the candidate transcript but lacking one or more ending words of the candidate transcript. In some implementations, the method further includes generating prior synthetic speech audio data including a prior synthetic speech of the prior candidate transcript based on processing the prior candidate transcript using a speech synthesis model. In some implementations, the method further includes processing the prior synthetic speech audio data using the ASR model to generate a prior ASR output reflecting a prior predictive text representation of the synthetic speech. In some implementations, the method further includes generating a prior loss based on a comparison of the prior ASR output and the prior candidate transcript. In some implementations, the method further includes selecting a prior candidate transcript for augmentation based on the prior loss. In some versions of these implementations, the method further includes generating an additional prior candidate transcript using a vocabulary corresponding to the ASR model prior to generating the candidate transcript, the additional prior candidate transcript including words at the beginning of the prior candidate transcript but lacking one or more ending words of the prior candidate transcript. In some implementations, the method further includes generating additional prior synthetic speech audio data including a synthetic speech of the additional prior candidate transcript based on processing the additional prior candidate transcript using the speech synthesis model. In some implementations, the method further includes processing the additional prior synthetic speech audio data using the ASR model to generate an additional prior ASR output reflecting an additional prior predictive text representation of the additional prior synthetic speech. In some implementations, the method further includes generating an additional prior loss based on a comparison of the additional prior ASR output to the additional prior candidate transcript.In some implementations, the method further includes selecting the leading candidate transcript for expansion rather than the additional leading candidate transcript based on (a) a leading loss of the leading candidate transcript and (b) an additional leading loss of the additional leading candidate transcript.

[0096] In some implementations, generating a loss based on a comparison of the ASR output and the candidate transcript includes generating a loss based on comparing a transcript portion of the ASR output to the candidate transcript.

[0097] In some implementations, generating a loss based on a comparison of the ASR output to the candidate transcript includes, for each word of the candidate transcript, comparing a probability portion of the ASR output of a corresponding portion of the ASR output to the word of the candidate transcript. In some implementations, the method further includes generating a loss based on the comparing.

[0098] In some implementations, the method further includes determining whether to utilize particular techniques for federated training of the ASR model and / or additional machine learning models based on the overall memory measure of the ASR model.

[0099] In some implementations, the method further includes transmitting the overall storage measure upon request by a third party.

[0100] In some implementations, the method further includes transmitting the candidate transcripts in response to a request by a third party.

[0101] In some implementations, the vocabulary corresponding to the ASR model includes a set of tokens, a set of characters, a set of alphabets, a set of words, a set of word fragments, and / or a set of phonemes.

[0102] In some implementations, a method is provided that is implemented by one or more processors, the method including receiving an automatic speech recognition ("ASR") model and a vocabulary corresponding to the ASR model. In some implementations, the method further includes generating a set of candidate transcripts based on the vocabulary of the ASR model. In some implementations, for each candidate transcript in the set of candidate transcripts, the method further includes generating synthetic speech audio data including a synthetic speech of the candidate transcript based on processing the candidate transcript using a speech synthesis model. In some implementations, the method further includes processing the synthetic speech audio data using the ASR model to generate an ASR output that reflects a predictive text representation of the synthetic speech. In some implementations, the method further includes generating a loss based on a comparison of the ASR output and the candidate transcript. In some implementations, the method further includes determining whether the ASR model unintentionally memorized one or more occurrences of a human speaking the corresponding candidate transcript in training data used to train the ASR model based on the loss.

[0103] The techniques of these and other implementations may include one or more of the following features.

[0104] In some implementations, the method further includes generating an overall memory measure in response to determining whether the ASR model unintentionally stored one or more occurrences in the set of candidate transcripts corresponding to one or more candidate transcripts. In some versions of these implementations, the method further includes identifying a subset of the candidate transcripts, where a loss corresponding to each of the candidate transcripts in the subset of candidate transcripts indicates that a probability that indicates the ASR model unintentionally stored the corresponding one or more occurrences meets a threshold. In some versions of these implementations, receiving the ASR model and a vocabulary corresponding to the ASR model includes receiving the ASR model and the vocabulary from a third party, and further includes sending the subset of the candidate transcripts to the third party.

[0105] In some implementations, the training data used to train the ASR model includes one or more subject training instances, and the method further includes determining whether the set of candidate transcripts includes the one or more subject training instances. In some implementations, the method further includes determining an overall memory measure as a function of whether the set of candidate transcripts includes the one or more subject training instances.

[0106] Additionally, some implementations include one or more processors (e.g., central processing unit (CPU)(s), graphics processing unit (GPU)(s), and / or tensor processing unit (TPU)(s)), which are capable of executing instructions stored in associated memory, the instructions being configured to perform any of the methods described herein. Some implementations also include one or more transitory or non-transitory computer-readable storage media that store computer instructions executable by the one or more processors to perform any of the methods described herein.

Claims

1. 1. A method implemented by one or more processors, comprising: generating candidate transcripts using a vocabulary corresponding to an automatic speech recognition ("ASR") model; generating synthetic speech audio data comprising a synthetic speech of the candidate transcript based on processing the candidate transcript with a speech synthesis model; processing the synthetic speech audio data using the ASR model to generate an ASR output that reflects a predictive text representation of the synthetic speech; generating a loss based on a comparison of the ASR output and the candidate transcript; determining whether the ASR model unintentionally memorized one or more occurrences of human speaking of the corresponding candidate transcript in training data used to train the ASR model based on the loss; generating an overall memory measure in response to the determination whether the ASR model unintentionally memorized the one or more occurrences; and A method comprising:

2. Generating the candidate transcripts using the vocabulary corresponding to the ASR model includes: Prior to generating the candidate transcripts, generating a leading candidate transcript using the vocabulary corresponding to the ASR model, the leading candidate transcript including beginning portions of words of the candidate transcript but lacking one or more ending words of the candidate transcript; generating prior synthetic speech audio data comprising prior synthetic speech of the prior candidate transcript based on processing the prior candidate transcript with the speech synthesis model; processing the prior synthetic speech audio data using the ASR model to generate a prior ASR output that reflects a prior predictive text representation of the synthetic speech; generating a prior loss based on a comparison of the prior ASR output and the prior candidate transcript; selecting the leading candidate transcripts for augmentation based on the leading loss; The method of claim 1 , comprising:

3. Prior to generating the candidate transcripts, generating an additional leading candidate transcript using the vocabulary corresponding to the ASR model, the additional leading candidate transcript including beginning portions of words of the leading candidate transcript but lacking one or more ending words of the leading candidate transcript; generating additional prior synthetic speech audio data including synthetic speech of the additional prior candidate transcript based on processing the additional prior candidate transcript with the speech synthesis model; processing the additional prior synthetic speech audio data using the ASR model to generate additional prior ASR outputs reflecting additional prior predictive text representations of the additional prior synthetic speech; generating additional prior losses based on a comparison of the additional prior ASR outputs to the additional prior candidate transcripts; selecting the prior candidate transcript for extension over the additional prior candidate transcript based on (a) the prior loss of the prior candidate transcript and (b) the additional prior loss of the additional prior candidate transcript; The method of claim 2 further comprising:

4. generating the loss based on a comparison of the ASR output and the candidate transcript, The method of claim 1 , comprising generating the loss based on comparing a transcript portion of the ASR output to the candidate transcript.

5. generating the loss based on a comparison of the ASR output and the candidate transcript, For each word in the candidate transcript, comparing the likely portions of the ASR output with the words of the candidate transcript; generating said loss based on said comparing; The method of claim 1 , comprising:

6. 10. The method of claim 1 , further comprising determining whether to utilize a particular technique for federated training of the ASR model and / or additional machine learning models based on the overall memory measure of the ASR models.

7. The method of claim 1 , further comprising transmitting the overall storage measure in response to a request by a third party.

8. The method of claim 1 , further comprising transmitting the candidate transcript in response to a request by a third party.

9. The method of claim 1 , wherein the vocabulary corresponding to the ASR model comprises a set of tokens, a set of characters, a set of alphabets, a set of words, a set of word fragments, and / or a set of phonemes.

10. 1. A method implemented by one or more processors, comprising: receiving an automatic speech recognition ("ASR") model and a vocabulary corresponding to the ASR model; generating a set of candidate transcripts based on the vocabulary of the ASR model; For each candidate transcript in the set of candidate transcripts, generating synthetic speech audio data comprising a synthetic speech of the candidate transcript based on processing the candidate transcript with a speech synthesis model; processing the synthetic speech audio data using the ASR model to generate an ASR output that reflects a predictive text representation of the synthetic speech; generating a loss based on a comparison of the ASR output and the candidate transcript; determining whether the ASR model unintentionally memorized one or more occurrences of human speaking of the corresponding candidate transcript in training data used to train the ASR model based on the loss; A method comprising:

11. 11. The method of claim 10, further comprising generating an overall memory measure in response to the determination whether the ASR model unintentionally memorized the one or more occurrences corresponding to one or more candidate transcripts in the set of candidate transcripts.

12. 12. The method of claim 11, further comprising identifying a subset of candidate transcripts, wherein the loss corresponding to each of the candidate transcripts in the subset of candidate transcripts indicates that a probability that indicates the ASR model unintentionally stored the corresponding one or more occurrences meets a threshold.

13. 13. The method of claim 12, wherein receiving the ASR model and the vocabulary corresponding to the ASR model comprises receiving the ASR model and the vocabulary from a third party, and further comprising transmitting the subset of the candidate transcripts to the third party.

14. the training data used to train the ASR model includes one or more subject training instances; determining whether the set of candidate transcripts includes the one or more target training instances; determining an overall memory measure according to whether the set of candidate transcripts includes the one or more target training instances; The method of claim 10 further comprising:

15. A computer program comprising instructions which, when executed by one or more processors of a computing system, cause said computing system to perform the method according to any of claims 1 to 14.

16. A client device, comprising one or more processors executing instructions stored in a memory of said client device, to perform a method according to any one of claims 1 to 14.

17. A computer readable storage medium storing instructions executable by one or more processors of a computing system to perform the method of any one of claims 1 to 14.

Images