Detecting Unintentional Memories in Language Model Fusion ASR Systems

Abstract

The method (400) includes inserting a set of canary text samples (134) into a corpus of training text samples (130) and training an external language model (160) with the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample, the method also includes generating a corresponding synthetic voice utterance (142) and generating an initial transcription (232) of the corresponding synthetic voice utterance. The method also includes rescoring the generated initial transcription for each corresponding synthetic voice utterance using the external language model. The method also includes determining a word error rate (WER) (152) of the external language model based on the rescored initial transcription and the canary text sample, and detecting memory of the canary text sample by the external language model based on the WER of the external language model.

Description

[Technical field]

[0001] The present disclosure relates to detecting unintentional memories in language model fusion ASR systems. [Background technology]

[0002] Automatic speech recognition (ASR), the process of taking audio input and transcribing the audio input into text, has become a highly important technology used in mobile and other devices. In general, automatic speech recognition attempts to provide an accurate transcription of what a person has spoken by taking an audio input (e.g., a voice utterance) and transcribing the audio input into text. Modern ASR models continue to improve in both accuracy (e.g., low word error rate (WER)) and latency (e.g., the delay between the client's speech and the transcription) based on ongoing developments in deep neural networks. In some instances, modern ASR models leverage language models (LMs) to disambiguate acoustically similar transcription hypotheses generated by the ASR model. However, one drawback of using LMs is the unintentional storage of potentially sensitive textual data during training of the LM. As a result, storing sensitive textual data creates privacy vulnerabilities in the LM, which may allow hackers to gain access to the sensitive textual data used to train the LM. Summary of the Invention

[0003] One aspect of the disclosure provides a computer-implemented method that, when executed on data processing hardware, causes the data processing hardware to perform operations for detecting unintentional memorization in a language model automatic speech recognition (ASR) system. The operations include inserting a set of canary text samples into a corpus of training text samples and training an external language model with the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample in the set of canary text samples, the operations include generating a corresponding synthetic voice utterance using a text-to-speech (TTS) system and generating an initial transcription of the corresponding synthetic voice utterance using a trained ASR model configured to receive the corresponding synthetic voice utterance as input. The operations also include rescoring the generated initial transcription for each corresponding synthetic voice utterance using the external language model trained on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. The operations also include determining a word error rate (WER) of the external language model based on the rescored initial transcription and the canary text sample, and detecting memory of the canary text sample by the external language model based on the WER of the external language model.

[0004] Implementations of the present disclosure may include one or more of the following optional features: In some implementations, a lower WER of the external language model corresponds to increased storage of canary text samples by the external language model. In some examples, the operations further include inserting the set of foreign text samples into a second corpus of training text samples; training a second external language model with the second corpus of training text samples and the set of foreign text samples inserted into the second corpus of training text samples; for each canary text sample in the set of canary text samples, receiving an initial transcription generated by the trained ASR model for a corresponding synthetic speech utterance; rescoring the initial transcription generated for each corresponding synthetic speech utterance using the second language model trained on the second corpus of training text samples and the set of foreign text samples inserted into the second corpus of training text samples; determining a second WER of the second external language model based on the initial test transcription and the canary text sample rescored by the second external language model; and detecting memory of the canary text sample by the external language model by comparing the WER of the external language model to the second WER of the second external language model.

[0005] The operations may further include mitigating the detected memory of the canary text samples by the external language model by applying per-sample gradient clipping by clipping gradients from a predetermined number of the canary text samples when training the external language model. In some implementations, the operations further include, for each canary text sample in the set of canary text samples, adding noise to a suffix portion of the corresponding synthetic voice utterance without adding any noise to a prefix portion of the corresponding synthetic voice utterance, and determining, using a classifier, that the corresponding canary text sample was used to train the external language model based on a rescored initial transcription generated for the corresponding synthetic voice utterance that matches the corresponding canary text sample. Each canary text sample in the set of canary text samples may include a fixed-length sequence of random alphanumeric characters, each separated by a space.

[0006] In some examples, inserting the set of canary text samples into the corpus of training text samples includes inserting each canary text sample of a first portion of the canary text samples in the set of canary text samples only once into the corpus of training text samples, and inserting each canary text sample of a second portion of the canary text samples in the set of canary text samples two or more times into the corpus of training text samples. In these examples, the second portion of the canary text samples includes a canary text sample that is different from the first portion of the canary text samples. The external language model may include an external neural language model, where the external neural language model may include a stack of transformer layers or conformer layers. In some implementations, the operations further include integrating the trained external language model with the trained ASR model, where the trained external language model is configured to rescore a probability distribution over the possible speech recognition hypotheses predicted by the trained ASR model.

[0007] In some examples, the trained ASR model includes a first encoder configured to receive as an input a sequence of acoustic frames and generate at each of a plurality of output steps a first high-order feature representation for a corresponding acoustic frame in the sequence of acoustic frames, a second encoder configured to receive as an input the first high-order feature representation generated by the first encoder at each of the plurality of output steps and generate at each of the plurality of output steps a second high-order feature representation for the corresponding first high-order feature frame, and a decoder configured to receive as an input the second high-order feature representation generated by the second encoder at each of the plurality of output steps and generate at each of a plurality of time steps a first probability distribution over possible speech recognition hypotheses, where the decoder or another decoder may be further configured to receive as an input the first high-order feature representation generated by the first encoder at each of the plurality of output steps and generate at each of the plurality of output steps a second probability distribution over possible speech recognition hypotheses.

[0008] In some implementations, the decoder includes a prediction network configured to receive as input, at each of the plurality of output steps, the sequence of N prior non-blank symbols output by the final softmax layer, generate a respective embedding for each non-blank symbol of the sequence of N prior non-blank symbols, and generate an average embedding by averaging the respective embeddings, and a collaborative network configured to receive as input, the average embedding generated by the prediction network at each of the plurality of output steps, and one of the first higher-order feature representation generated by the first encoder at each of the plurality of output steps when the ASR model is operating in streaming mode, or the second higher-order feature representation generated by the second encoder at each of the plurality of output steps when the ASR model is operating in non-streaming mode. In these implementations, the decoder is further configured to generate, at each of the plurality of output steps, one of the second probability distribution over possible speech recognition hypotheses when the ASR model is operating in streaming mode, or the first probability distribution over possible speech recognition hypotheses when the ASR model is operating in non-streaming mode. The first encoder may include a causal encoder having an initial stack of conformer layers, and the second encoder includes a non-causal encoder having a final stack of conformer layers that overlaps the initial stack of conformer layers. In some examples, the first encoder and the second encoder of the ASR model are trained to use hybrid autoregressive transducer factorization to facilitate integration of an external language model trained on text-only data, including a corpus of training text samples and a set of canary text samples inserted into the corpus of training text samples.

[0009] Another aspect of the disclosure provides a system including data processing hardware and memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include inserting a set of canary text samples into a corpus of training text samples and training an external language model with the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample in the set of canary text samples, the operations include generating a corresponding synthetic voice utterance using a text-to-speech (TTS) system and generating an initial transcription of the corresponding synthetic voice utterance using a trained ASR model configured to receive as input the corresponding synthetic voice utterance. The operations also include rescoring the generated initial transcription for each corresponding synthetic voice utterance using the external language model trained on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. The operations also include determining a word error rate (WER) of the external language model based on the rescored initial transcription and the canary text sample, and detecting memory of the canary text sample by the external language model based on the WER of the external language model.

[0010] Implementations of the present disclosure may include one or more of the following optional features: In some implementations, a lower WER of the external language model corresponds to increased storage of canary text samples by the external language model. In some examples, the operations further include inserting the set of foreign text samples into a second corpus of training text samples; training a second external language model with the second corpus of training text samples and the set of foreign text samples inserted into the second corpus of training text samples; for each canary text sample in the set of canary text samples, receiving an initial transcription generated by the trained ASR model for a corresponding synthetic speech utterance; rescoring the initial transcription generated for each corresponding synthetic speech utterance using the second language model trained on the second corpus of training text samples and the set of foreign text samples inserted into the second corpus of training text samples; determining a second WER of the second external language model based on the initial test transcription and the canary text sample rescored by the second external language model; and detecting memory of the canary text sample by the external language model by comparing the WER of the external language model to the second WER of the second external language model.

[0011] The operations may further include mitigating the detected memory of the canary text samples by the external language model by applying per-sample gradient clipping by clipping gradients from a predetermined number of the canary text samples when training the external language model. In some implementations, the operations further include, for each canary text sample in the set of canary text samples, adding noise to a suffix portion of the corresponding synthetic voice utterance without adding any noise to a prefix portion of the corresponding synthetic voice utterance, and determining, using a classifier, that the corresponding canary text sample was used to train the external language model based on a rescored initial transcription generated for the corresponding synthetic voice utterance that matches the corresponding canary text sample. Each canary text sample in the set of canary text samples may include a fixed-length sequence of random alphanumeric characters, each separated by a space.

[0012] In some examples, inserting the set of canary text samples into the corpus of training text samples includes inserting each canary text sample of a first portion of the canary text samples in the set of canary text samples only once into the corpus of training text samples, and inserting each canary text sample of a second portion of the canary text samples in the set of canary text samples two or more times into the corpus of training text samples. In these examples, the second portion of the canary text samples includes a canary text sample that is different from the first portion of the canary text samples. The external language model may include an external neural language model, where the external neural language model may include a stack of transformer layers or conformer layers. In some implementations, the operations further include integrating the trained external language model with the trained ASR model, where the trained external language model is configured to rescore a probability distribution over the possible speech recognition hypotheses predicted by the trained ASR model.

[0013] In some examples, the trained ASR model includes a first encoder configured to receive as an input a sequence of acoustic frames and generate at each of a plurality of output steps a first high-order feature representation for a corresponding acoustic frame in the sequence of acoustic frames, a second encoder configured to receive as an input the first high-order feature representation generated by the first encoder at each of the plurality of output steps and generate at each of the plurality of output steps a second high-order feature representation for the corresponding first high-order feature frame, and a decoder configured to receive as an input the second high-order feature representation generated by the second encoder at each of the plurality of output steps and generate at each of a plurality of time steps a first probability distribution over possible speech recognition hypotheses, where the decoder or another decoder may be further configured to receive as an input the first high-order feature representation generated by the first encoder at each of the plurality of output steps and generate at each of the plurality of output steps a second probability distribution over possible speech recognition hypotheses.

[0014] In some implementations, the decoder includes a prediction network configured to receive as input, at each of the plurality of output steps, the sequence of N prior non-blank symbols output by the final softmax layer, generate a respective embedding for each non-blank symbol of the sequence of N prior non-blank symbols, and generate an average embedding by averaging the respective embeddings, and a collaborative network configured to receive as input, the average embedding generated by the prediction network at each of the plurality of output steps, and one of the first higher-order feature representation generated by the first encoder at each of the plurality of output steps when the ASR model is operating in streaming mode, or the second higher-order feature representation generated by the second encoder at each of the plurality of output steps when the ASR model is operating in non-streaming mode. In these implementations, the decoder is further configured to generate, at each of the plurality of output steps, one of the second probability distribution over possible speech recognition hypotheses when the ASR model is operating in streaming mode, or the first probability distribution over possible speech recognition hypotheses when the ASR model is operating in non-streaming mode. The first encoder may include a causal encoder having an initial stack of conformer layers, and the second encoder includes a non-causal encoder having a final stack of conformer layers that overlaps the initial stack of conformer layers. In some examples, the first encoder and the second encoder of the ASR model are trained to use hybrid autoregressive transducer factorization to facilitate integration of an external language model trained on text-only data, including a corpus of training text samples and a set of canary text samples inserted into the corpus of training text samples.

[0015] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will become apparent from the description and drawings, and from the claims. [Brief description of the drawings]

[0016] [Figure 1] 1 is a schematic diagram of an exemplary speech environment using a cascade encoder and language model architecture for automatic speech recognition; [Figure 2A] FIG. 1 is a schematic diagram of an example process for detecting unintentional memorization of a canary text sample. [Figure 2B] FIG. 1 is a schematic diagram of an example process for detecting unintentional memorization of a canary text sample. [Diagram 3] 1A and 1B are schematic diagrams of an exemplary training process for training an external language model. [Figure 4] 1 is a flow chart of an exemplary arrangement of operations of a method for detecting unintentional memory in a language model fusion automatic speech recognition system. [Diagram 5] FIG. 1 is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Like reference symbols in the various drawings indicate like elements.

[0018] Traditionally, an end-to-end (E2E) automatic speech recognition (ASR) model includes an encoder and a decoder as its main components. For example, the encoder generates encoded input audio data, and the decoder generates candidate transcriptions corresponding to the input audio data. Many modern ASR models further leverage an external language model (LM) configured to use linguistic information to disambiguate acoustically similar candidate transcriptions generated by the ASR model. For example, the LM may determine that "how to get to the dog park" is much more likely to be an accurate transcription than "how to get to the dog bark" based on linguistic information that "how to get to the dog park" is more likely to be spoken by a user. In particular, by leveraging the LM, the ASR model may significantly improve its performance (e.g., word error rate (WER)) for recognizing rare or out-of-vocabulary (OOV) words spoken by a user.

[0019] The LM is trained on millions of training text samples, including general text samples and user-specific text samples such as contact lists, typing patterns, and other user-specific information. Thus, by training the LM on user-specific text samples for a particular user, the LM generates transcriptions that are specifically tailored to the user. For example, the LM may improve an incorrect transcription of "Call Hailee" generated by an ASR model to a correct transcription of "Call Haleigh" based on the LM training on contact names associated with the user.

[0020] However, a major drawback of training a LM with user-specific text samples is that the LM may unintentionally store potentially sensitive information that is unique to the user. For example, training a LM with a user's credit card number allows the LM to generate accurate transcriptions when the user speaks the credit card number, but the LM may also unintentionally store the credit card number. As used herein, the user-specific text samples used to train the LM and unintentionally stored by the LM are referred to as canary text samples. The canary text samples may contain sensitive information associated with the user. The storage of canary text samples may be exploited by a hacker who is able to identify these stored canary text samples, thereby exploiting potentially sensitive information. However, there are several obstacles in determining whether the LM has unintentionally stored canary text samples. Namely, the ASR model is configured to receive audio data (rather than text data) as input and generate a single transcript per utterance as output. Thus, when direct access of the LM is not available to obtain a list of N best transcriptions, traditional attacks against the LM cannot determine any stored text data.

[0021] Accordingly, embodiments herein are directed to a method and system for detecting canary text samples unintentionally stored by an external LM. More specifically, the method includes inserting a set of canary text samples into a corpus of training text samples and training an external LM with the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample, a text-to-speech (TTS) system generates a corresponding synthetic voice utterance and a trained ASR model generates an initial transcription of the corresponding synthetic voice utterance. The trained external LM rescores the generated initial transcription for each corresponding synthetic voice utterance. Using the rescored initial transcription and the canary text sample, the method determines a word error rate (WER) of the external LM and detects the storage of the canary text sample by the external LM. As will become apparent, detecting the storage of the canary text sample may include using another external LM and / or using a classifier that is not trained on the canary text sample. Detecting the stored canary text sample merely identifies a potential security vulnerability of the LM. To that end, the methods and systems may be further directed to applying sample-by-sample gradient clipping by clipping gradients from a specified number of canary text samples to alleviate the external LM from storing canary text samples during training.

[0022] 1 is an example of an audio environment 100. In the audio environment 100, a way in which a user 104 interacts with a computing device, such as a user device 10, may be through voice input. The user device 10 (also generally referred to as device 10) is configured to capture sounds (e.g., streaming audio data) from one or more users 104 in the audio environment 100. Here, streaming audio data may refer to voice utterances 106 by the user 104 that function as audible queries, commands to the user device 10, or audible communications captured by the user device 10. A voice-enabled system of the user device 10 may act on the queries or commands by responding to the queries and / or causing the commands to be executed / fulfilled by one or more downstream applications.

[0023] The user device 10 may correspond to any computing device that is associated with a user 104 and can receive audio data. Some examples of the user device 10 include, but are not limited to, a mobile device (e.g., a smart watch), a smart appliance, an Internet of Things (IoT) device, an in-vehicle infotainment system, a smart display, a smart speaker, and the like. The user device 10 includes data processing hardware 12 and memory hardware 14 that communicates with the data processing hardware 12 and stores instructions that, when executed by the data processing hardware 12, cause the data processing hardware 12 to perform one or more operations. The user device 10 further includes an audio system 16 having audio capture devices (e.g., microphones) 16, 16a for capturing and converting sound utterances 106 in the audio environment 100 into electrical signals, and audio output devices (e.g., speakers) 16, 16b for communicating audible audio signals (e.g., as output data from the user device 10). In the illustrated example, the user device 10 implements a single audio capture device 16a, however, the user device 10 may implement an array of audio capture devices 16a without departing from the scope of this disclosure, such that one or more capture devices 16a of the array may be in communication with the audio system 16 while not being physically present on the user device 10.

[0024] In the speech environment 100, an automatic speech recognition (ASR) system 118 implements an ASR model (e.g., a trained ASR model) 200 and a first external language model (LM) 160, each of which resides on the user device 10 of the user 104 and / or on a remote computing device 60 (e.g., one or more remote servers of a distributed system executing in a cloud computing environment) in communication with the user device 10 via a network 40. In some examples, the ASR model 200 includes a recurrent neural network-transducer (RNN-T) model architecture. The user device 10 and / or the remote computing device 60 also include an audio subsystem 108 configured to receive an utterance 106 spoken by the user 104 and captured by the audio capture device 16a, and convert the utterance 106 into a corresponding digital format associated with an input acoustic frame 110 that can be processed by the ASR system 118. In the example, a user speaks a respective utterance 106, and the audio subsystem 108 converts the utterance 106 into corresponding audio data (e.g., a sequence of acoustic frames) 110 for input to the ASR system 118. The ASR model 200 then receives as input the sequence of acoustic frames 110 corresponding to the utterance 106, and at each output step, generates / predicts a corresponding transcription 120 (e.g., a speech recognition result / hypothesis) of the utterance 106 as the ASR model 200 receives (e.g., processes) each acoustic frame 110 in the acoustic frames 110.

[0025] In the illustrated example, the ASR model 200 may perform streaming speech recognition to generate initial speech recognition results (e.g., candidate hypotheses) 120, 120a and generate final speech recognition results (e.g., final hypotheses) 120, 120b by improving the initial speech recognition result 120a. The initial and final speech recognition results 120a, 120b may correspond to either partial speech recognition results or the entire speech recognition result. In other words, the initial and final speech recognition results 120a, 120b may correspond to either a portion of the utterance 106 or an entire portion of the utterance 106. For example, a partial speech recognition result may correspond to a portion of a speech utterance or a portion of a spoken term. However, as will become clear, the ASR model 200 performs additional processing on the final speech recognition result 120b, which may cause the final speech recognition result 120b to lag behind the initial speech recognition result 120a. 2, the first external LM 160 may rescore the transcription 120 generated by the ASR model 200 to improve the transcription 120 in either streaming or non-streaming mode. The first external LM 160 may be external to the ASR model 200 or may be integrated with the ASR model 200.

[0026] The user device 10 and / or the remote computing device 60 also execute a user interface generator 107 configured to present a representation of the transcription 120 of the utterance 106 to the user 104 of the user device 10. As described in more detail below, the user interface generator 107 may display initial speech recognition results 120a in a streaming manner during time 1 and later display final speech recognition results 120b in a streaming manner during time 2. Notably, the ASR model 200 outputs final speech recognition results 120b in a streaming manner, even if the final speech recognition results 120b improve upon the initial speech recognition results 120a. In some configurations, the transcription 120 output from the ASR system 118 is processed (e.g., by a natural language understanding (NLU) module executing on the user device 10 or the remote computing device 60) to execute the user command / query specified by the utterance 106. Additionally or alternatively, a text-to-speech system (not shown) (e.g., running on any combination of the user device 10 or the remote computing device 60) may convert the transcription 120 into synthetic speech for audible output by the user device 10 and / or other devices.

[0027] In the illustrated example, a user 104 interacts with a program or application 50 (e.g., a digital assistant application 50) of the user device 10 that uses an ASR system 118. For example, FIG. 1 shows a user 104 communicating with the digital assistant application 50 and the digital assistant application 50 displaying a digital assistant interface 18 on the screen of the user device 10 to depict a conversation between the user 104 and the digital assistant application 50. In this example, the user 104 asks the digital assistant application 50, "What time is the concert tonight?" This question from the user 104 is a speech utterance 106 that is captured by the audio capture device 16a and processed by the audio system 16 of the user device 10. In this example, the audio system 16 receives the speech utterance 106 and converts the speech utterance 106 into a sequence of acoustic frames 110 for input to the ASR system 118.

[0028] Continuing with the example, as the ASR model 200 receives a sequence of acoustic frames 110 corresponding to an utterance 106 spoken by the user 104, it encodes the sequence of acoustic frames 110 and then decodes the encoded sequence of acoustic frames 110 into an initial speech recognition result 120a. During time 1, the user interface generator 107 presents to the user 104 of the user device 10, via the digital assistant interface 18, a representation of the initial speech recognition result 120a of the utterance 106 in a streaming manner such that words, word pieces, and / or individual characters are displayed on the screen as they are spoken. In some examples, the first look-ahead audio context is equal to zero.

[0029] During time 2, the user interface generator 107 presents to the user 104 of the user device 10, via the digital assistance interface 18, a representation of the final speech recognition result 120b of the utterance 106 in a streaming manner such that words, word pieces, and / or individual characters are displayed on the screen as soon as they are generated by the ASR model 200. In some implementations, the user interface generator 107 replaces the representation of the initial speech recognition result 120a presented at time 1 with the representation of the final speech recognition result 120b presented at time 2, where time 1 and time 2 may include timestamps corresponding to when the user interface generator 107 presents the respective speech recognition result 120. In this example, the timestamp for time 1 indicates that the user interface generator 107 presents the initial speech recognition result 120a at an earlier time than the final speech recognition result 120b. For example, the final speech recognition result 120b, which is ultimately displayed as the transcription 120, may correct any terms that may have been misrecognized in the initial speech recognition result 120a, since the final speech recognition result 120b is presumed to be more accurate than the initial speech recognition result 120a. In this example, the streaming initial speech recognition result 120a output by the ASR model 200 is displayed on the screen of the user device 10 at time 1 and is associated with low latency, providing the user 104 with a sense of responsiveness that the user's query is being processed, while the final speech recognition result 120b output by the ASR model 200 and displayed on the screen at time 2 utilizes additional speech recognition and / or language models to improve the quality of the speech recognition in terms of accuracy, but at the expense of increased latency. However, since the initial speech recognition result 120a is displayed as the user speaks the utterance 106, the higher latency associated with generating and ultimately displaying the final speech recognition result 120b is not readily noticeable to the user 104.

[0030] In the example shown in FIG. 1, the digital assistant application 50 may use natural language processing to respond to a question posed by the user 104. Natural language processing generally refers to the process of interpreting written language (e.g., initial speech recognition results 120a and / or final speech recognition results 120b) and determining whether the written language prompts any action. In this example, the digital assistant application 50 uses natural language processing to recognize that a question from the user 104 is about the user's schedule and more specifically a concert that is on the user's schedule. By recognizing these details using natural language processing, the automated assistant returns a response 19 to the user's query, which states, "The venue's doors open at 6:30 p.m. and the concert begins at 8 p.m." In some configurations, the natural language processing occurs at a remote server 60 that communicates with the data processing hardware 12 of the user device 10.

[0031] 2A and 2B show schematic diagrams of a first exemplary process 201 (FIG. 2A) and a second exemplary process 202 (FIG. 2B) for detecting unintentional storage of a canary text sample 134 by a first external LM 160. The processes 201, 202 execute on a remote server 60 (FIG. 1) and include a trained ASR model 200 having a cascade encoder 204 and a decoder 206. As will become apparent hereinafter, the ASR model 200 may operate in streaming mode, non-streaming mode, and / or in both streaming and non-streaming modes in parallel. Additionally, the processes 201, 202 may use one or more components external to the ASR model 200, including, but not limited to, a text-to-speech (TTS) system 140 and the first external LM 160.

[0032] 3A and 3B, in some implementations, a training process 300 is configured to train a first external LM 160 and a second external LM 170. As will become apparent, the second external LM 170 may be used as a baseline LM to detect whether the first external LM 160 has unintentionally memorized any of the canary text samples 134. In particular, FIG. 3A shows a schematic diagram of a first training process 300, 300a configured to train a first external LM (e.g., the first LM) 160 using a first corpus of training text samples 130. Here, the first corpus of training text samples 130 includes a set of training text utterances 132, 132a-n and a set of canary text samples 134, 134a-n. The set of training text utterances 132 may include general text samples that are not user-specific or do not include sensitive information about the user. For example, the training text utterances 132 may include "Hey Google," "What appointments do you have this week?", and "How's the weather today?". The first training process 300a also obtains a set of canary text samples 134, where each canary text sample 134 includes a fixed-length sequence (or a dynamic-length sequence) of output labels. Each output label may include alphanumeric characters, graphemes, phonemes, N-grams, word pieces, and / or words. In some cases, each output label is separated by spaces. Furthermore, each canary text sample 134 may represent user-specific information or sensitive information associated with a user. For example, a canary text sample 134 may correspond to a password "oegdbu" or a credit card number "1111 2222 3333 4444." However, in some examples, to avoid revealing any actual sensitive information associated with the user, the canary text sample 134 may include random alphanumeric sequences having a format similar to sensitive information (e.g., fictitious sensitive information) such as passwords, serial numbers, credit card numbers, and / or other sensitive information, without actually containing any sensitive information associated with the user.

[0033] The first training process 300a inserts each canary text sample 134 into the first corpus of training text samples 130. Thus, each training text sample 130 in the first corpus of training text samples 130 may correspond to a respective one of the training text utterances 132 or a respective one of the canary text samples 134. Furthermore, the first corpus of training text samples 130 may include any number of training text samples 130. In some implementations, the first training process 300a may insert each canary text sample 134 of a first portion of the canary text samples 134 in the set of canary text samples 134 into the first corpus of training text samples 130 only once (e.g., once) and insert each canary text sample 134 of a second portion of the canary text samples 134 in the set of canary text samples 134 into the first corpus of training text samples 130 more than once. Here, the second portion of the canary text samples 134 includes different canary text samples 134 than the first portion of the canary text samples 134. Thus, each canary text sample 134 inserted into the first corpus of training text samples 130 may have a different number of occurrences in the first corpus of training text samples 130.

[0034] The first LM 160 may be configured to receive as input each training text sample 130 (e.g., training text utterance 132 or canary text sample 134) in the first corpus of training text samples 130 and generate as output a training transcription 162 of the corresponding training text sample 130 in the first corpus of training text samples 130. In some examples, the first LM 160 receives a subset of output labels for each training text sample 130 and generates the training transcription 162 to predict a remaining subset of the output levels of the text samples. For example, if the training text sample 130 is "1 2 3 4", the first LM 160 may receive the output label "1 2 3" and predict that the next output label is "4". In other examples, the first LM 160 may receive a hidden sequence of output labels for each training text sample 130, such that one or more of the output labels are masked with a masking value (e.g., a null value).

number

number

number

[0035] The first training loss module 310 receives the training transcriptions 162 generated by the first LM 160 for each training text sample 130 and determines a first training loss 312 by comparing the training transcriptions 162 (e.g., predicted transcriptions) with the corresponding training text samples 130 (e.g., ground truth labels). The first training process 300a then provides the first training loss 312 as feedback to the first LM 160 and updates parameters of the first LM 160 based on the first training loss 312. Thus, the first training process 300a trains the first LM 160 to generate accurate transcriptions using a set of training text utterances 132 and a set of canary text samples 134. Notably, during the first training process 300a, the first LM 160 may unintentionally store one or more canary text samples 134. As will become apparent hereinafter, any of the canary text samples 134 stored by the first LM 160 may be detected by processes 201, 202 (FIG. 2).

[0036] Now referring to FIG. 3B, the second training process 300b is configured to train a second external LM (e.g., second LM) 170 using a second corpus of training text samples 131. Here, the second corpus of training text samples 131 includes a set of training text utterances 132 and a set of foreign text samples 136, 136a-n. That is, in contrast to the first training process 300a (FIG. 3A), the second training process 300b takes a set of foreign text samples 136 and inserts them into the second corpus of training text samples 131 instead of the canary text sample 134. Here, each foreign text sample 136 may include output labels of the same fixed length sequence (or dynamic length sequence) as the canary text sample 134. That is, each output label of the foreign text sample 136 may include alphanumeric characters, phonemes, graphemes, N-grams, word pieces, and / or words, optionally separated by spaces. Further, each foreign text sample 136 may represent user-specific or sensitive information about the user. However, as will become apparent, each foreign text sample 136 in the set of foreign text samples 136 is different from each canary text sample 134 in the set of canary text samples 134. Thus, a foreign text sample 136 may also correspond to a password "oegdbu" or a credit card number "1111 2222 3333 4444" only if these text samples are not already included in the set of canary text samples 134.

[0037] Similarly, the second training process 300b inserts each foreign text sample 136 into the second corpus of training text samples 131. Thus, each training text sample 131 in the second corpus of training text samples 131 may correspond to a respective one of the training text utterances 132 or a respective one of the foreign text samples 136. The second LM 170 may be configured to receive as input each training text sample 131 (e.g., training text utterance 132 or foreign text sample 136) in the second corpus of training text samples 131 and generate as output a training transcription 172 of the corresponding training text sample 131 in the second corpus of training text samples 131. In some examples, the second LM 170 is configured to receive a subset of output labels for each training text sample 131 and generate the training transcription 172 to predict a remaining subset of output levels of the text sample. In another example, the second LM 170 receives a hidden sequence of output labels for each training text sample 130, whereby one or more of the output labels are masked with a masking value

number

[0038] The second training loss module 320 receives the training transcriptions 172 generated by the second language model 170 for each training text sample 131 and determines a second training loss 322 by comparing the training transcriptions 172 (e.g., predicted transcriptions) with the corresponding training text samples 131 (e.g., ground truth labels). The second training process 300b then provides the second training loss 322 as feedback to the second LM 170 and updates parameters of the second LM 170 based on the second training loss 322. Thus, the second training process 300b trains the second LM 170 to generate accurate transcriptions using the set of training text utterances 132 and the set of foreign text samples 136. Notably, the second LM 170 has not been trained using any of the canary text samples 134. Thus, because the second LM 170 did not use any of the canary text samples 134 during training and likely could not have memorized any of the canary text samples 134, the second LM 170 may represent a baseline LM for detecting whether the first LM 160 unintentionally memorized any of the canary text samples 134 during training. Thus, the second LM 170 is used only to detect memorization of the canary text samples and is not used for inference.

[0039] 2A and 2B, in order to detect whether the first LM 160 has stored any of the canary text samples 134, the processes 201, 202 need to receive the canary text samples 134 as input. However, the ASR model 200 is configured to receive audio data (e.g., acoustic frames 110) as input, rather than text input. To that end, the TTS system 140 is configured to generate, at each output step, for each canary text sample 134 in the set of canary text samples 134, a corresponding synthetic speech utterance 142. Each synthetic speech utterance 142 corresponds to a respective sequence of acoustic frames 110 suitable for processing by the cascade encoder 204 of the ASR model 200.

[0040] The cascaded encoder 204 refers to a model structure in which the encoding path includes two encoders 210, 220 that are cascaded such that the output of the first encoder 210 provides the input of the second encoder 220 before decoding. Here, the first encoder 210 and the second encoder 220 may be cascaded regardless of the underlying architecture of each encoder. The encoders 210, 220 may each include a stack of multi-head (e.g., 8-head) attention layers (e.g., 17 layers). In some examples, the stack of multi-head attention layers of the encoders 210, 220 includes a stack of 512-dimensional conformer layers. In other examples, a transformer layer may be used instead of a conformer layer.

[0041] The first encoder 210 may be a causal encoder including an initial stack of conformer layers (e.g., 15 conformer layers) with a multi-head (e.g., 8-head) attention mechanism, each of which is used as a self-attention layer. Here, each conformer layer of the first encoder 210 may use a causal convolutional layer and a left context attention layer to restrict the first encoder 210 from using any future input (e.g., right context equal to zero). Meanwhile, the second encoder 220 may be a non-causal encoder including a final stack of conformer layers (e.g., two conformer layers) that overlap the initial stack of conformer layers. Here, each conformer layer of the second encoder 220 may use a non-causal convolutional layer and a right context attention layer, thereby allowing the second encoder 220 to use (e.g., address) future inputs. That is, the second encoder 220 may receive and process an additional right context (e.g., 2.88 seconds of a future audio frame) to generate the encoder output. Accordingly, the first and second encoders 210, 220 may be referred to interchangeably herein as the "causal encoder 210" and the "non-causal encoder 220," respectively.

[0042] 2A and 2B, the first encoder 210 receives a sequence of d-dimensional feature vectors (e.g., a sequence of acoustic frames 110) x=(x1, x2, ..., x T )(in the formula,

number

[0043] The decoder 206 may include a recurrent neural network-transducer (RNN-T) architecture having a collaboration network 230 and a prediction network 240. The prediction network 240 may include a V2 embedding lookup table. The prediction network 240 is configured to receive as input the sequence of N previous non-blank symbols output by the final softmax layer and generate a respective embedding for each non-blank symbol in the sequence of N previous non-blank symbols. That is, although not shown, the ASR model 200 may include a softmax layer that receives the output of the decoder 206. In some implementations, the softmax layer is separate from the decoder 206 and is configured to receive the output y from the decoder 206. r The output of the softmax layer is then used in a beam search process to select the orthographic elements. In some examples, the softmax layer is integrated with the decoder 206, so that the output y rrepresents the output of the softmax layer. The prediction network 240 is further configured to generate an average embedding 242 by averaging each of the embeddings generated for each of the non-blank symbols.

[0044] In the non-streaming mode, the decoder 206 uses a joint network 230 to combine the first and second high-level feature representations 212 and 222 output by the cascade encoder 204, as well as the average embedding 242 generated by a prediction network 240 to generate an initial transcription (e.g., decoder output) 232. The decoder output 232 is a sequence of N previous non-blank symbols {y i-1 ...,y i-N} and given the input x, the current subword unit y i Probability distribution over

number

[0045] The decoder 206 is configured to generate an initial transcription 232 that includes a probability distribution over possible speech recognition hypotheses at each output step. In other words, the joint layer 230 generates a probability distribution over possible speech recognition hypotheses at each output step (e.g., time step). Thus, the initial transcription 232 may be interchangeably referred to herein as a probability distribution over possible speech recognition hypotheses 232. Here, a "possible speech recognition hypothesis" corresponds to a set of output labels / symbols (also called "phonetic units") each representing a grapheme (e.g., a symbol / character) or wordpiece in a specified natural language. For example, when the natural language is English, the set of output labels may include twenty-seven (27) symbols, e.g., one label for each of the 26 letters of the English alphabet and one label designating a space. Thus, the joint network 230 may output a set of values ​​indicating the likelihood of occurrence of each of a given set of output labels. This set of values ​​may be a vector (e.g., one-hot vector) and may indicate a probability distribution over the set of output labels. In some cases, the output labels are graphemes (e.g., individual characters, potential punctuation and other symbols, etc.), but the set of output labels is not limited thereto. For example, the set of output labels can include whole wordpieces and / or words in addition to or instead of graphemes. The output labels could also be other types of speech units, such as phonemes or subphonemes. The output distribution of the collaborative network can include posterior probability values ​​for each of the different output labels. Thus, if there are 100 different output labels representing graphemes or other symbols, the output of the collaborative network can include 100 different probability values, one for each output label. The probability distribution can then be used to select and assign scores to candidate orthographic elements (e.g., graphemes, wordpieces, and / or words) in a beam search process (e.g., with a softmax layer) to determine the initial transcription 232.

[0046] 2A and 2B, in some configurations, the ASR model 200 operates in both streaming and non-streaming modes in parallel. When operating simultaneously in both streaming and non-streaming modes, the ASR model 200 first performs streaming speech recognition on the audio data 110 using the first encoder 210 to generate a first high-level feature representation 210 for both the second encoder 220 and the decoder 206. The decoder 206 then generates a first probability distribution 232 over the possible speech recognition hypotheses, which is then rescored by the first LM 160 to generate the initial speech recognition result 120a. Optionally, in streaming mode, the decoder 206 may directly output the first probability distribution 232 over the possible speech recognition hypotheses, thereby bypassing the first LM 160. The ASR model 200 also performs non-streaming speech recognition on the audio data 110, where the second encoder 220 uses the first high-level feature representation 212 received from the first encoder 210 to generate a second high-level feature representation 222. The decoder 206 then generates a second probability distribution 232 over the possible speech recognition hypotheses, which is then rescored by the first LM 160 to generate the final speech recognition result 120b.

[0047] In another configuration, the ASR model 200 operates only in streaming mode. This may occur, for example, when the user 104 is using an application that requires the smallest possible latency, such as voice search or on-device dictation. Here, the ASR model 200 performs streaming speech recognition on the audio data 110 using only the first encoder 210 to generate a first high-level feature representation 212 for the decoder 206. The decoder 206 then generates a first probability distribution 232 over the possible speech recognition hypotheses, which is then rescored by the first LM 160 to generate the initial speech recognition result 120a. The streaming mode of the ASR model 200 quickly generates the initial speech recognition result 120a, so that any inaccuracies in the terms are generally acceptable to the user.

[0048] In yet another configuration, the ASR model 200 operates only in a non-streaming mode. A non-streaming mode may occur, for example, when the user 104 is viewing a transcription of a voicemail left on his / her phone. As mentioned above, this type of application benefits from using future context to improve performance at the expense of increased processing times. Here, the ASR model 200 first uses the first encoder 210 to generate a first high-level feature representation 212 for the second encoder 220, but the decoder 206 does not decode the first high-level feature representation 212. The ASR model 200 then performs non-streaming speech recognition on the encoded audio data 110, where the second encoder 220 uses the first high-level feature representation 212 received from the first encoder 210 to generate a second high-level feature representation 222. The decoder 206 then generates a second probability distribution 232 over the possible speech recognition hypotheses, which is then rescored by the first LM 160 to generate the final speech recognition result 120b. Because the non-streaming mode of the ASR model 200 accurately produces the final speech recognition result 120b, the latency to display the accurate transcription is generally acceptable to the user.

[0049] In some examples, the first LM 160 includes an external neural language model. Here, the external neural language model can include a stack of multi-head (e.g., 8-head) self-attention layers (e.g., 12 layers), where each layer includes a model dimension of 768 and a feedforward layer dimension of 2048. The stack of multi-head self-attention layers may include a stack of transformer layers or conformer layers. For example, the stack of conformer layers may include a one-way conformer layer that goes back a predetermined number of tokens (e.g., 31 tokens) for each output word piece model prediction. Here, the first LM 160 is trained to predict 4,096 word pieces. In some cases, the second LM 170 (FIG. 2A) includes the same structure as the first LM 160.

[0050] Integrating the ASR model 200 with an external LM typically requires shallow fusion. However, overconfidence in the cascade encoder 204 and decoder 206 can make weighting difficult and often lead to a high rate of word deletion. Here, the trained first LM 160 configured to rescore the probability distribution 232 over possible speech recognition hypotheses predicted by the trained ASR model 200 can be integrated with the trained ASR model 200 using shallow fusion. Thus, the first encoder 210 and the second encoder 220 of the ASR model 200 can be trained to facilitate the integration of the first LM 160 trained on text-only data, including the first corpus of training text samples 130 and a set of canary text samples 134 inserted into the first corpus of training set samples 130 (FIG. 3A) using hybrid autoregressive transducer (HAT) factorization. In other words, HAT factorization may be utilized during training of the ASR model 200 to facilitate integration with the first LM 160 by removing the internal loss scores of the cascaded encoder 204 and the decoder 206 and optimizing the decoder 206 according to:

number

[0051] Now, referring specifically to FIG. 2A, in some implementations, the first process 201 executes the detection loss module 150 and the second LM 170 to detect whether the first LM 160 memorized any canary text samples 134 during training (FIG. 3A) for each set of canary text samples 134 bases. That is, the first training process 201 determines whether the first LM 160 memorized one or more canary text samples 134 without specifically identifying which particular canary text samples 134 were memorized. Here, the first process 201 executes the first and second LMs 160, 170 after training has been performed. Notably, the second LM 170 is used only as a baseline LM for detecting unintentional memorization by the first LM 160, whereby the second LM 170 is not used during inference by the ASR system 118 (FIG. 1). Simply put, the second LM 170 represents a baseline LM because the second LM 170 was trained on a second corpus of training text samples 131 (FIG. 3B), which includes training samples that are similar to the canary texts, without actually including any of the canary text samples 134.

[0052] For each canary text sample 134, the first LM 160 rescores the initial transcription 232 generated by the decoder 206 for the corresponding synthetic voice utterance 142 to generate the transcription 120. Similarly, for each canary text sample 134, the second LM 170 rescores the initial transcription generated by the decoder 206 for the corresponding synthetic voice utterance 142 to generate a detected transcription 174. Thus, for each canary text sample 134, the first process 201 generates a corresponding transcription 120 and a corresponding detected transcription 174, both of which are associated with the same respective canary text sample 134.

[0053] In some examples, the detection loss module 150 is configured to receive the transcription 120 generated by the first LM and the detection transcription 174 generated by the second LM 170 for each canary text sample 134 and determine a WER for each LM 160, 170. That is, the detection loss module 150 determines a first WER 152 for the first LM 160 based on comparing the transcription 120 to the corresponding canary text sample 134 (e.g., ground truth label). Similarly, the detection loss module 150 determines a second WER 154 for the second LM 170 based on comparing the detection transcription 174 to the corresponding canary text sample 134 (e.g., ground truth label). Thus, the first and second WERs 152, 154 represent an aggregate WER for the entire set of canary text samples 134. The first and second WERs 152, 154 may include a cosine similarity between the corresponding canary text sample 134 and the transcript 120 or the detected transcript 174, respectively.

[0054] Thus, the detection loss module 150 may detect that the first LM 160 has stored a canary text sample 134 by comparing the first WER 152 of the first LM 160 and the second WER 154 of the second LM 170. In particular, the detection loss module 150 may determine a difference in the WER between the first WER 152 and the second WER 154 and determine whether the difference in the WER meets a difference threshold. In response to determining that the difference meets the difference threshold, the detection loss module 150 detects that the first LM 160 has stored one or more canary text samples 134 for each set of canary text samples 134, without specifically identifying which particular canary text samples 134 were stored. Simply put, if the first LM 160 produces transcriptions 120 of a set of canary text samples that are, on average across the set of canary text samples 134, significantly more accurate than the detected transcriptions 174 produced by the second LM 170, the detection loss module 150 determines that one or more of the canary text samples 134 were stored by the first LM 160 during training (FIG. 3A). Advantageously, the first process 201 does not need to access the N-best list or confidence values ​​produced by the first LM 160, which are generally unavailable to users without direct access to the first LM 160, by simply comparing the WERs of the first and second LMs 160, 170. In some instances, the detection loss module 150 detects that the first LM 160 has stored the canary text sample 134 by determining whether the first WER 152 meets a WER threshold, without ever even considering the second WER 154. Here, a lower first WER 152 of the first LM 160 corresponds to increased storage of the canary text sample 134 by the first LM 160.

[0055] 2B in particular, in some implementations, the second process 202 executes a classifier 180 to detect, on a sample-by-sample basis, whether the first LM 160 stored a canary text sample 134 during training (FIG. 3A). Notably, in contrast to the first process 201 (FIG. 2A), the second process 202 identifies the particular canary text sample 134 stored by the first LM 160. Moreover, the second process 202 detects storage by the first LM 160 without leveraging any other LMs.

[0056] In some scenarios, the synthetic speech utterance 142 generated by the TTS system 140 is too clear, so the decoder 206 accurately transcribes the speech without utilizing the first LM 160. In other scenarios, the synthetic speech utterance 142 generated by the TTS system 140 is too unclear, so the decoder 206 does not generate any transcription of the speech, and therefore the first LM 160 cannot rescore the transcription. To that end, the TTS system 140 extends the synthetic speech utterance 142 generated for each canary text sample 134. In particular, the TTS system generates the extended synthetic speech utterance 142, 142A by adding noise to the suffix portion 144 of the corresponding synthetic speech utterance 142 without adding any noise to the prefix portion 146 of the corresponding synthetic speech utterance 142. By expanding only the suffix portion 144, the TTS system 140 ensures that the second process 202 leverages the first LM 160 to rescore the initial transcription 232 without overly obscuring the expanded synthetic voice utterance 142A, so that the decoder 206 does not generate any transcription of the expanded synthetic voice utterance 142A.

[0057] For each canary text sample 134, the first LM 160 rescores the initial transcription 232 generated by the decoder 206 for the corresponding augmented synthetic speech utterance 142A to generate the transcription 120. For each canary text sample 134, the classifier 180 is configured to receive the transcription 120 generated by the first LM 160 for the corresponding augmented synthetic speech utterance 142A and the corresponding canary text sample 134. Here, the classifier 180 determines that the corresponding canary text sample 134 was used to train the first LM 160 based on the transcription 120 generated for the corresponding augmented synthetic speech utterance 142A that matches the corresponding canary text sample 134. In other words, the classifier 180 determines whether the canary text sample 134 used to train the first LM 160 was unintentionally stored by the first LM 160 during training. Here, the classifier 180 may determine whether the first LM 160 has stored the respective canary text sample 134 by determining that, for each canary text sample 134, the transcription 120 generated by the first LM 160 satisfies a threshold difference between the transcription 120 and the respective canary text sample 134. In some configurations, the transcription 120 must exactly match the respective canary text sample 134 to be classified as having been stored by the first LM 160. Thus, for each canary text sample 134, the classifier 180 outputs a corresponding indicator 182 that indicates whether the respective canary text sample 134 was or was not stored by the first LM 160.

[0058] However, the processes 201, 202 simply detect whether the first LM 160 has stored any canary text samples 134, without reducing storage. To that end, referring back to FIG. 3A, the first exemplary training process 300a may further include mitigating the detected storage of the canary text samples 134 by the first LM 160 by applying per-sample gradient clipping while training the first LM 160. In some examples, the first exemplary training process 300a clips the gradient of each training text sample 130 to a fixed maximum clipping criterion and then adds random noise to the average clipped gradient. In these examples, gradient clipping ensures robust privacy of the first LM 160, at the significant expense of reduced generalization during inference. Thus, the first exemplary training process 300a applies sample-by-sample gradient clipping to each canary text sample 134 to clip gradients from a predetermined number of the canary text samples 134, thereby limiting the sensitivity of the clipped gradients of the canary text samples. Advantageously, applying sample-by-sample gradient clipping in this manner ensures less memory by the first LM 160 during inference without sacrificing the generalization of the first LM 160. Optionally, sample-by-sample gradient clipping may be applied for each training text sample 130.

[0059] 4 is a flowchart of an exemplary arrangement of operations of a computer-implemented method 400 for detecting that a language model has unintentionally stored a canary text sample. The data processing hardware 510 (FIG. 5) may execute instructions stored in the memory hardware 520 (FIG. 5) that cause the data processing hardware 510 to perform the operations of the method 400. The data processing hardware 510 may include the data processing hardware 12 of the user device 10 or the data processing hardware present in the server 60. The memory hardware 520 may include the memory hardware 14 of the user device 10 or the memory hardware present in the server 60. At operation 402, the method 400 includes inserting the set of canary text samples 134 into a first corpus of training text samples 130. At operation 404, the method 400 includes training a first LM 160 with the first corpus of training text samples 130 and the set of canary text samples 134 inserted into the first corpus of training text samples 130. For each canary text sample 134 in the set of canary text samples 134, the method 400 performs operations 406 and 408. At operation 406, the method 400 includes generating a corresponding synthetic voice utterance 142 for each canary text sample 134 using the TTS system 140. At operation 408, the method 400 includes generating an initial transcription 232 for the corresponding synthetic voice utterance 142 using the trained ASR model 200 configured to receive the corresponding synthetic voice utterance 142 as input.

[0060] At operation 410, the method 400 includes using the first LM 160 to rescore the generated initial transcription 232 for each corresponding synthetic speech utterance. Here, the rescored initial transcription output by the first LM 160 may correspond to the transcription 120. For example, when the trained ASR model 200 operates in streaming mode, the first LM 160 outputs the rescored initial transcription corresponding to the initial speech recognition result 120, 120a. On the other hand, when the trained ASR model 200 operates in non-streaming mode, the first LM 160 outputs the rescored initial transcription corresponding to the final speech recognition result 120, 120b. At operation 412, the method 400 includes determining a first WER 152 of the first LM 160 based on the rescored initial transcription (e.g., transcription) 120 and the canary text sample (e.g., ground truth label) 134. At operation 414, the method 400 includes detecting storage of the canary text sample 134 by the first LM 160 based on the first WER 152 of the first LM 160. In some examples, the method 400 detects storage of the canary text sample 134 by comparing the first WER 152 of the first LM 160 to the second WER 154 of the second LM 170. In other examples, the method 400 detects storage of the canary text sample by using the classifier 180.

[0061] 5 is a schematic diagram of an exemplary computing device (e.g., system) 500 that may be used to implement the systems and methods described in this document. Computing device 500 is intended to represent various types of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The components illustrated herein, their connections and relationships, and their functionality are for illustrative purposes only and are not intended to limit the implementation of the invention described and / or claimed in this document.

[0062] Computing device 500 includes a processor 510, memory 520, storage device 530, a high-speed interface / controller 540 that connects to memory 520 and high-speed expansion port 550, and a low-speed interface / controller 560 that connects to low-speed bus 570 and storage device 530. Each component 510, 520, 530, 540, 550, and 560 are interconnected using various buses and may be mounted on a common motherboard or may exist in other manners as desired. Processor (e.g., data processing hardware) 510 may process instructions for execution within computing device 500, including instructions stored in memory 520 or storage device 530, and display graphical information for a graphical user interface (GUI) on an external input / output device, such as a display 580 connected to high-speed interface 540. In other implementations, multiple processors and / or multiple buses may be used as desired, along with multiple memories and memory types. Also, multiple computing devices 500 may be connected, with each device providing a portion of the required operations (eg, as a bank of servers, a group of blade servers, or a multi-processor system).

[0063] The memory (e.g., memory hardware) 520 stores information non-transiently within the computing device 500. The memory 520 may be a computer-readable medium, a volatile memory unit(s), or a non-volatile memory unit(s). The non-transient memory 520 may be a physical device used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program state information) for use by the computing device 500. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0064] The storage device 530 can provide mass storage for the computing device 500. In some implementations, the storage device 530 is a computer-readable medium. In various different implementations, the storage device 530 can be a device array, including a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or a storage area network or other configuration of devices. In additional implementations, the computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer-readable medium or a machine-readable medium, such as the memory 520, the storage device 530, or a memory on the processor 510.

[0065] The high-speed controller 540 manages bandwidth-intensive operations of the computing device 500, and the low-speed controller 560 manages low-bandwidth-intensive operations. Such an allocation of roles is merely an example. In some implementations, the high-speed controller 540 is coupled to the memory 520, the display 580 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 550 that can accept various expansion cards (not shown). In some implementations, the low-speed controller 560 is coupled to the storage device 530 and the low-speed expansion port 590. The low-speed expansion port 590, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled, for example, via a network adapter, to one or more input / output devices such as a keyboard, a pointing device, a scanner, or a network device such as a switch or router.

[0066] The computing device 500 can be implemented in many different forms, as shown in the figure: for example, it may be implemented as a standard server 500a, or multiple times within a group of such servers 500a, as a laptop computer 500b, or as part of a rack server system 500c.

[0067] Various implementations of the systems and techniques described herein may be realized in digital electronic and / or optical circuitry, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may be specialized or general purpose and may include implementations in one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, at least one input device, and at least one output device coupled to receive data and instructions from and transmit data and instructions to the storage system.

[0068] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language, and / or assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0069] The processes and logic flows described herein can be implemented by one or more programmable processors, also referred to as data processing hardware, executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be implemented by special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Processors suitable for executing computer programs include, for example, both general purpose and special purpose microprocessors, as well as any one or more processors of any type of digital computer. Generally, a processor receives instructions and data from a read-only memory, a random access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices, such as magnetic disks, magneto-optical disks, or optical disks, for storing data, or is operatively connected to receive data from or transmit data to them, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all types of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0070] To provide for interaction with a user, one or more aspects of the present disclosure can be implemented in a computer having a display device (e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor) or touch screen for displaying information to a user, and optionally a keyboard and pointing device, e.g., a mouse or trackball, by which the user can provide input to the computer. Other types of devices can also be used to provide interaction with a user, e.g., feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic, spoken language, or tactile input. Additionally, the computer can interact with a user by, for example, sending documents to and receiving documents from a device used by the user in response to requests received from a web browser.

[0071] Although several embodiments have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the disclosure. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A computer-implemented method (400), when executed on data processing hardware (510), causing the data processing hardware (510) to: inserting a set of canary text samples (134) into a corpus of training text samples (130); training an external language model (160) with the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130); For each canary text sample (134) in the set of canary text samples (134), generating a corresponding synthetic voice utterance (142) using a text-to-speech (TTS) system (140); generating an initial transcription (232) of the corresponding synthetic speech utterance (142) using a trained automatic speech recognition (ASR) model (200) configured to receive the corresponding synthetic speech utterance (142) as input; rescoring the initial transcriptions (232) generated for each corresponding synthetic speech utterance (142) using the external language model (160) trained on the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130); determining a word error rate (WER) (152) of the external language model (160) based on the rescored initial transcription (120) and the canary text sample (134); detecting storage of the canary text sample (134) by the external language model (160) based on the WER (152) of the external language model (160); The computer-implemented method (400) performs operations including:

2. 2. The computer-implemented method of claim 1, wherein a lower WER of the external language model corresponds to increased storage of the canary text samples by the external language model.

3. The operation, inserting a set of foreign text samples (136) into a second corpus of training text samples (136); training a second foreign language model (170) with the second corpus of training text samples (131) and the set of foreign text samples (136) inserted into the second corpus of training text samples (131); for each canary text sample (134) in the set of canary text samples (134), receiving the initial transcription (232) generated by the trained ASR model (200) for the corresponding synthetic speech utterance (142); rescoring the initial transcriptions (232) generated for each corresponding synthetic speech utterance (142) using the second external language model (170) trained on a second corpus of training text samples (131) and a set of foreign text samples (136) inserted into the second corpus of training text samples (131); determining a second WER (154) of the second external language model (170) based on the initial transcription (232) and the canary text sample (134) rescored by the second external language model (170); detecting storage of the canary text sample (134) by the external language model (160) by comparing the WER (152) of the external language model (160) with the second WER (154) of the second external language model (170); The computer-implemented method (400) of claim 1 or 2, further comprising:

4. 3. The computer-implemented method of claim 1, wherein the operations further comprise: mitigating the detected storage of the canary text samples by the external language model when training the external language model by applying sample-wise gradient clipping by clipping gradients from a predetermined number of the canary text samples.

5. The operation comprises: for each canary text sample (134) in the set of canary text samples (134): adding noise to a suffix portion (144) of the corresponding synthetic speech utterance (142) without adding any noise to a prefix portion (146) of the corresponding synthetic speech utterance (144); determining, using a classifier (180), based on the rescored initial transcription (120) generated for the corresponding synthetic speech utterance (142) that matches the corresponding canary text sample (134), that the corresponding canary text sample (134) was used to train the external language model (160); The computer-implemented method (400) of claim 1 or 2, further comprising:

6. 3. The computer-implemented method of claim 1, wherein each canary text sample in the set of canary text samples comprises a fixed-length sequence of random alphanumeric characters, each separated by a space.

7. inserting the set of canary text samples (134) into the corpus of training text samples (134), inserting each canary text sample (134) of a first portion of the canary text samples in the set of canary text samples (134) only once into the corpus of training text samples (130); inserting each canary text sample (134) of a second portion of canary text samples in the set of canary text samples (134) into the corpus of training text samples (130) two or more times, the second portion of canary text samples including canary text samples (134) that are different from the first portion of canary text samples; The computer-implemented method (400) of claim 1 or 2, comprising:

8. The computer-implemented method of claim 1 or 2, wherein the external language model comprises an external neural language model.

9. 10. The computer-implemented method of claim 8, wherein the external neural language model comprises a stack of transformer or conformer layers.

10. 3. The computer-implemented method of claim 1, wherein the operations further include integrating the trained external language model with the trained ASR model, the trained external language model being configured to rescore a probability distribution over possible speech recognition hypotheses predicted by the trained ASR model.

11. The trained ASR model (200) A first encoder (210), receiving as input a sequence of acoustic frames (110); generating, at each of a plurality of output steps, a first high-level feature representation (212) of a corresponding acoustic frame (110) in the sequence of acoustic frames (110); the first encoder (210) configured to: A second encoder (220), receiving as input the first high-level feature representation (212) produced by the first encoder (210) at each of the plurality of output steps; generating a second high-level feature representation (222) for a corresponding first high-level feature frame (212) at each of the plurality of output steps; the second encoder (220) configured to A decoder (206), receiving as input the second high-level feature representation (222) produced by the second encoder (220) at each of the plurality of output steps; generating a first probability distribution (232) over possible speech recognition hypotheses at each of the plurality of output steps; The decoder (206) is configured to The computer-implemented method (400) of claim 1 or 2, comprising:

12. The decoder (206) or another decoder receiving as input the first high-level feature representation (212) produced by the first encoder (210) at each of the plurality of output steps; generating a second probability distribution (232) over the possible speech recognition hypotheses at each of the plurality of output steps; The computer-implemented method (400) of claim 11, further configured to:

13. The decoder (206) A prediction network (240), wherein at each of the plurality of output steps: receiving as input the sequence of N previous non-blank symbols output by the final softmax layer; generating a respective embedding for each non-blank symbol in the sequence of N previous non-blank symbols; generating an average embedding (242) by averaging the respective embeddings; The prediction network (240) configured to: A collaborative network (230), comprising: as inputs, the mean embedding (242) generated by the prediction network (240) at each of the plurality of output steps; and the first high-level feature representation (212) generated by the first encoder (210) at each of the plurality of output steps when the ASR model (200) is operating in a streaming mode; or the second high-level feature representation (222) generated by the second encoder (220) at each of the plurality of output steps when the ASR model (200) is operating in a non-streaming mode. and In each of the plurality of output steps, the second probability distribution (232) over possible speech recognition hypotheses when the ASR model (200) is operating in the streaming mode; or When the ASR model (200) is operating in the non-streaming mode, the first probability distribution (232) over possible speech recognition hypotheses. and The collaborative network (230) configured to The computer-implemented method (400) of claim 12, comprising:

14. the first encoder (210) comprises a causal encoder including an initial stack of conformer layers; the second encoder (220) comprising a non-causal encoder including a final stack of conformer layers overlying the initial stack of conformer layers; 12. The computer-implemented method (400) of claim 11.

15. 12. The computer-implemented method of claim 11, wherein the first encoder (210) and the second encoder (220) of the ASR model are trained using hybrid autoregressive transducer factorization to facilitate integration of the external language model (160) trained on text-only data including a corpus of the training text samples (130) and a set of the canary text samples (134) inserted into the corpus of training text samples (130).

16. A system (500), Data processing hardware (510); a memory hardware (520) in communication with the data processing hardware (510), the memory hardware (520), when executed on the data processing hardware (510), providing to the data processing hardware (510): inserting a set of canary text samples (134) into a corpus of training text samples (130); training an external language model (160) with the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130); For each canary text sample (134) in the set of canary text samples (134), generating a corresponding synthetic voice utterance (142) using a text-to-speech (TTS) system (140); generating an initial transcription (232) of the corresponding synthetic speech utterance (142) using a trained automatic speech recognition (ASR) model (200) configured to receive the corresponding synthetic speech utterance (142) as input; rescoring the initial transcriptions (232) generated for each corresponding synthetic speech utterance (142) using the external language model (160) trained on the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130); determining a word error rate (WER) (152) of the external language model (160) based on the rescored initial transcription (120) and the canary text sample (134); detecting storage of the canary text sample (134) by the external language model (160) based on the WER (152) of the external language model (160); said memory hardware (520) storing instructions for performing operations including: A system (500).

17. 20. The system of claim 16, wherein a lower WER of the external language model corresponds to increased storage of the canary text samples by the external language model.

18. The operation, inserting a set of foreign text samples (136) into a second corpus of training text samples (136); training a second foreign language model (170) with the second corpus of training text samples (131) and the set of foreign text samples (136) inserted into the second corpus of training text samples (131); for each canary text sample (134) in the set of canary text samples (134), receiving the initial transcription (232) generated by the trained ASR model (200) for the corresponding synthetic speech utterance (142); rescoring the initial transcriptions (232) generated for each corresponding synthetic speech utterance (142) using the second external language model (170) trained on a second corpus of training text samples (131) and a set of foreign text samples (136) inserted into the second corpus of training text samples (131); determining a second WER (154) of the second external language model (170) based on the initial transcription (232) and the canary text sample (134) rescored by the second external language model (170); detecting storage of the canary text sample (134) by the external language model (160) by comparing the WER (152) of the external language model (160) with the second WER (154) of the second external language model (170); 18. The system (500) of claim 16 or 17, further comprising:

19. 18. The system (500) of claim 16 or 17, wherein the operations further comprise: mitigating the detected memory of the canary text samples (134) by the external language model (160) by applying sample-wise gradient clipping by clipping gradients from a predetermined number of the canary text samples (134) when training the external language model (160).

20. The operation comprises: for each canary text sample (134) in the set of canary text samples (134): adding noise to a suffix portion (144) of the corresponding synthetic speech utterance (142) without adding any noise to a prefix portion (146) of the corresponding synthetic speech utterance (144); determining, using a classifier (180), based on the rescored initial transcription (120) generated for the corresponding synthetic speech utterance (142) that matches the corresponding canary text sample (134), that the corresponding canary text sample (134) was used to train the external language model (160); 18. The system (500) of claim 16 or 17, further comprising:

21. 18. The system (500) of claim 16 or 17, wherein each canary text sample (134) in the set of canary text samples (134) comprises a fixed-length sequence of random alphanumeric characters, each separated by a space.

22. inserting the set of canary text samples (134) into the corpus of training text samples (134), inserting each canary text sample (134) of a first portion of the canary text samples in the set of canary text samples (134) only once into the corpus of training text samples (130); inserting each canary text sample (134) of a second portion of canary text samples in the set of canary text samples (134) into the corpus of training text samples (130) two or more times, the second portion of canary text samples including canary text samples (134) that are different from the first portion of canary text samples; 18. The system (500) of claim 16 or 17, comprising:

23. The system (500) of claim 16 or 17, wherein the external language model (160) comprises an external neural language model.

24. 24. The system (500) of claim 23, wherein the external neural language model comprises a stack of transformer or conformer layers.

25. 18. The system (500) of claim 16 or 17, wherein the operations further include integrating the trained external language model (160) with the trained ASR model (200), wherein the trained external language model (160) is configured to rescore a probability distribution (232) over possible speech recognition hypotheses predicted by the trained ASR model (200).

26. The trained ASR model (200) A first encoder (210), receiving as input a sequence of acoustic frames (110); generating, at each of a plurality of output steps, a first high-level feature representation (212) of a corresponding acoustic frame (110) in the sequence of acoustic frames (110); the first encoder (210) configured to: A second encoder (220), receiving as input the first high-level feature representation (212) produced by the first encoder (210) at each of the plurality of output steps; generating a second high-level feature representation (222) for a corresponding first high-level feature frame (212) at each of the plurality of output steps; the second encoder (220) configured to A decoder (206), receiving as input the second high-level feature representation (222) produced by the second encoder (220) at each of the plurality of output steps; generating a first probability distribution (232) over possible speech recognition hypotheses at each of the plurality of output steps; The decoder (206) configured to:

18. The system (500) of claim 16 or 17, comprising:

27. The decoder (206) or another decoder receiving as input the first high-level feature representation (212) produced by the first encoder (210) at each of the plurality of output steps; generating a second probability distribution (232) over the possible speech recognition hypotheses at each of the plurality of output steps; 27. The system (500) of claim 26, further configured to:

28. The decoder (206) A prediction network (240), wherein at each of the plurality of output steps: receiving as input the sequence of N previous non-blank symbols output by the final softmax layer; generating a respective embedding for each non-blank symbol in the sequence of N previous non-blank symbols; generating an average embedding (242) by averaging the respective embeddings; and A collaborative network (230), comprising: as inputs, the mean embedding (242) generated by the prediction network (240) at each of the plurality of output steps; and the first high-level feature representation (212) generated by the first encoder (210) at each of the plurality of output steps when the ASR model (200) is operating in a streaming mode; or the second high-level feature representation (222) generated by the second encoder (220) at each of the plurality of output steps when the ASR model (200) is operating in a non-streaming mode. and In each of the plurality of output steps, the second probability distribution (232) over possible speech recognition hypotheses when the ASR model (200) is operating in the streaming mode; or When the ASR model (200) is operating in the non-streaming mode, the first probability distribution (232) over possible speech recognition hypotheses. and The collaborative network (230) configured to:

28. The system (500) of claim 27, comprising:

29. the first encoder (210) comprises a causal encoder including an initial stack of conformer layers; the second encoder (220) comprising a non-causal encoder including a final stack of conformer layers overlying the initial stack of conformer layers; 27. The system (500) of claim 26.

30. 27. The system of claim 26, wherein the first encoder and the second encoder of the ASR model are trained using hybrid autoregressive transducer factorization to facilitate integration of the external language model trained on text-only data including a corpus of the training text samples and a set of the canary text samples inserted into the corpus of training text samples.

Images