Detecting unintentional memory in language model fused ASR systems

Abstract

A method (400) includes inserting a set of canny text samples (134) into a corpus of training text samples (130), and training an external language model (160) on the corpus of training text samples and the set of canny text samples inserted into the corpus of training text samples. For each canny text sample, the method further includes generating a corresponding synthetic speech utterance (142) and generating an initial transcription (232) for the corresponding synthetic speech utterance. The method further includes re-scoring the initial transcription generated for each corresponding synthetic speech utterance using an external language model. The method further includes determining a word error rate (WER) of the external language model based on the re-scored initial transcription and the canny text sample (152), and detecting memory of the canny text sample by the external language model based on the WER of the external language model.

Description

Technical Field

[0001] The present disclosure relates to detecting unintentional memory in language model fusion ASR systems. Background Art

[0002] Automatic speech recognition (ASR), i.e., the process of taking audio input and transcribing it into text, has largely become an important technology used in mobile devices and other devices. Generally speaking, automatic speech recognition attempts to provide an accurate transcription of what people say by taking audio input (e.g., voice utterances) and transcribing the audio input into text. Based on the continuous development of deep neural networks, modern ASR models continue to improve in terms of both accuracy (e.g., low word error rate (WER)) and latency (e.g., the delay between the client speaking and the transcription). In some cases, modern ASR models utilize language models (LMs) to disambiguate acoustically similar transcription hypotheses generated by the ASR model. However, one disadvantage of using LMs includes unintentionally memorizing potentially sensitive text data during the training of the LMs. Therefore, memorizing sensitive text data introduces a privacy vulnerability to the LM, whereby hackers can access the sensitive text data used to train the LM. Summary of the invention

[0003] One aspect of the present disclosure provides a computer-implemented method that, when executed on data processing hardware, causes the data processing hardware to perform operations for detecting unintentional memory in a language model automatic speech recognition (ASR) system. The operations include inserting a set of canary text samples into a corpus of training text samples, and training an external language model on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample in the set of canary text samples, the operations include generating a corresponding synthetic speech utterance using a text-to-speech (TTS) system, and generating an initial transcription for the corresponding synthetic speech utterance using a trained ASR model configured to receive the corresponding synthetic speech utterance as input. The operations also include rescoring the initial transcription generated for each corresponding synthetic speech utterance using an external language model trained on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. The operations also include determining a word error rate (WER) of the external language model based on the re-scored initial transcription and the canary text sample, and detecting memorization of the canary text sample by the external language model based on the WER of the external language model.

[0004] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, a lower WER of the external language model corresponds to an increased memory of the canary text sample by the external language model. In some examples, the operation further includes: inserting a set of irrelevant text samples into a second corpus of training text samples; training a second external language model on the second corpus of training text samples and the set of irrelevant text samples inserted into the second corpus of training text samples; for each canary text sample in the set of canary text samples, receiving an initial transcription generated by the trained ASR model for the corresponding synthetic speech utterance; using a second language model trained on the second corpus of training text samples and the set of irrelevant text samples inserted into the second corpus of training text samples to re-score the initial transcription generated for each corresponding synthetic speech utterance; determining a second WER of the second external language model based on the initial test transcription re-scored by the second external language model and the canary text sample; and detecting the memory of the canary text sample by the external language model by comparing the WER of the external language model with the second WER of the second external language model.

[0005] The operations may further include: when training the external language model, applying per-sample gradient clipping by clipping gradients from a specified number of canary text samples to reduce detected memorization of canary text samples by the external language model. In some implementations, the operations further include: for each canary text sample in the set of canary text samples, adding noise to a suffix portion of the corresponding synthetic speech utterance without adding any noise to a prefix portion of the corresponding synthetic speech utterance, and using a classifier to determine the corresponding canary text sample for training the external language model based on matching the re-scored initial transcription generated for the corresponding synthetic speech utterance with the corresponding canary text sample. Each canary text sample in the set of canary text samples may include a fixed-length sequence of random alphanumeric characters, each separated by spaces.

[0006] In some examples, inserting a set of canary text samples into a corpus of training text samples includes: inserting each canary text sample in a first portion of canary text samples in the set of canary text samples into the corpus of training text samples once, and inserting each canary text sample in a second portion of canary text samples in the set of canary text samples into the corpus of training text samples twice or more. In these examples, the second portion of canary text samples includes canary text samples different from the first portion of canary text samples. The external language model may include an external neural language model. Here, the external neural language model may include a stack of transformer layers or Conformer layers. In some implementations, the operation further includes integrating the trained external language model with the trained ASR model, wherein the trained external language model is configured to re-score the probability distribution of possible speech recognition hypotheses predicted by the trained ASR model.

[0007] In some examples, the trained ASR model includes: a first encoder configured to receive a sequence of acoustic frames as input and, at each of a plurality of output steps, generate a first high-order feature representation for a corresponding acoustic frame in the sequence of acoustic frames; a second encoder configured to receive as input the first high-order feature representation generated by the first encoder at each of a plurality of output steps and, at each of a plurality of output steps, generate a second high-order feature representation for the corresponding first high-order feature frame; and a decoder configured to receive as input the second high-order feature representation generated by the second encoder at each of a plurality of output steps and generate a first probability distribution of possible speech recognition hypotheses at each of a plurality of time steps. Here, the decoder or another decoder may be further configured to receive as input the first high-order feature representation generated by the first encoder at each of a plurality of output steps and generate a second probability distribution of possible speech recognition hypotheses at each of a plurality of output steps.

[0008] In some implementations, the decoder includes: a prediction network configured to receive as input a sequence of N previous non-blank symbols output by a last Softmax layer at each output step in a plurality of output steps, generate a corresponding embedding for each non-blank symbol in the sequence of N previous non-blank symbols, and generate an average embedding by averaging the corresponding embeddings; and a joint network configured to receive as input the average embedding generated by the prediction network at each output step in a plurality of output steps and one of the following: a first high-order feature representation generated by a first encoder at each output step in a plurality of output steps when the ASR model operates in streaming mode, or a second high-order feature representation generated by a second encoder at each output step in a plurality of output steps when the ASR model operates in non-streaming mode. In these implementations, the decoder is further configured to generate at each output step in a plurality of output steps one of the following: a second probability distribution of possible speech recognition hypotheses when the ASR model operates in streaming mode, or a first probability distribution of possible speech recognition hypotheses when the ASR model operates in non-streaming mode. The first encoder may include a causal encoder having an initial stack of Conformer layers, and the second encoder includes a non-causal encoder having a final stack of Conformer layers overlaid on the initial stack of Conformer layers. In some examples, the first encoder and the second encoder of the ASR model are trained using a hybrid autoregressive transducer decomposition to facilitate integration of an external language model trained on plain text data, the plain text data including a corpus of training text samples and a set of canary text samples inserted into the corpus of training text samples.

[0009] Another aspect of the present disclosure provides a system comprising data processing hardware and memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include inserting a set of canary text samples into a corpus of training text samples, and training an external language model on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample in the set of canary text samples, the operations include generating a corresponding synthetic speech utterance using a text-to-speech (TTS) system, and generating an initial transcription for the corresponding synthetic speech utterance using a trained ASR model configured to receive the corresponding synthetic speech utterance as input. The operations also include rescoring the initial transcription generated for each corresponding synthetic speech utterance using an external language model trained on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. The operations also include determining a word error rate (WER) of the external language model based on the re-scored initial transcription and the canary text sample, and detecting memorization of the canary text sample by the external language model based on the WER of the external language model.

[0010] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, a lower WER of the external language model corresponds to an increased memory of the canary text sample by the external language model. In some examples, the operation further includes: inserting a set of irrelevant text samples into a second corpus of training text samples; training a second external language model on the second corpus of training text samples and the set of irrelevant text samples inserted into the second corpus of training text samples; for each canary text sample in the set of canary text samples, receiving an initial transcription generated by the trained ASR model for the corresponding synthetic speech utterance; using a second language model trained on the second corpus of training text samples and the set of irrelevant text samples inserted into the second corpus of training text samples to re-score the initial transcription generated for each corresponding synthetic speech utterance; determining a second WER of the second external language model based on the initial test transcription re-scored by the second external language model and the canary text sample; and detecting the memory of the canary text sample by the external language model by comparing the WER of the external language model with the second WER of the second external language model.

[0011] The operations may further include: when training the external language model, applying per-sample gradient clipping by clipping gradients from a specified number of canary text samples to reduce detected memorization of canary text samples by the external language model. In some implementations, the operations further include: for each canary text sample in the set of canary text samples, adding noise to a suffix portion of the corresponding synthetic speech utterance without adding any noise to a prefix portion of the corresponding synthetic speech utterance, and using a classifier to determine the corresponding canary text sample for training the external language model based on matching the re-scored initial transcription generated for the corresponding synthetic speech utterance with the corresponding canary text sample. Each canary text sample in the set of canary text samples may include a fixed-length sequence of random alphanumeric characters, each separated by spaces.

[0012] In some examples, inserting a set of canary text samples into a corpus of training text samples includes: inserting each canary text sample in a first portion of canary text samples in the set of canary text samples into the corpus of training text samples once, and inserting each canary text sample in a second portion of canary text samples in the set of canary text samples into the corpus of training text samples twice or more. In these examples, the second portion of canary text samples includes canary text samples different from the first portion of canary text samples. The external language model may include an external neural language model. Here, the external neural language model may include a stack of transformer layers or Conformer layers. In some implementations, the operation further includes integrating the trained external language model with the trained ASR model, wherein the trained external language model is configured to re-score the probability distribution of possible speech recognition hypotheses predicted by the trained ASR model.

[0013] In some examples, the trained ASR model includes: a first encoder configured to receive a sequence of acoustic frames as input and, at each of a plurality of output steps, generate a first high-order feature representation for a corresponding acoustic frame in the sequence of acoustic frames; a second encoder configured to receive as input the first high-order feature representation generated by the first encoder at each of a plurality of output steps and, at each of a plurality of output steps, generate a second high-order feature representation for the corresponding first high-order feature frame; and a decoder configured to receive as input the second high-order feature representation generated by the second encoder at each of a plurality of output steps and generate a first probability distribution of possible speech recognition hypotheses at each of a plurality of time steps. Here, the decoder or another decoder may be further configured to receive as input the first high-order feature representation generated by the first encoder at each of a plurality of output steps and generate a second probability distribution of possible speech recognition hypotheses at each of a plurality of output steps.

[0014] In some implementations, the decoder includes: a prediction network configured to receive as input a sequence of N previous non-blank symbols output by a last Softmax layer at each output step in a plurality of output steps, generate a corresponding embedding for each non-blank symbol in the sequence of N previous non-blank symbols, and generate an average embedding by averaging the corresponding embeddings; and a joint network configured to receive as input the average embedding generated by the prediction network at each output step in a plurality of output steps and one of the following: a first high-order feature representation generated by a first encoder at each output step in a plurality of output steps when the ASR model operates in streaming mode, or a second high-order feature representation generated by a second encoder at each output step in a plurality of output steps when the ASR model operates in non-streaming mode. In these implementations, the decoder is further configured to generate at each output step in a plurality of output steps one of the following: a second probability distribution of possible speech recognition hypotheses when the ASR model operates in streaming mode, or a first probability distribution of possible speech recognition hypotheses when the ASR model operates in non-streaming mode. The first encoder may include a causal encoder having an initial stack of Conformer layers, and the second encoder includes a non-causal encoder having a final stack of Conformer layers overlaid on the initial stack of Conformer layers. In some examples, the first encoder and the second encoder of the ASR model are trained using a hybrid autoregressive transducer decomposition to facilitate integration of an external language model trained on plain text data, the plain text data including a corpus of training text samples and a set of canary text samples inserted into the corpus of training text samples.

[0015] The details of one or more implementations of the present disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a diagram of an example speech environment using a cascaded encoder and language model architecture for automatic speech recognition.

[0017] Figure 2A and Figure 2B is a diagram of an example process for detecting unintentional memory of a canary text sample.

[0018] Figure 3A and Figure 3B is a diagram of an example training process for training an external language model.

[0019] Figure 4is a flow chart of an example arrangement of operations of a method of detecting unintentional memory in a language model fused automatic speech recognition system.

[0020] Figure 5 is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0021] Like reference numerals in the various drawings indicate like elements. DETAILED DESCRIPTION

[0022] Traditionally, an end-to-end (E2E) automatic speech recognition (ASR) model includes an encoder and a decoder as main components. For example, the encoder generates encoded input audio data, and the decoder generates candidate transcriptions corresponding to the input audio data. Many modern ASR models further utilize an external language model (LM), which is configured to use language information to disambiguate acoustically similar candidate transcriptions generated by the ASR model. For example, based on the language information that "directions to dog park" has a higher probability of being spoken by the user, the LM may determine that "directions to dog park" is more likely to be an accurate transcription than "directions to dog bark". Specifically, by utilizing the LM, the ASR model can significantly improve the performance (e.g., word error rate (WER)) of recognizing rare words or out-of-vocabulary (OOV) words spoken by the user.

[0023] The LM is trained on millions of training text samples, including both generic text samples and user-specific text samples, such as contact lists, typing patterns, and other user-specific information. Thus, training the LM on user-specific text samples for a particular user results in the LM generating transcriptions that are specifically tailored for that user. For example, the LM can improve an incorrect transcription "call Hailee" generated by an ASR model to a correct transcription "call Haleigh" based on training the LM on contact names associated with the user.

[0024] However, a major disadvantage of using user-specific text samples to train LM is that LM may unintentionally memorize potentially sensitive information specific to the user. For example, using a user's credit card number to train LM can make LM produce accurate transcription when the user speaks his credit card number, but LM may also unintentionally memorize the credit card number. As used herein, user-specific text samples that are unintentionally memorized by LM for training LM are referred to as canary text samples. Canary text samples may include sensitive information associated with the user. The memory of canary text samples can be exploited by hackers, who can determine these memorized canary text samples, thereby exploiting potential sensitive information. However, there are some obstacles in determining whether LM has unintentionally memorized canary text samples. That is, the ASR model is configured to receive audio data (rather than text data) as input and generate a single transcription as output for each utterance. Therefore, when LM cannot be directly accessed to obtain a list of the N best transcriptions, conventional attacks on LM cannot determine any memorized text data.

[0025] Therefore, the implementation of the present invention relates to a method and system for detecting canary text samples that are unintentionally memorized by an external LM. More specifically, the method includes: inserting a set of canary text samples into a corpus of training text samples, and training an external LM on the corpus of training text samples and the set of canary text samples inserted into the corpus of training text samples. For each canary text sample, a text-to-speech (TTS) system generates a corresponding synthetic speech utterance, and a trained ASR model generates an initial transcription for the corresponding synthetic speech utterance. The trained external LM re-scores the initial transcription generated for each corresponding synthetic speech utterance. The method uses the re-scored initial transcription and the canary text sample to determine the word error rate (WER) of the external LM and detect the memory of the canary text sample by the external LM. It will be apparent that detecting the memory of the canary text sample may include using another external LM that is not trained on the canary text sample and / or using a classifier. Detecting the memorized canary text sample only identifies potential security vulnerabilities of the LM. To this end, the method and system may further involve applying per-sample gradient clipping by clipping gradients from a specified number of canary text samples to reduce the external LM's memorization of canary text samples during training.

[0026] Figure 1is an example of a speech environment 100. In the speech environment 100, the way in which a user 104 interacts with a computing device such as a user device 10 may be through voice input. The user device 10 (also generally referred to as a device 10) is configured to capture sound (e.g., streamed audio data) from one or more users 104 within the speech environment 100. Here, the streamed audio data may refer to the spoken utterance 106 of the user 104, which serves as an audible query, a command to the user device 10, or an audible communication captured by the device 10. The voice-enabled system of the user device 10 may process the query or command by answering the query and / or causing one or more downstream applications to execute / fulfill the command.

[0027] The user device 10 may correspond to any computing device associated with the user 104 and capable of receiving audio data. Some examples of the user device 10 include, but are not limited to, mobile devices (e.g., smart watches), smart home appliances, Internet of Things (IoT) devices, in-vehicle infotainment systems, smart displays, smart speakers, etc. The user device 10 includes data processing hardware 12 and memory hardware 14 that communicates with the data processing hardware 12 and stores instructions that, when executed by the data processing hardware 12, cause the data processing hardware 12 to perform one or more operations. The user device 10 further includes an audio system 16 having: an audio capture device (e.g., a microphone) 16, 16a for capturing spoken utterances 106 within the speech environment 100 and converting the spoken utterances 106 into electrical signals; and a speech output device (e.g., a speaker) 16, 16b for transmitting audible audio signals (e.g., as output data from the user device 10). Although the user device 10 implements a single audio capture device 16 a in the example shown, the user device 10 may implement an array of audio capture devices 16 a without departing from the scope of the present disclosure, wherein one or more capture devices 16 a in the array may not physically reside on the user device 10 but may communicate with the audio system 16 .

[0028] In the speech environment 100, an automatic speech recognition (ASR) system 118 implements an ASR model (e.g., a trained ASR model) 200 and a first external language model (LM) 160, each of which resides on a user device 10 of a user 104 and / or resides on a remote computing device 60 (e.g., one or more remote servers of a distributed system executed in a cloud computing environment) that communicates with the user device 10 via a network 40. In some examples, the ASR model 200 includes a recurrent neural network transducer (RNN-T) model architecture. The user device 10 and / or the remote computing device 60 also includes an audio subsystem 108 that is configured to receive an utterance 106 spoken by the user 104 and captured by the audio capture device 16a, and convert the utterance 106 into a corresponding digital format associated with an input acoustic frame 110 that can be processed by the ASR system 118. In this example, a user speaks a corresponding utterance 106, and the audio subsystem 108 converts the utterance 106 into corresponding audio data (e.g., a sequence of acoustic frames) 110 for input to an ASR system 118. Thereafter, the ASR model 200 receives as input a sequence of acoustic frames 110 corresponding to the utterance 106, and generates / predicts a corresponding transcription 120 (e.g., speech recognition results / hypotheses) of the utterance 106 at each output step as the ASR model 200 receives (e.g., processes) each acoustic frame 110 in the sequence of acoustic frames 110.

[0029] In the example shown, the ASR model 200 can perform streaming speech recognition to produce initial speech recognition results (e.g., candidate hypotheses) 120, 120a, and generate final speech recognition results (e.g., final hypotheses) 120, 120b by improving the initial speech recognition results 120a. The initial sound recognition results 120a and the final speech recognition results 120b can correspond to partial speech recognition results or entire speech recognition results. In other words, the initial sound recognition results 120a and the final speech recognition results 120b can correspond to a portion of the speech 106 or an entire portion of the speech 106. For example, a partial speech recognition result may correspond to a portion of a spoken speech or even a portion of a spoken word. However, it will be apparent that the ASR model 200 performs additional processing on the final speech recognition result 120b, whereby the final speech recognition result 120b may be delayed relative to the initial speech recognition result 120a. 2, the first external LM 160 may re-score the transcription 120 generated by the ASR model 200 to improve the transcription 120 in either the streaming mode or the non-streaming mode. The first external LM 160 may reside external to the ASR model 200 or integrated with the ASR model 200.

[0030] The user device 10 and / or the remote computing device 60 also executes a user interface generator 107 that is configured to present a representation of a transcription 120 of the utterance 106 to the user 104 of the user device 10. As described in more detail below, the user interface generator 107 can display the initial speech recognition result 120a in a streaming manner during time 1, and then display the final speech recognition result 120b in a streaming manner during time 2. Notably, the ASR model 200 outputs the final speech recognition 120b in a streaming manner, even if the final speech recognition result 120b improves upon the initial speech recognition result 120. In some configurations, the transcription 120 output from the ASR system 118 is processed (e.g., by a natural language understanding (NLU) module executed on the user device 10 or the remote computing device 60) to execute the user command / query specified by the utterance 106. Additionally or alternatively, a text-to-speech system (not shown) (e.g., executing on any combination of user device 10 or remote computing device 60) may convert transcription 120 into synthesized speech for audible output by user device 10 and / or another device.

[0031] In the example shown, the user 104 interacts with a program or application 50 (e.g., a digital assistant application 50) of the user device 10 using the ASR system 118. For example, Figure 1 A user 104 is depicted communicating with a digital assistant application 50 and the digital assistant application 50 displays a digital assistant interface 18 on a screen of a user device 10 depicting a conversation between the user 104 and the digital assistant application 50. In this example, the user 104 asks the digital assistant application 50, "What time is the concert tonight?" This question from the user 104 is a spoken utterance 106 that is captured by an audio capture device 16a and processed by an audio system 16 of the user device 10. In this example, the audio system 16 receives the spoken utterance 106 and converts the spoken utterance 106 into a sequence of acoustic frames 110 for input to an ASR system 118.

[0032] Continuing with the example, the ASR model 200, upon receiving a sequence of acoustic frames 110 corresponding to an utterance 106 spoken by the user 104, encodes the sequence of acoustic frames 110 and then decodes the encoded sequence of acoustic frames 110 into an initial speech recognition result 120a. During time 1, the user interface generator 107 presents a representation of the initial speech recognition result 120a of the utterance 106 to the user 104 of the user device 10 in a streaming manner via the digital assistant interface 18, such that words, word fragments, and / or individual characters appear on the screen immediately as they are spoken. In some examples, the first look-ahead audio context is equal to zero.

[0033] During time 2, the user interface generator 107 presents a representation of the final speech recognition result 120b of the utterance 106 to the user 104 of the user device 10 via the digital assistant interface 18 in a streaming manner, so that the words, word fragments and / or individual characters appear on the screen immediately when generated by the ASR model 200. In some implementations, the user interface generator 107 replaces the representation of the initial speech recognition result 120a presented at time 1 with the representation of the final speech recognition result 120b presented at time 2. Here, time 1 and time 2 may include timestamps corresponding to when the user interface generator 107 presented the corresponding speech recognition results 120. In this example, the timestamp of time 1 indicates that the user interface generator 107 presented the initial speech recognition result 120a at an earlier time than the final speech recognition result 120b. For example, because the final speech recognition result 120b is considered more accurate than the initial speech recognition result 120a, the final speech recognition result 120b that is ultimately displayed as the transcription 120 can repair any terms that may have been misrecognized in the initial speech recognition result 120a. In this example, the streaming initial speech recognition result 120a output by the ASR model 200 is displayed on the screen of the user device 10 at time 1 and is associated with low latency, and provides the user 104 with a response that its query is being processed, while the final speech recognition result 120b output by the ASR model 200 and displayed on the screen at time 2 utilizes additional speech recognition models and / or language models to improve the quality of speech recognition in terms of accuracy, but increases latency. However, since the initial speech recognition result 120a is displayed while the user is speaking the utterance 106, the higher latency associated with generating and ultimately displaying the final speech recognition result 120b is not noticeable to the user 104.

[0034] exist Figure 1In the example shown, the digital assistant application 50 can use natural language processing to respond to questions asked by the user 104. Natural language processing generally refers to the process of interpreting written language (e.g., the initial speech recognition results 120a and / or the final speech recognition results 120b) and determining whether the written language prompts any action. In this example, the digital assistant application 50 uses natural language processing to recognize that the question from the user 104 relates to the user's schedule, and more specifically, to the concert on the user's schedule. By using natural language processing to recognize these details, the automated assistant returns a response 19 to the user's query, where the response 19 indicates "Venue doors open at 6:30PM and concert startsat 8pm". In some configurations, natural language processing occurs on a remote server 60 that communicates with the data processing hardware 12 of the user device 10.

[0035] Figure 2A and Figure 2B A first example process 201 for detecting inadvertent memorization of a canary text sample 134 by a first external LM 160 is shown. Figure 2A ) and a second example process 202 ( Figure 2B ) is a schematic diagram. Processes 201 and 202 are performed on the remote server 60 ( Figure 1 ) is executed at and includes a trained ASR model 200 having a cascaded encoder 204 and a decoder 206. It will be apparent that the ASR model 200 is capable of operating in a streaming mode, a non-streaming mode, and / or both a streaming mode and a non-streaming mode operating in parallel. In addition, the processes 201, 202 may employ one or more components located external to the ASR model 200, including but not limited to a text-to-speech (TTS) system 140 and a first external LM 160.

[0036] Reference now Figure 3A and Figure 3B In some implementations, the training process 300 is configured to train the first external LM 160 and the second external LM 170. It will be apparent that the second external LM 170 can be used as a baseline LM for detecting whether the first external LM 160 has inadvertently memorized any of the canary text samples 134. Specifically, Figure 3AA schematic diagram of a first training process 300, 300a is shown, which is configured to train a first external LM (e.g., a first LM) 160 using a first corpus of training text samples 130. Here, the first corpus of training text samples 130 includes a collection of training text utterances 132, 132a-n and a collection of canary text samples 134, 134a-n. The collection of training text utterances 132 may include generic text samples that are not user-specific or do not contain sensitive information about the user. For example, the training text utterances 132 may include "hey Google", "what appointments do I have this week?" and "what is the weather today?" The first training process 300a also obtains a collection of canary text samples 134, wherein each canary text sample 134 includes a fixed-length sequence (or a dynamic-length sequence) of output tags. Each output tag may include alphanumeric characters, graphemes, phonemes, N-grams, word pieces, and / or words. In some cases, each output tag is separated by a space. In addition, each canary text sample 134 may represent user-specific information or sensitive information associated with the user. For example, a canary text sample 134 may correspond to a password "oe gdbu" or a credit card number "1111 2222 3333 4444". However, in some examples, in order to avoid leaking any actual sensitive information associated with the user, a canary text sample 134 may include a random alphanumeric character sequence (e.g., fictitious sensitive information) whose format is similar to sensitive information (such as a password, serial number, credit card number, and / or other secret information), but does not actually include any sensitive information associated with the user.

[0037] The first training process 300a inserts each canary text sample 134 into the first corpus of training text samples 130. Therefore, each training text sample 130 in the first corpus of training text samples 130 may correspond to a corresponding training text utterance in the training text utterances 132 or a corresponding canary text sample in the canary text samples 134. In addition, the first corpus of training text samples 130 may include any number of training text samples 130. In some implementations, the first training process 300a may insert each canary text sample 134 in a first portion of canary text samples 134 in the set of canary text samples 134 into the first corpus of training text samples 130 a single time (e.g., once), and insert each canary text sample 134 in a second portion of canary text samples 134 in the set of canary text samples 134 into the first corpus of training text samples 130 two or more times. Here, the second portion of canary text samples 134 includes canary text samples 134 that are different from the first portion of canary text samples 134. Therefore, each canary text sample 134 inserted into the first corpus of training text samples 130 may have a different number of occurrences in the first corpus of training text samples 130 .

[0038] The first LM 160 may be configured to receive as input each training text sample 130 (e.g., a training text utterance 132 or a canary text sample 134) in a first corpus of training text samples 130, and generate as output a training transcription 162 for the corresponding training text sample 130 in the first corpus of training text samples 130. In some examples, the first LM 160 receives a subset of output labels for each training text sample 130, and generates the training transcription 162 to predict a remaining subset of output labels for the text sample. For example, for the training text sample 130 "1 2 3 4", the first LM 160 may receive the output labels "1 2 3" and predict the next output label to be "4". In other examples, the first LM 160 receives a masked sequence of output labels for each training text sample 130, thereby assigning a masked value (e.g., a null value) to one or more output labels. And generate training transcription 162 to predict the output label of the masked output label. For example, for the training text sample 130 "pass word", the first LM 160 may receive the output label And predict the mask output value is "a".

[0039] The first training loss module 310 receives the training transcription 162 generated by the first LM 160 for each training text sample 130 and determines a first training loss 312 by comparing the training transcription 162 (e.g., the predicted transcription) with the corresponding training text sample 130 (e.g., the true value label). Thereafter, the first training process 300a provides the first training loss 312 as feedback to the first LM 160 and updates the parameters of the first LM 160 based on the first training loss 312. Thus, the first training process 300a trains the first LM 160 using a set of training text utterances 132 and a set of canary text samples 134 to generate accurate transcriptions. Notably, during the first training process 300a, the first LM 160 may inadvertently memorize one or more canary text samples 134. It will be apparent that any canary text sample 134 memorized by the first LM 160 can be detected by the processes 201, 202 (FIG. 2).

[0040] Reference now Figure 3B , the second training process 300b is configured to train a second external LM (e.g., second LM) 170 using a second corpus of training text samples 131. Here, the second corpus of training text samples 131 includes a set of training text utterances 132 and a set of unrelated text samples 136, 136a-n. That is, similar to the first training process 300a ( Figure 3A ) In contrast, the second training process 300b obtains a set of irrelevant text samples 136 and inserts them into the second corpus of training text samples 131 to replace canary text samples 134. Here, each irrelevant text sample 136 may include a fixed-length sequence (or a dynamic-length sequence) of output tags that are the same as the canary text sample 134. That is, each output tag of the irrelevant text sample 136 may include alphanumeric characters, phonemes, graphemes, N-grams, word pieces, and / or words that are optionally separated by spaces. In addition, each irrelevant text sample 136 may represent user-specific information or sensitive information about the user. However, it will be apparent that each irrelevant text sample 136 in the set of irrelevant text samples 136 is different from each canary text sample 134 in the set of canary text samples 134. Therefore, the irrelevant text sample 136 may also correspond to the password "o eg dbu" or the credit card number "11112222 3333 4444", but the premise is that these text samples have not been included in the set of canary text samples 134.

[0041] Similarly, the second training process 300b inserts each irrelevant text sample 136 into the second corpus of training text samples 131. Therefore, each training text sample 131 in the second corpus of training text samples 131 can correspond to a corresponding training text utterance in the training text utterance 132 or a corresponding irrelevant text sample in the irrelevant text samples 136. The second LM 170 can be configured to receive each training text sample 131 (e.g., training text utterance 132 or irrelevant text sample 136) in the second corpus of training text samples 131 as input, and generate a training transcription 172 as output for the corresponding training text sample 131 in the second corpus of training text samples 131. In some examples, the second LM 170 is configured to receive a subset of output labels for each training text sample 131 and generate a training transcription 172 to predict the remaining subset of output labels for the text sample. In other examples, the second LM 170 receives a masked sequence of output labels for each training text sample 130, thereby assigning mask values ​​to one or more output labels. ) and generate training transcriptions 162 to predict output labels of mask values.

[0042] The second training loss module 320 receives the training transcription 172 generated by the second language model 170 for each training text sample 131, and determines a second training loss 322 by comparing the training transcription 172 (e.g., the predicted transcription) with the corresponding training text sample 131 (e.g., the true value label). Thereafter, the second training process 300b provides the second training loss 322 as feedback to the second LM 170, and updates the parameters of the second LM 170 based on the second training loss 322. Therefore, the second training process 300b uses the set of training text utterances 132 and the set of irrelevant text samples 136 to train the second LM 170 to generate accurate transcriptions. It is worth noting that the second LM 170 is not trained using any canary text samples in the canary text samples 134. Therefore, the second LM 170 may represent a baseline LM for detecting whether the first LM 160 has inadvertently memorized any of the canary text samples 134 during training, since the second LM 170 has not used any of the canary text samples 134 during training and therefore it is unlikely to have memorized any of the canary text samples 134. Therefore, the second LM 170 is only used to detect memorization of canary text samples and is not used during inference.

[0043] Return to reference Figure 2A and Figure 2BIn order to detect whether the first LM 160 has memorized any of the canary text samples 134, the processes 201, 202 need to receive the canary text samples 134 as input. However, the ASR model 200 is configured to receive audio data (e.g., acoustic frames 110) as input rather than text input. To this end, the TTS system 140 is configured to generate a corresponding synthesized speech utterance 142 for each corresponding canary text sample 134 in the set of canary text samples 134 at each output step. Each synthesized speech utterance 142 corresponds to a corresponding sequence of acoustic frames 110 suitable for processing by the cascade encoder 204 of the ASR model 200.

[0044] The cascade encoder 204 refers to a model structure whereby the encoding path includes two encoders 210, 220, which are cascaded so that the output of the first encoder 210 feeds the input of the second encoder 220 before decoding. Here, the first encoder 210 and the second encoder 220 can be cascaded regardless of the underlying architecture of each encoder. The encoders 210, 220 can each include a stack of multi-head (e.g., 8-head) attention layers (e.g., 17 layers). In some examples, the stack of multi-head attention layers of the encoders 210, 220 includes a stack of 512-dimensional Conformer layers. In other examples, a transformer layer can be used instead of a Conformer layer.

[0045] The first encoder 210 can be a causal encoder that includes an initial stack of Conformer layers (e.g., 15 Conformer layers), each layer having a multi-head (e.g., 8-head) attention mechanism used as a self-attention layer. Here, each Conformer layer of the first encoder 210 can use causal convolution and a left context attention layer to restrict the first encoder 210 from using any future input (e.g., the right context is equal to zero). On the other hand, the second encoder 220 can be a non-causal encoder that includes a final stack of Conformer layers (e.g., 2 Conformer layers) overlaid on the initial stack of Conformer layers. Here, each Conformer layer of the second encoder 220 can use a non-causal convolution and a right context attention layer, allowing the second encoder 220 to use (e.g., pay attention to) future inputs. That is, the second encoder 220 can receive and process an additional right context (e.g., a future acoustic frame of 2.88 seconds) to generate an encoder output. Therefore, the first encoder 210 and the second encoder 220 can be interchangeably referred to herein as "causal encoder 210" and "non-causal encoder 220", respectively.

[0046] Continue to refer Figure 2A and Figure 2B , the first encoder 210 receives a d-dimensional feature vector sequence (eg, an acoustic frame 110 sequence) x=(x1, x2, . . . , x T ),in And at each output step, a first high-order feature representation 212 is generated for a corresponding acoustic frame 110 in the sequence of acoustic frames 110. Similarly, the second encoder 220 is cascaded to the first encoder 210, and receives the first high-order feature representation 212 as input, and generates a second high-order feature representation 222 for the corresponding first high-order feature representation 212 at each output step. It is worth noting that the second encoder 220 pays attention to the additional right context to generate each second high-order feature representation 222. However, in some cases, the second encoder 220 generates the second high-order feature representation 222 without receiving any acoustic frame 110 as input. In these cases, the second encoder 220 generates the second high-order feature representation 222 using only the first high-order feature representation 212 as input. The cascaded encoder 204 can operate in a streaming manner so that at each output step, the cascaded encoder 204 generates a first high-order feature representation 212 and a second high-order feature representation 222 corresponding to a portion of the utterance or the entire utterance.

[0047] The decoder 206 may include a recurrent neural network-transducer (RNN-T) architecture having a joint network 230 and a prediction network 240. The prediction network 240 may include a V2 embedding lookup table. The prediction network 240 is configured to receive as input a sequence of N previous non-blank symbols output by the last Softmax layer, and generate a corresponding embedding for each non-blank symbol in the sequence of N previous non-blank symbols. That is, although not shown, the ASR model 200 may include a Softmax layer that receives the output of the decoder 206. In some implementations, the Softmax layer is separate from the decoder 206 and processes the output y from the decoder 206. r The output of the Softmax layer is then used in a beam search process to select orthographic elements. In some examples, the Softmax layer is integrated with the decoder 206 so that the output y of the decoder 206 is r Denotes the output of the Softmax layer. The prediction network 240 is further configured to generate an average embedding 242 by averaging the corresponding embeddings generated for each non-blank symbol.

[0048] In the non-streaming mode, the decoder 206 combines the first high-order feature representation 212 and the second high-order feature representation 222 output by the cascade encoder 204 and the average embedding 242 generated by the prediction network 240 using the joint network 230 to generate an initial transcription (e.g., decoder output 232). The decoder output 232 can be the current subword unit yi (Given a sequence of N previous non-empty symbols {y i-1 ,...,y i-N}) and the probability distribution of input x, P(y i |y i-1 , ...y0,x). In non-streaming mode, the decoder output 232 is output to the first LM 160, which re-scores / improves the initial decoder output 232 from the decoder 206 using a technique such as trellis re-scoring or n-best re-ordering. In other words, the decoder 206 produces the initial transcription 232, and the first LM 160 re-scores the initial transcription 232 to generate a transcription (e.g., a re-scored initial transcription) 120.

[0049] The decoder 206 is configured to generate an initial transcription 232 at each output step, which includes a probability distribution of possible speech recognition hypotheses. In other words, the joint layer 230 generates a probability distribution of possible speech recognition hypotheses at each output step (e.g., time step). Therefore, the initial transcription 232 can be interchangeably referred to as the probability distribution 232 of possible speech recognition hypotheses in this article. Here, "possible speech recognition hypotheses" correspond to a set of output labels / symbols (also referred to as "speech units"), each of which represents a grapheme (e.g., symbol / character) or a word piece in a specified natural language. For example, when the natural language is English, the set of output labels can include twenty-seven (27) symbols, for example, one label for each of the 26 letters in the English alphabet, and one label specifies a space. Therefore, the joint network 230 can output a set of values, which indicates the probability of each of the predetermined set of output labels occurring. The set of values ​​can be a vector (e.g., a unique hot vector) and can indicate a probability distribution of a set of output labels. In some cases, the output labels are graphemes (e.g., individual characters, and possibly punctuation and other symbols), but the set of output labels is not limited to this. For example, the set of output labels may include word fragments and / or entire words in addition to or in lieu of graphemes. The output labels may also be other types of speech units, such as phonemes or subphonemes. The output distribution of the joint network may include a posterior probability value for each different output label. Thus, if there are 100 different output labels representing graphemes or other symbols, the output of the joint network may include 100 different probability values, one for each output label. The probability distribution may then be used (e.g., by a Softmax layer) to select candidate orthographic elements (e.g., graphemes, word fragments, and / or words) and assign scores to them in a beam search process for use in determining an initial transcription 232.

[0050] Continue to refer Figure 2A and Figure 2B In some configurations, the ASR model 200 operates in both streaming mode and non-streaming mode in parallel. When operating in both streaming mode and non-streaming mode simultaneously, the ASR model 200 first performs streaming speech recognition on the audio data 110 using the first encoder 210 to generate a first high-order feature representation 210 for the second encoder 220 and the decoder 206. The decoder 206 then generates a first probability distribution 232 of possible speech recognition hypotheses, which is then rescored by the first LM 160 to produce an initial speech recognition result 120a. Alternatively, in streaming mode, the decoder 206 may directly output the first probability distribution 232 of possible speech recognition hypotheses, thereby bypassing the first LM 160. The ASR model 200 also performs non-streaming speech recognition on the audio data 110, wherein the second encoder 220 generates a second high-order feature representation 222 using the first high-order feature representation 212 received from the first encoder 210. The decoder 206 then generates a second probability distribution 232 of possible speech recognition hypotheses, which is then re-scored by the first LM 160 to produce the final speech recognition result 120b.

[0051] In other configurations, the ASR model 200 operates only in streaming mode. For example, this may occur when the user 104 is using an application that requires as little latency as possible, such as voice search or on-device dictation. Here, the ASR model 200 performs streaming speech recognition on the audio data 110 using only the first encoder 210 to generate a first high-order feature representation 212 for the decoder 206. The decoder 206 then generates a first probability distribution 232 of possible speech recognition hypotheses, which is then rescored by the first LM 160 to produce an initial speech recognition result 120a. Because the streaming mode of the ASR model 200 quickly produces the initial speech recognition result 120a, the inaccuracy of any term is generally acceptable to the user.

[0052] In other configurations, the ASR model 200 operates only in non-streaming mode. For example, non-streaming mode may occur when the user 104 is viewing a transcription of a voicemail left on his / her mobile phone. As described above, such applications benefit from using future contexts to improve performance in exchange for increased processing time. Here, the ASR model 200 first generates a first high-order feature representation 212 for the second encoder 220 using the first encoder 210, but the decoder 206 does not decode the first high-order feature representation 212. Then, the ASR model 200 performs non-streaming speech recognition on the encoded audio data 110, wherein the second encoder 220 uses the first high-order feature representation 212 received from the first encoder 210 to generate a second high-order feature representation 222. Then, the decoder 206 generates a second probability distribution 232 of possible speech recognition hypotheses, which is then re-scored by the first LM 160 to produce a final speech recognition result 120b. Because the non-streaming mode of the ASR model 200 accurately produces the final speech recognition result 120b, the delay in displaying the accurate transcription is generally acceptable to the user.

[0053] In some examples, the first LM 160 includes an external neural language model. Here, the external neural language model may include a stack of multi-head (e.g., 8-head) self-attention layers (e.g., 12 layers), each of which includes a model dimension of 768 and a feed-forward layer dimension of 2048. The stack of multi-head self-attention layers may include a stack of transformer layers or a stack of Conformer layers. For example, the stack of Conformer layers may include a unidirectional Conformer layer that looks back at a predetermined number of tokens (e.g., 31 tokens) predicted by each output word piece model. Here, the first LM 160 is trained to predict 4,096 word pieces. In some cases, the second LM 170 ( Figure 2A ) includes the same structure as the first LM 160.

[0054] Integrating the ASR model 200 with an external LM typically requires shallow fusion. However, overconfidence of the cascaded encoder 204 and decoder 206 can make weighting difficult and often result in extensive word deletion. Here, a trained first LM 160 configured to re-score a probability distribution 232 of possible speech recognition hypotheses predicted by the trained ASR model 200 can be integrated with the trained ASR model 200 using shallow fusion. Thus, a first encoder 210 and a second encoder 220 of the ASR model 200 can be trained using a hybrid autoregressive Transducer (HAT) decomposition to facilitate integration of the first LM 160 trained on plain text data comprising a first corpus of training text samples 130 and a collection of canary text samples 134 inserted into the first corpus of training text samples 130 ( Figure 3A ). In other words, the internal loss scores of the cascaded encoder 204 and decoder 206 can be factored out during the training of the ASR model 200 using the HAT decomposition to facilitate integration with the first LM 160 by optimizing the decoder 206 according to the following:

[0055]

[0056] In Equation 1, p(y|x) represents the logarithmic posterior score of the ASR model 200, pILM(y) represents the internal language model score of the ASR model 200, pLM represents the language model score of the first LM 160, and (λ1, λ2) represents the interpolation weight.

[0057] Now specific reference Figure 2A In some implementations, the first process 201 executes the detection loss module 150 and the second LM 170 to detect whether the first LM 160 is during training ( Figure 3A ) memorizes any canary text sample 134 based on each set of canary text samples 134. That is, the first training process 201 determines whether the first LM 160 memorizes one or more canary text samples 134 without specifically identifying which particular canary text samples 134 are memorized. Here, the first process 201 executes the first LM 160 and the second LM 170 after training has occurred. It is worth noting that the second LM 170 is only used as a baseline LM for detecting unintentional memorization of the first LM 160, and thus the second LM 170 is not used in the ASR system 118 ( Figure 1 ) is used during inference of the training text samples 131. In short, the second LM 170 represents a baseline LM because the second LM 170 is trained on the second corpus ( Figure 3B), which includes training samples similar to canary text samples, but does not actually include any of the canary text samples 134.

[0058] For each corresponding canary text sample 134, the first LM 160 re-scores the initial transcription 232 generated by the decoder 206 for the corresponding synthetic speech utterance 142 to generate the transcription 120. Similarly, for each corresponding canary text sample 134, the second LM 170 re-scores the initial transcription generated by the decoder 206 for the corresponding synthetic speech utterance 142 to generate the detection transcription 174. Thus, for each corresponding canary text sample 134, the first process 201 generates a corresponding transcription 120 and a corresponding detection transcription 174, both of which are associated with the same corresponding canary text sample 134.

[0059] In some examples, the detection loss module 150 is configured to receive, for each corresponding canary text sample 134, the transcription 120 generated by the first LM and the detection transcription 174 generated by the second LM 170 to determine the WER of each corresponding LM 160, 170. That is, the detection loss module 150 determines the first WER 152 of the first LM 160 based on comparing the transcription 120 with the corresponding canary text sample 134 (e.g., the true value label). Similarly, the detection loss module 150 determines the second WER 154 of the second LM 170 by comparing the detection transcription 174 with the corresponding canary text sample 134 (e.g., the true value label). Therefore, the first WER 152 and the second WER 154 represent the overall WER of the entire set of canary text samples 134. The first WER 152 and the second WER 154 can include the cosine similarity between the corresponding canary text sample 134 and the transcription 120 or the detection transcription 174, respectively.

[0060] Thus, the detection loss module 150 may detect that the first LM 160 has memorized the canary text sample 134 by comparing the first WER 152 of the first LM 160 with the second WER 154 of the second LM 170. Specifically, the detection loss module 150 may determine a WER difference between the first WER 152 and the second WER 154, and determine whether the WER difference satisfies a difference threshold. In response to determining that the difference satisfies the difference threshold, the detection loss module 150 detects that the first LM 160 has memorized one or more canary text samples 134 on a per-set basis of the canary text samples 134, without specifically identifying which particular canary text samples 134 are memorized. In short, if the first LM 160 generates a transcription 120 for a set of canary text samples that is, on average, much more accurate than the detection transcription 174 generated by the second LM 170 for the entire set of canary text samples 134, then the detection loss module 150 determines that the first LM 160 has memorized one or more canary text samples 134 during training ( Figure 3A ). Advantageously, by comparing only the WERs of the first LM 160 and the second LM 170, the first process 201 does not need to access the N-best list or confidence level values ​​generated by the first LM 160, which are typically not available to a user without direct access to the first LM 160. In some examples, the detection loss module 150 detects that the first LM 160 has memorized the canary text sample 134 by determining whether the first WER 152 satisfies a WER threshold (without considering the case of the second WER 154). Here, the lower first WER 152 of the first LM 160 corresponds to the first LM 160's increased memorization of the canary text sample 134.

[0061] Now specific reference Figure 2B In some implementations, the second process 202 executes the classifier 180 to detect whether the first LM 160 is during training ( Figure 3A ) memorizes the canary text sample 134 on a per-sample basis. It is worth noting that, unlike the first process 201 ( Figure 2A ) In contrast, the second process 202 identifies a specific canary text sample 134 memorized by the first LM 160. Furthermore, the second process 202 detects the memory of the first LM 160 without utilizing any other LM.

[0062] In some scenarios, the synthesized speech utterance 142 generated by the TTS system 140 is too clear so that the decoder 206 can accurately transcribe the speech without utilizing the first LM 160. In other scenarios, the synthesized speech utterance 142 generated by the TTS system 140 is too unclear so that the decoder 206 does not generate any transcription for the speech, and thus the first LM 160 cannot re-score the transcription. To this end, the TTS system 140 enhances the synthesized speech utterance 142 generated for each canary text sample 134. Specifically, the TTS system generates the enhanced synthesized speech utterances 142, 142A by adding noise to the suffix portion 144 of the corresponding synthesized speech utterance 142 without adding any noise to the prefix portion 146 of the corresponding synthesized speech utterance 142. By enhancing only the suffix portion 144 , the TTS system 140 ensures that the second process 202 re-scores the initial transcription 232 using the first LM 160 without making the enhanced synthesized speech utterance 142A so unintelligible that the decoder 206 does not generate any transcription for the enhanced synthesized speech utterance 142A.

[0063] For each corresponding canary text sample 134, the first LM 160 re-scores the initial transcription 232 generated by the decoder 206 for the corresponding enhanced synthetic speech utterance 142A to generate the transcription 120. The classifier 180 is configured to receive, for each corresponding canary text sample 134, the transcription 120 generated by the first LM 160 for the corresponding enhanced synthetic speech utterance 142A and the corresponding canary text sample 134. Here, the classifier 180 determines that the corresponding canary text sample 134 is used to train the first LM 160 based on the transcription 120 generated for the corresponding enhanced synthetic speech utterance 142A matching the corresponding canary text sample 134. In other words, the classifier 180 determines whether the canary text sample 134 used to train the first LM 160 is unintentionally memorized by the first LM 160 during training. Here, for each corresponding canary text sample 134, the classifier 180 may determine whether the first LM 160 has memorized the corresponding canary text sample 134 by determining that the transcription 120 generated by the first LM 160 satisfies a threshold difference between the transcription 120 and the corresponding canary text sample 134. In some configurations, the transcription 120 must completely match the corresponding canary text sample 134 in order for it to be classified as memorized by the first LM 160. Therefore, the classifier 180 outputs a corresponding indicator 182 for each corresponding canary text sample 134, indicating whether the corresponding canary text sample 134 is memorized by the first LM 160.

[0064] However, the processes 201, 202 only detect whether the first LM 160 has memorized any canary text samples 134, but do not reduce the memory. Figure 3A , the first example training process 300a may further include mitigating the detected memory of the first LM 160 for the canary text sample 134 by applying per-sample gradient clipping when training the first LM 160. In some cases, the first example training process 300a clips the gradient of each training text sample 130 to a fixed maximum clipping norm and then adds random noise to the average clipped gradient. In these cases, gradient clipping ensures strong privacy of the first LM 160, but at the expense of greatly reduced generalization ability during reasoning. Therefore, the first example training process 300a applies per-sample gradient clipping to each canary text sample 134, which clips the gradient from a specified number of canary text samples 134, thereby limiting the sensitivity of the clipped gradient of the canary text sample. Advantageously, applying per-sample gradient clipping in this manner ensures that the memory of the first LM 160 is reduced without sacrificing the generalization of the first LM 160 during reasoning. Optionally, per-sample gradient clipping can be applied to each training text sample 130.

[0065] Figure 4 4 is a flow chart of an exemplary arrangement of operations for a computer-implemented method 400 for detecting that a language model has inadvertently memorized a canary text sample. Figure 5 ) can execute the stored in the memory hardware 520 ( Figure 5) on the user device 10, which causes the data processing hardware 510 to perform the operations of the method 400. The data processing hardware 510 may include the data processing hardware 12 of the user device 10 or the data processing hardware residing on the server 60. The memory hardware 520 may include the memory hardware 14 of the user device 10 or the memory hardware residing on the server 60. At operation 402, the method 400 includes inserting a set of canary text samples 134 into a first corpus of training text samples 130. At operation 404, the method 400 includes training the first LM 160 on the first corpus of training text samples 130 and the 134 sets of canary text samples inserted into the first corpus of the training text samples 130. For each corresponding canary text sample 134 in the 134 sets of canary text samples, the method 400 performs operations 406 and 408. At operation 406, the method 400 includes generating a corresponding synthesized speech utterance 142 for the corresponding canary text sample 134 using the TTS system 140. At operation 408 , the method 400 includes generating an initial transcription 232 for the corresponding synthesized speech utterance 142 using the trained ASR model 200 configured to receive the corresponding synthesized speech utterance 142 as input.

[0066] At operation 410, the method 400 includes rescoring the initial transcription 232 generated for each corresponding synthesized speech utterance using the first LM 160. Here, the rescored initial transcript output by the first LM 160 may correspond to the transcription 120. For example, when the trained ASR model 200 operates in streaming mode, the first LM 160 outputs the rescored initial transcription corresponding to the initial speech recognition results 120, 120a. On the other hand, when the trained ASR model 200 operates in non-streaming mode, the first LM 160 outputs the rescored initial transcription corresponding to the final speech recognition results 120, 120b. In operation 412, the method 400 includes determining a first WER 152 of the first LM 160 based on the rescored initial transcription (e.g., transcription) 120 and the canary text sample (e.g., true value label) 134. At operation 414, the method 400 includes detecting a memory of the canary text sample 134 by the first LM 160 based on the first WER 152 of the first LM 160. In some examples, the method 400 detects the memory of the canary text sample 134 by comparing the first WER 152 of the first LM 160 to the second WER 154 of the second LM 170. In other examples, the method 400 detects the memory of the canary text sample by using the classifier 180.

[0067] Figure 5 5 is a schematic diagram of an example computing device 500 (e.g., system) that can be used to implement the systems and methods described in this document. Computing device 500 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions are intended to be exemplary only and are not intended to limit implementations of the inventions described and / or claimed in this document.

[0068] The computing device 500 includes a processor 510, a memory 520, a storage device 530, a high-speed interface / controller 540 connected to the memory 520 and a high-speed expansion port 550, and a low-speed interface / controller 560 connected to a low-speed bus 570 and the storage device 530. Each of the components 510, 520, 530, 540, 550, and 560 is interconnected using various buses and can be installed on a common motherboard or installed in other ways as appropriate. The processor (e.g., data processing hardware) 510 can process instructions for execution within the computing device 500, including instructions stored in the memory 520 or on the storage device 530, to display graphical information of a graphical user interface (GUI) on an external input / output device (such as a display 580 coupled to the high-speed interface 540). In other implementations, multiple processors and / or multiple buses and multiple memories and multiple types of memories can be used as appropriate. Additionally, multiple computing devices 500 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).

[0069] Memory (e.g., memory hardware) 520 stores information non-temporarily within computing device 500. Memory 520 may be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. Non-temporary memory 520 may be a physical device used to temporarily or permanently store programs (e.g., instruction sequences) or data (e.g., program state information) for use by computing device 500. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0070] Storage device 530 is capable of providing mass storage for computing device 500. In some implementations, storage device 530 is a computer-readable medium. In various different implementations, storage device 530 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or an array of devices (including devices in a storage area network or other configuration). In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 520, storage device 530, or memory on processor 510.

[0071] The high-speed controller 540 manages bandwidth-intensive operations of the computing device 500, while the low-speed controller 560 manages less bandwidth-intensive operations. Such a division of responsibilities is exemplary only. In some implementations, the high-speed controller 540 is coupled to the memory 520, the display 580 (e.g., through a graphics processor or accelerator), and a high-speed expansion port 550 that can accept various expansion cards (not shown). In some implementations, the low-speed controller 560 is coupled to the storage device 530 and the low-speed expansion port 590. The low-speed expansion port 590 (which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)) can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device, such as a switch or a router, for example, through a network adapter.

[0072] As shown, computing device 500 may be implemented in a variety of different forms. For example, computing device may be implemented as a standard server 500a or multiple times in a group of such servers 500a, as a laptop computer 500b, or as part of a rack server system 500c.

[0073] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuit systems, integrated circuit systems, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor that can be either special purpose or general purpose and can be coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0074] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0075] The processes and logic flows described in this specification may be performed by one or more programmable processors (also referred to as data processing hardware) that execute one or more computer programs to perform functions by operating on input data and generating outputs. The processes and logic flows may also be performed by a dedicated logic circuit system, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). For example, processors suitable for executing computer programs include both general-purpose microprocessors and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as a magnetic disk, a magneto-optical disk, or an optical disk, or may be operably coupled to receive data from one or more mass storage devices or to transfer data to one or more mass storage devices or both. However, a computer need not have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0076] To provide interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube), an LCD (liquid crystal display) monitor, or a touch screen) for displaying information to the user, and optionally a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, speech, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0077] A variety of implementations have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of the present disclosure. Therefore, other implementations are within the scope of the appended claims.

Claims

1. A computer-implemented method (400) which, when executed on data processing hardware (510), causes the data processing hardware (510) to perform an operation, characterized in that The operations include: inserting the set of canary text samples (134) into the corpus of training text samples (130); training an external language model (160) on said corpus of training text samples (130) and said set of canary text samples (134) inserted into said corpus of training text samples (130); For each canary text sample (134) in the set of canary text samples (134): generating corresponding synthesized speech utterances (142) using a text-to-speech (TTS) system (140); and generating an initial transcription (232) for the corresponding synthesized speech utterance (142) using a trained automatic speech recognition (ASR) model (200) configured to receive the corresponding synthesized speech utterance (142) as input; rescoring the initial transcription (232) generated for each corresponding synthesized speech utterance (142) using the external language model (160) trained on the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130); determining a word error rate (WER) (152) of the external language model (160) based on the re-scored initial transcription (120) and the canary text sample (134); and Memorization of the canary text sample (134) by the external language model (160) is detected based on the WER (152) of the external language model (160).

2. The computer-implemented method (400) of claim 1, wherein: The lower WER (152) of the external language model (160) corresponds to an increased memory of the canary text sample (134) by the external language model (160).

3. The computer-implemented method (400) of claim 1 or 2, wherein: The operations further include: inserting the set of unrelated text samples (136) into a second corpus of training text samples (136); training a second external language model (170) on said second corpus of training text samples (131) and said set of unrelated text samples (136) inserted into said second corpus of training text samples (131); For each canary text sample (134) in the set of canary text samples (134), receiving the initial transcription (232) generated by the trained ASR model (200) for the corresponding synthesized speech utterance (142); rescoring the initial transcription (232) generated for each corresponding synthesized speech utterance (142) using the second external language model (170) trained on the second corpus of training text samples (131) and the set of unrelated text samples (136) inserted into the second corpus of training text samples (131); determining a second WER (154) of the second external language model (170) based on the initial transcription (232) and the canary text sample (134) re-scored by the second external language model (170); and Memorization of the canary text sample (134) by the external language model (160) is detected by comparing the WER (152) of the external language model (160) with the second WER (154) of the second external language model (170).

4. The computer-implemented method (400) of any one of claims 1 to 3, characterized in that: The operations further include, while training the external language model (160), applying per-sample gradient clipping by clipping gradients from a specified number of the canary text samples (134) to mitigate detected memorization of the canary text samples (134) by the external language model (160).

5. The computer-implemented method (400) of any one of claims 1 to 4, characterized in that: The operations further include, for each canary text sample (134) in the set of canary text samples (134): adding noise to a suffix portion (144) of the corresponding synthesized speech utterance (142) without adding any noise to a prefix portion (146) of the corresponding synthesized speech utterance (144); as well as The corresponding canary text sample (134) is determined to be used for training the external language model (160) based on matching the re-scored initial transcription (120) generated for the corresponding synthesized speech utterance (142) with the corresponding canary text sample (134) using a classifier (180).

6. The computer-implemented method (400) of any one of claims 1 to 5, wherein: Each canary text sample (134) in the set of canary text samples (134) comprises a fixed-length sequence of random alphanumeric characters each separated by spaces.

7. The computer-implemented method (400) of any one of claims 1 to 6, characterized in that: Inserting the set of canary text samples (134) into the corpus of training text samples (134) comprises: inserting each canary text sample (134) in a first portion of the set of canary text samples (134) into the corpus of training text samples (130) a single time; and Each canary text sample (134) in a second portion of canary text samples in the set of canary text samples (134) is inserted into the corpus of training text samples (130) two or more times, the second portion of canary text samples including canary text samples (134) that are different from the first portion of canary text samples.

8. The computer-implemented method (400) of any one of claims 1 to 7, wherein: The external language model (160) comprises an external neural language model.

9. The computer-implemented method (400) of claim 8, wherein: The external neural language model includes a stack of transformer layers or conformer layers.

10. The computer-implemented method (400) of any one of claims 1 to 9, wherein: The operations further include integrating a trained external language model (160) with the trained ASR model (200), the trained external language model (160) being configured to re-score a probability distribution (232) of possible speech recognition hypotheses predicted by the trained ASR model (200).

11. The computer-implemented method (400) of any one of claims 1 to 10, wherein: The trained ASR model (200) comprises: A first encoder (210), wherein the first encoder is configured to: receiving as input a sequence of acoustic frames (110); and At each of a plurality of output steps, generating a first high-order feature representation (212) for a corresponding acoustic frame (110) in the sequence of acoustic frames (110); A second encoder (220), the second encoder being configured to: receiving as input the first high-order feature representation (212) generated by the first encoder (210) at each of the plurality of output steps; and At each of the plurality of output steps, generating a second high-order feature representation (222) for the corresponding first high-order feature frame (212); and A decoder (206), the decoder being configured to: receiving as input the second high-order feature representation (222) generated by the second encoder (220) at each of the plurality of output steps; and At each of the plurality of output steps, a first probability distribution of possible speech recognition hypotheses is generated (232).

12. The computer-implemented method (400) of claim 11, wherein: The decoder (206) or another decoder is further configured to: receiving as input the first high-order feature representation (212) generated by the first encoder (210) at each of the plurality of output steps; as well as At each of the plurality of output steps, a second probability distribution of possible speech recognition hypotheses is generated (232).

13. The computer-implemented method (400) of claim 12, wherein: The decoder (206) comprises: A prediction network (240) configured to, at each output step of the plurality of output steps: Receive as input the sequence of N previous non-blank symbols output by the last Softmax layer; For each non-whitespace symbol in the sequence of N previous non-whitespace symbols, generating a corresponding embedding; and generating an average embedding by averaging the corresponding embeddings (242); and A joint network (230), the joint network being configured to: Receiving as input the average embedding (242) generated by the prediction network (240) at each of the plurality of output steps and one of: the first high-order feature representation (212) generated by the first encoder (210) at each of the plurality of output steps when the ASR model (200) operates in streaming mode; or the second high-order feature representation (222) generated by the second encoder (220) at each of the plurality of output steps when the ASR model (200) operates in a non-streaming mode; and At each output step in the plurality of output steps, one of the following is generated: the second probability distribution (232) of possible speech recognition hypotheses when the ASR model (200) operates in the streaming mode; or The first probability distribution (232) of possible speech recognition hypotheses when the ASR model (200) operates in the non-streaming mode.

14. The computer-implemented method (400) of any one of claims 11 to 13, characterized in that: The first encoder (210) comprises a causal encoder comprising an initial stack of Conformer layers; and The second encoder (220) comprises a non-causal encoder comprising a final stack of Conformer layers overlying the initial stack of Conformer layers.

15. The computer-implemented method (400) of any one of claims 11 to 14, characterized in that: The first encoder (210) and the second encoder (220) of the ASR model are trained using hybrid autoregressive transducer decomposition to facilitate integration of the external language model (160) trained on plain text data, the plain text data comprising the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130).

16. A system (500), characterized in that: include: Data processing hardware (510); as well as Memory hardware (520) in communication with the data processing hardware (510), the memory hardware (520) storing instructions that, when executed on the data processing hardware (510), cause the data processing hardware (510) to perform operations, the operations comprising: inserting the set of canary text samples (134) into the corpus of training text samples (130); training an external language model (160) on said corpus of training text samples (130) and said set of canary text samples (134) inserted into said corpus of training text samples (130); For each canary text sample (134) in the set of canary text samples (134): generating corresponding synthesized speech utterances (142) using a text-to-speech (TTS) system (140); and generating an initial transcription (232) for the corresponding synthesized speech utterance (142) using a trained automatic speech recognition (ASR) model (200) configured to receive the corresponding synthesized speech utterance (142) as input; rescoring the initial transcription (232) generated for each corresponding synthesized speech utterance (142) using the external language model (160) trained on the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130); determining a word error rate (WER) (152) of the external language model (160) based on the re-scored initial transcription (120) and the canary text sample (134); and Memorization of the canary text sample (134) by the external language model (160) is detected based on the WER (152) of the external language model (160).

17. The system (500) of claim 16, wherein: The lower WER (152) of the external language model (160) corresponds to an increased memory of the canary text sample (134) by the external language model (160).

18. The system (500) according to claim 16 or 17, characterized in that The operations further include: inserting the set of unrelated text samples (136) into a second corpus of training text samples (136); training a second external language model (170) on said second corpus of training text samples (131) and said set of unrelated text samples (136) inserted into said second corpus of training text samples (131); For each canary text sample (134) in the set of canary text samples (134), receiving the initial transcription (232) generated by the trained ASR model (200) for the corresponding synthesized speech utterance (142); rescoring the initial transcription (232) generated for each corresponding synthesized speech utterance (142) using the second external language model (170) trained on the second corpus of training text samples (131) and the set of unrelated text samples (136) inserted into the second corpus of training text samples (131); determining a second WER (154) of the second external language model (170) based on the initial transcription (232) and the canary text sample (134) re-scored by the second external language model (170); and Memorization of the canary text sample (134) by the external language model (160) is detected by comparing the WER (152) of the external language model (160) with the second WER (154) of the second external language model (170).

19. The system (500) according to any one of claims 16 to 18, characterized in that The operations further include, while training the external language model (160), applying per-sample gradient clipping by clipping gradients from a specified number of the canary text samples (134) to mitigate detected memorization of the canary text samples (134) by the external language model (160).

20. The system (500) according to any one of claims 16 to 19, characterized in that The operations further include, for each canary text sample (134) in the set of canary text samples (134): adding noise to a suffix portion (144) of the corresponding synthesized speech utterance (142) without adding any noise to a prefix portion (146) of the corresponding synthesized speech utterance (144); as well as The corresponding canary text sample (134) is determined to be used for training the external language model (160) based on matching the re-scored initial transcription (120) generated for the corresponding synthesized speech utterance (142) with the corresponding canary text sample (134) using a classifier (180).

21. The system (500) according to any one of claims 16 to 20, characterized in that Each canary text sample (134) in the set of canary text samples (134) comprises a fixed-length sequence of random alphanumeric characters each separated by spaces.

22. The system (500) according to any one of claims 16 to 21, characterized in that Inserting the set of canary text samples (134) into the corpus of training text samples (134) comprises: inserting each canary text sample (134) in a first portion of the set of canary text samples (134) into the corpus of training text samples (130) a single time; and Each canary text sample (134) in a second portion of canary text samples in the set of canary text samples (134) is inserted into the corpus of training text samples (130) two or more times, the second portion of canary text samples including canary text samples (134) that are different from the first portion of canary text samples.

23. The system (500) according to any one of claims 16 to 22, characterized in that The external language model (160) comprises an external neural language model.

24. The system (500) of claim 23, wherein: The external neural language model includes a stack of transformer layers or conformer layers.

25. The system (500) according to any one of claims 16 to 24, characterized in that The operations further include integrating a trained external language model (160) with the trained ASR model (200), the trained external language model (160) being configured to re-score a probability distribution (232) of possible speech recognition hypotheses predicted by the trained ASR model (200).

26. The system (500) according to any one of claims 16 to 25, characterized in that The trained ASR model (200) comprises: A first encoder (210), wherein the first encoder is configured to: receiving as input a sequence of acoustic frames (110); and At each of a plurality of output steps, generating a first high-order feature representation (212) for a corresponding acoustic frame (110) in the sequence of acoustic frames (110); A second encoder (220), the second encoder being configured to: receiving as input the first high-order feature representation (212) generated by the first encoder (210) at each of the plurality of output steps; and At each of the plurality of output steps, generating a second high-order feature representation (222) for the corresponding first high-order feature frame (212); and A decoder (206), the decoder being configured to: receiving as input the second high-order feature representation (222) generated by the second encoder (220) at each of the plurality of output steps; and At each of the plurality of output steps, a first probability distribution of possible speech recognition hypotheses is generated (232).

27. The system (500) of claim 26, wherein: The decoder (206) or another decoder is further configured to: receiving as input the first high-order feature representation (212) generated by the first encoder (210) at each of the plurality of output steps; as well as At each of the plurality of output steps, a second probability distribution of possible speech recognition hypotheses is generated (232).

28. The system (500) of claim 27, wherein: The decoder (206) comprises: A prediction network (240) configured to, at each output step of the plurality of output steps: Receive as input the sequence of N previous non-blank symbols output by the last Softmax layer; For each non-whitespace symbol in the sequence of N previous non-whitespace symbols, generating a corresponding embedding; and generating an average embedding by averaging the corresponding embeddings (242); and A joint network (230), the joint network being configured to: Receiving as input the average embedding (242) generated by the prediction network (240) at each of the plurality of output steps and one of: the first high-order feature representation (212) generated by the first encoder (210) at each of the plurality of output steps when the ASR model (200) operates in streaming mode; or the second high-order feature representation (222) generated by the second encoder (220) at each of the plurality of output steps when the ASR model (200) operates in a non-streaming mode; and At each output step in the plurality of output steps, one of the following is generated: the second probability distribution (232) of possible speech recognition hypotheses when the ASR model (200) operates in the streaming mode; or The first probability distribution (232) of possible speech recognition hypotheses when the ASR model (200) operates in the non-streaming mode.

29. The system (500) according to any one of claims 26 to 28, characterized in that: The first encoder (210) comprises a causal encoder comprising an initial stack of Conformer layers; and The second encoder (220) comprises a non-causal encoder comprising a final stack of Conformer layers overlying the initial stack of Conformer layers.

30. The system (500) according to any one of claims 16 to 29, characterized in that The first encoder (210) and the second encoder (220) of the ASR model are trained using hybrid autoregressive transducer decomposition to facilitate integration of the external language model (160) trained on plain text data, the plain text data comprising the corpus of training text samples (130) and the set of canary text samples (134) inserted into the corpus of training text samples (130).

Images