Phrase extraction for ASR models

Abstract

The ASR model phrase extraction method (400) includes obtaining audio data (112) characterizing an utterance (106) and a ground truth transcription (114) corresponding to the utterance, and modifying the audio data to obfuscate a particular phrase (116) spoken in the utterance. The method also includes processing the modified audio data (300) using a trained ASR model (150) to generate a predicted transcription (154) of the utterance, and determining whether the predicted transcription includes the particular phrase by comparing the predicted transcription of the utterance to the ground truth transcription of the utterance. When the predicted transcription includes the particular phrase, the method includes generating an output (180) indicating that the trained ASR model leaked the particular phrase from a training dataset (210) used to train the ASR model.

Description

[Technical Field]

[0001] This disclosure relates to a phrase extraction mechanism for an ASR model. [Background technology]

[0002] An automatic speech recognition (ASR) model receives audio data as input and generates a transcription, such as a predicted transcription, as output based on the received audio data input. An ASR model is trained with training data that includes audio data with corresponding transcriptions. However, an ASR model may leak training data that it stored during training. Data leakage of private and / or sensitive training data is generally undesirable. However, verifying the robustness of an ASR model to detect training data leakage can be difficult. Summary of the Invention

[0003] One aspect of the present disclosure provides a computer-implemented method for phrase extraction for an automatic speech recognition (ASR) model. The computer-implemented method, when executed on data processing hardware, causes the data processing hardware to perform operations including obtaining audio data characterizing an utterance and a corresponding ground truth transcription of the utterance, and modifying the audio data to obfuscate a particular phrase spoken in the utterance. The operations also include processing the modified audio data using a trained ASR model to generate a predicted transcription of the utterance, and determining whether the predicted transcription includes the particular phrase by comparing the predicted transcription of the utterance with the ground truth transcription of the utterance. When the predicted transcription includes the particular phrase, the operations include generating an output indicating that the trained ASR model leaked the particular phrase from a training dataset used to train the ASR model.

[0004] Implementations of the present disclosure may include one or more of the following optional features: In some implementations, the operations further include generating an output indicating that the trained ASR model has leaked another phrase from the training dataset used to train the ASR model when the predicted transcription includes another phrase in place of the particular phrase from the ground truth transcription associated with the same category of information as the particular phrase. In some examples, the operations further include generating an output indicating that the trained ASR model has not leaked any information from the training dataset used to train the ASR model when the predicted transcription does not include the particular phrase or another phrase in place of the particular phrase from the ground truth transcription associated with the same category of information as the particular phrase.

[0005] In some implementations, the audio data includes an audio waveform. In these implementations, the audio waveform may correspond to a human voice. Alternatively, the audio waveform may conform to a synthetic voice.

[0006] In some examples, modifying the audio data includes identifying, based on the ground truth transcription, a segment of the audio data that aligns with a particular phrase in the ground truth transcription, and performing data augmentation on the identified segment of the audio data to obfuscate the particular phrase spoken in the utterance. In these examples, performing data augmentation on the identified segment of the audio data may include adding noise to the identified segment of the audio data. Alternatively, performing data augmentation on the identified segment of the audio data may include replacing the identified segment of the audio data with noise.

[0007] In some implementations, the operations further include processing a ground truth transcription of the utterance to identify any phrases included in the ground truth transcription that are associated with a specific category of information, wherein modifying the audio data is performed in response to identifying that a particular phrase included in the ground truth transcription is associated with a specific category of information, in these implementations, the specific category of information may include a name, an address, a date, a zip code, a patient's diagnosis, an account number, or a telephone number.

[0008] Another aspect of the present disclosure provides a phrase extraction system for an ASR model. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations including obtaining audio data characterizing an utterance and a corresponding ground truth transcription of the utterance, and modifying the audio data to obfuscate a particular phrase spoken in the utterance. The operations also include processing the modified audio data using a trained ASR model to generate a predicted transcription of the utterance, and determining whether the predicted transcription includes a particular phrase by comparing the predicted transcription of the utterance with the ground truth transcription of the utterance. When the predicted transcription includes the particular phrase, the operations include generating an output indicating that the trained ASR model leaked the particular phrase from a training dataset used to train the ASR model.

[0009] Various aspects may include one or more of the following optional features: In some implementations, the operations further include generating an output indicating that the trained ASR model has leaked another phrase from the training dataset used to train the ASR model when the predicted transcription includes another phrase in place of the particular phrase from the ground truth transcription associated with the same information category as the particular phrase. In some examples, the operations further include generating an output indicating that the trained ASR model has not leaked any information from the training dataset used to train the ASR model when the predicted transcription does not include the particular phrase or another phrase in place of the particular phrase from the ground truth transcription associated with the same information category as the particular phrase.

[0010] In some implementations, the audio data includes an audio waveform. In these implementations, the audio waveform may correspond to a human voice. Alternatively, the audio waveform may conform to a synthetic voice.

[0011] In some examples, modifying the audio data includes identifying a segment of the audio data that aligns with a particular phrase in the ground truth transcription based on the ground truth transcription, and performing data augmentation on the identified segment of the audio data to obfuscate the particular phrase spoken in the utterance. In these examples, performing data augmentation on the identified segment of the audio data may include adding noise to the identified segment of the audio data. Alternatively, performing data augmentation on the identified segment of the audio data may include replacing the identified segment of the audio data with noise.

[0012] In some implementations, the operations further include processing a ground truth transcription of the utterance to identify any phrases included in the ground truth transcription that are associated with a specific category of information, wherein modifying the audio data is performed in response to identifying a particular phrase included in the ground truth transcription as being associated with a specific category of information, in these implementations, the specific category of information may include a name, an address, a date, a zip code, a patient's diagnosis, an account number, or a telephone number.

[0013] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a schematic diagram of an example phrase extraction mechanism for an automatic speech recognition (ASR) model. [Figure 2] FIG. 2 is a schematic diagram of an exemplary phrase identifier for identifying one or more particular phrases in a transcription. [Figure 3] FIG. 1 is a schematic diagram of an exemplary phrase extraction of one or more specific phrases within a transcription. [Figure 4] 1 is a flowchart of an exemplary configuration of operations of a method for implementing a phrase extraction mechanism of an ASR model. [Figure 5] FIG. 1 is a schematic diagram of an example computing device that may be used to implement the systems and methods described herein. DETAILED DESCRIPTION OF THE INVENTION

[0015] Like reference symbols in the various drawings indicate like elements.

[0016] Embodiments herein are directed to a phrase extraction mechanism for use in detecting leakage of training data used to train an automatic speech recognition (ASR) model. Identifying phrases in a speech utterance that are replaced or masked with noisy audio data and analyzing the output of the trained ASR model to determine whether the trained ASR model output an actual transcription of the utterance demonstrates whether leakage of training data has occurred from the trained ASR model. The phrase extraction mechanism can be used to test whether various types of sensitive information in speech waveforms, such as addresses, zip codes, and patient diagnoses, have been leaked from the trained ASR model.

[0017] 1 illustrates an example of a speech environment including a system 100 with a phrase extraction mechanism 108 executing on a computing device 102. The computing device 102 (also generally referred to as device 102) is configured to capture sounds (e.g., audio data) from one or more users 104 within the speech environment. Here, audio data 112 may refer to utterances 106 spoken by the users 104 that serve as test utterances for recognition by a trained ASR model 150. A speech-enabled system of the device 102, such as the trained ASR model 150, can perform speech recognition on the audio data 112 and generate transcriptions 154 of the corresponding utterances 106 spoken by the users 104.

[0018] The computing device 102 includes data processing hardware 111 and memory hardware 113 that communicates with the data processing hardware 111 and stores instructions that, when executed by the data processing hardware 111, cause the data processing hardware 111 to perform one or more operations. The computing device 102 may include a distributed system (e.g., a cloud computing environment). The computing device 102 also includes a display 105 and / or is in communication with one or more displays. The display 105 can display a leak report 182 that indicates whether the trained ASR model 150 has leaked potentially sensitive data from the training dataset 210 ( FIG. 2 ) used to train the ASR model 150. The computing device 102 may include an audio system with an audio capture device (e.g., a microphone) that captures and converts spoken speech 106 within the audio environment 100 into an electrical signal, and an audio output device (e.g., a speaker) that communicates an audible audio signal (e.g., output audio data from the device 102). The ASR model 150 and / or the leakage detector 170 may be implemented on the computing device 102 as the audio-text aligner 120 and the data extender 130, or the ASR model 150 and / or the leakage detector 170 may be implemented on another computing device in communication with the computing device 102.

[0019] As described in more detail below, the leakage detector 170 may check or verify the robustness of the model 150 by determining whether the model 150 still recognizes particular phrases 116 from the utterance 106 that have been replaced with noise in the audio data 300 modified by the data augmenter 130, and may output an indication 180 indicating whether the model 150 leaked one or more particular phrases 116 from the training dataset 210 used to train the model 150. For example, the leakage detector 170 may output an indication 180 indicating that the model 150 has leaked potentially sensitive, rare training data that the model 150 stored during training. FIG. 1 shows that the computing device 102 captures audio data 112 including an audio waveform of a human voice that characterizes the utterance 106, "Mr. Soames and Mr. Havisham were overwhelmed by this information," spoken by a user 104. Optionally, the user 104 can input a text representation / transcription 114 of the utterance 106, and the text-to-speech (TTS) system 110 can convert the transcription 114 into audio data 112 including a synthesized voice audio waveform characterizing the utterance 106.

[0020] In some implementations, the leakage detector 170 determines whether the model 150 has leaked one or more specific phrases 116 from the training dataset 210 used to train the model 150 without receiving audio data 112 corresponding to the one or more specific phrases 116. In other words, the audio data 112, including audio waveforms of human speech or synthetic speech, may include only other terms / phrases from the ground truth transcription 114 while omitting one or more specific phrases 116. In these implementations, for a transcription 114 that includes only one omitted specific phrase 116, the audio data 112 is divided into two segments, one ending with a start boundary where the specific phrase 116 is located, and a second audio segment starting with a end boundary where the specific phrase 116 is located. The data augmenter 130 can generate a mask / augmented audio segment 302 (FIG. 3) associated with the duration of the noise and generate modified audio data 300 by combining two audio segments of the audio data 112 with the mask / augmented audio segment 302 inserted between them. Thus, the leakage detector 170 compares the modified audio data 300 with the ground truth transcription 112 (including the specific phrase 116) to check whether the ASR model 150 has leaked either the specific phrase 116 or a similar phrase 117 of a different length from the training dataset 210 used to train the model.

[0021] For example, rather than transmitting the transcription 114, "I know Mr. Havisham has this information," to the TTS system 110 to convert into audio data 112 corresponding to synthetic speech, the transcription 114 can be split into the segments "I know Mr. Havisham" and "I have this information," omitting the specific phrase 116, "Havisham," and input to the TTS system 110. The ground truth transcription 114 input to the leakage detector 170 still includes the specific phrase 116, "Havisham." Once the TTS system 110 converts the two segments of the transcription 114 into corresponding synthetic speech segments in the audio data 112, the data extender 130 can generate the corresponding masked audio segment 302 and combine the two synthetic speech segments / phrases, including the masked audio segment 302 located between them, to generate modified audio data 300. Here, the masked audio segment 302 may include a noisy audio segment having a duration associated with the particular phrase 116, "Havisham." In particular, the data augmenter 130 may alter the duration of the augmented / masked audio segment 302 to attempt to determine whether the ASR model 150 is leaking a similar phrase 117 (e.g., another character name, such as Hale, if the duration is shortened). Similarly, when the audio data 112 corresponds to a human voice spoken by the user 104, the user 104 may simply speak the two phrases, "I know Mr." and "I have this information." In this scenario, the user 104 may provide some input indicating that the data augmenter 130 should generate a corresponding masked audio segment 302 to insert between the two speech phrases to generate the modified audio data 300.

[0022] As will become apparent, regardless of whether the audio data 112 includes human audio waveforms or synthetic speech, the transcription 114 of the utterance 106 also serves as a corresponding ground truth transcription 114 of the utterance 106 for comparison with a predicted transcription 154 output by an ASR model 150 trained based on speech recognition performed on modified audio data 300 that obfuscates / masks specific phrases 116a, 116b, “Somes” and “Havisham,” from the original input utterance 106. The ground truth transcription 114 includes one or more specific phrases 116 along with other terms / phrases associated with the audio data 112 and corresponding temporal information 115. The specific phrases 116 include phrases that are obfuscated or expanded / substituted with noise to render them unrecognizable in the modified audio data 300. In some examples, the user 104 provides input to the computing device 102 explicitly identifying the specific phrases 116 to be obfuscated. In other examples, the phrase extraction mechanism 108 identifies particular phrases contained in the transcription 114 that are associated with specific categories of information. In these examples, the user 104 can provide input indicating one or more specific categories of information, and the phrase extraction mechanism 108 can process the ground truth transcription 114 to identify any phrases associated with the specific categories of information. Thus, a particular phrase 116 can be identified based on explicit user input indicating a selection of the phrase 116, or a particular phrase 116 can be identified as being associated with a specific category of information. As used herein, a phrase may include one or more terms, such as a name (e.g., Havisham or John Smith), a city (e.g., New Haven), a zip code sequence (e.g., 4-8-3-0-4), an address (e.g., 1234 Lavender Ln), or a date of birth (e.g., January 1, 2000).The temporal information 115 may include timestamps for the start and end boundaries of each word / term in the transcription 114. In other words, the temporal information 115 defines the start and end boundaries of each word / term in the transcription 114. In some implementations, if the transcription 114 is paired with corresponding audio data and is derived from a training dataset 210 used to train the model 150, the temporal information 115 may be obtained via annotated timestamps during training and stored with the training dataset 210. Alternatively, if the transcription 114 is provided to the TTS system 110 and converted into audio data 112 corresponding to synthetic speech, the TTS system 110 may generate the temporal information 115 when performing text-to-speech conversion on the transcription 114.

[0023] The phrase extraction mechanism 108 further includes an audio-to-text aligner 120 configured to receive the audio data 112 and a transcription 114 including one or more phrases 116 and corresponding time information 115, and to identify one or more segments 122 of the audio data 112 that align with the one or more particular phrases 116 identified in the transcription 114. In other words, the audio-to-text aligner 120 receives the transcription 114 including the time information 115, uses the time information 115 to identify segments 122 of the audio data 112 that align with the particular phrases 116 in the transcription 114, and outputs the audio data 112 including each identified segment 122. More specifically, the audio-to-text aligner 120 receives the transcription 114 including the corresponding time information 115, and the transcription 114 includes the content, "Mr. Soames and Mr. Havisham were overwhelmed by this information." Here, "Somes" is a specific phrase 116a and "Havisham" is a specific phrase 116b. The identified specific phrases 116a, 116b may belong to the same information category 212 ( FIG. 2 ) (e.g., character names) in the training dataset 210. The audio-text aligner 120 uses the temporal information 115 to align the transcription 114 with the audio data 112, enabling the audio-text aligner 120 to identify segments 122a, 122b in the audio data 112 that convey the specific phrases 116a, 116b. In this example, the audio-text aligner 120 identifies the segment 122a of the audio data 122 that matches the specific phrase "Somes" 116a of the transcription 114, and also identifies the segment 122b of the audio data 122 that matches the specific phrase "Havisham" 116b of the transcription 114.Once the audio text aligner 120 identifies segments 122a, 122b of the audio data 112 that match particular phrases 116a, 116b, the audio data 112 including the identified segments 122a, 122b is provided to a data extender 130, which extends the audio data 112 to generate modified audio data 300.

[0024] The data enhancer 130 receives audio data 112 (e.g., from the audio text aligner 120) including one or more identified segments 122 corresponding to one or more specific phrases 116 and performs data enhancement on each identified segment 122 of the audio data 112 to obfuscate the corresponding specific phrases 116 in the audio data 112 that characterize the utterance 106. That is, the data enhancer 130 receives the audio data 112 and enhances the audio data 112 based on the identified segments 122 corresponding to the specific phrases 116 to generate modified audio data 300 including enhanced audio segments 302 ( FIG. 3 ) that obfuscate the specific phrases 116. In some implementations, the data enhancer 130 modifies the audio data 112 using a data enhancement technique to distort each identified segment 122 that aligns with the corresponding identified specific phrase 116. Examples herein refer to the data enhancer 130 modifying the audio data 112 by adding noise of a duration corresponding to the length of the identified segment 122. Other data augmentation techniques may also be applied in addition to or instead of noise.

[0025] After the data augmenter 130 modifies the audio data 112 by obfuscating the particular phrase 116 in the identified segment 122 to generate modified audio data 300, the model 150 receives the modified audio data 300 and uses the modified audio data 300 to generate, as output, a corresponding predicted transcription 154 of the modified audio data 300. That is, the model 150 receives the modified audio data 300 and generates, based on the modified audio data 300, the predicted transcription 154 of the modified audio data 300. The leakage detector 170 receives, as input, the predicted transcription 154 of the modified audio data 300 and the ground truth transcription 114, and generates, as output, an indication 180 of whether the trained ASR model 300 leaked either the particular phrase 116 or a similar phrase 117 from the training dataset 210 used to train the ASR model 150. That is, the leakage detector 170 compares the predicted transcription 154 of the modified audio data 300 with the ground truth transcription 114 of the utterance 106 to determine whether the predicted transcription 154 contains the particular phrase 116 that was replaced with noise in the modified audio data 300. In the illustrated example, the output indication 180 indicates that the correct particular phrase 116b, "Havisham," was extracted from the modified audio data 300, even though the modified audio data 300 augments the corresponding audio segment 122b with noise. If the leakage detector 170 determines that the predicted transcription 154 does not include the particular phrase 116 or another similar phrase 117 used in place of the particular phrase 116 from the transcription 114 associated with the same information category 212 as the particular phrase 116, the leakage detector 170 generates an output 180 indicating that the model 150 has not leaked any particular phrases 116 from the information categories 212 of the training dataset 210 used to train the model 150.

[0026] In some examples, the predicted transcription 154 includes another phrase 117 associated with the same information category 212 ( FIG. 2 ) as the particular phrase 116 from the transcription 114. For example, the information category 212 may include a character name, an address, a zip code, a date of birth, a patient's diagnosis, or any other information category 212 that refers to the type of training dataset 210 used to train the model 150. In these examples, the leakage detector 170 generates an output 180 indicating that the model 150 leaked another phrase 117 from the training dataset 210 used to train the model 150. In the illustrated example, the output indication 180 indicates that a similar phrase 117, “Hale,” was extracted from the modified audio data 300 at the location of the corresponding audio segment 122a that was noise-augmented to obfuscate the particular phrase 116a, “Soames.” Here, "Hale" and "Soames" both correspond to character names in the audiobook used to train the ASR model 300, and the ASR model 150 recognizing the phrase "Hale" indicates that the trained ASR model 150 is leaking data from the training dataset 210 by replacing the particular phrase 116a "Soames" with the similar phrase 117 "Hale."

[0027] In some examples, the output 180 includes a leak report 182 for the user 104 to review. For example, the leak report 182 may be displayed on the display / screen 105 of the user device 102 or another display device associated with the user 104. The user device 102 may execute a user interface generator configured to present a representation of the leak report 182 to the user 104 of the user device 102. The leak report 182 may include a notification / alert when a leak is detected or a notification that the model 150 has not leaked any information from the training dataset 210. In some examples, the leak report 182 includes a compilation of the outputs 180 from multiple audio data 112 and transcription 114 pairs, indicating the percentage of specific phrases extracted (e.g., the specific phrase 116 itself, or other similar phrases 117 used in place of the specific phrase 116 and associated with the same information category 212) and therefore leaked by the model 150. The leak report 180 may also indicate that the trained ASR model 150 has leaked data from the training dataset 210 associated with some specific information categories 212, but not others. For example, the leak report 182 may indicate that a leak was detected in the information category 212 related to names / proper nouns, but not in other information categories 212 related to birth dates, account numbers, etc. In these scenarios, the leak report 182's indication that the ASR model 150 has leaked data from some information categories 212 and not others can be based on the information categories associated with specific phrases 116 identified as replaced with noise from the multiple audio data 112 and transcription 114 pairs input to the phrase extraction model 108.

[0028] In some embodiments, if the leak detector 170 determines that the model 150 has leaked a particular phrase 116 and / or similar phrases 117 included in a category 212 of information from the training dataset 210 used to train the model 150, the leak detector 170 further generates a model update 184 for the model 150 to improve the security / privacy of the model 150. That is, the model update 184 may update the model 150 to prevent further data leaks (e.g., the particular phrase 116 or similar phrases 117). In these embodiments, the ASR system 108 may be automated, whereby the leak detector 170 automatically provides the model update 184 to the model 150. The model update 184 may include parameters that cause the model 150 to output only the symbols "######" or spaces in future predicted transcriptions 154 when an identified segment 122 of audio data 112 that matches a particular phrase 116 occurs and the particular phrase 116 is not included in the predicted transcription 154 output by the model 150. Additionally or alternatively, the model update 184 may be included in the breach report 182 to communicate steps that the user 102 should take to improve the security of the model 150.

[0029] Referring briefly to FIG. 3 , the data enhancer 130 performs data enhancement on the identified segments 122a, 122b in the audio data 112 to obfuscate specific phrases 116a, 116b spoken in the utterance 106 and generate corresponding masked segments 302. As used herein, the terms “masked segments” and “augmented segments” may be used interchangeably. The data enhancer 130 may perform data enhancement by adding noise to the identified segments 122a, 122b in the audio data 112. In some examples, the data enhancer 130 concatenates the identified segments 122a, 122b in the audio data 112 with a noisy audio segment to generate enhanced audio segments 302a, 302b that include both the audio data 112 and the added noisy audio segment. In other examples, the data enhancer 130 generates the enhanced audio segments 302a, 302b by replacing the audio data 112 in the identified segments 122a, 122b with the noisy audio segment. In other words, the data extender 130 removes the audio data 112 occurring in the identified segments 122a, 122b and inserts the corresponding noisy audio segments to generate extended audio segments 302a, 302b in the modified audio data 300.

[0030] In some implementations, the data extender 130 includes padding in the extended audio segments 302a, 302b to limit certain phrases 116a, 116b within the identified segments 122a, 122b from overflowing into the remainder of the audio data 112. The padding may be an amount of time (e.g., 100 milliseconds) separating the identified segments 122a, 122b from the remainder of the audio data 112. In these implementations, padding is added before and after each identified segment 122a, 122b. Once the padding is added to the identified segments 122a, 122b, the data extender 130 performs data extension on the padding and the identified segments 122a, 122b to generate extended audio segments 302a, 302b within the modified audio data 300.

[0031] Returning to FIG. 1 , modified audio data 300 including expanded audio segments 302 a, 302 b that obfuscate identified segments 122 a, 122 b is provided to model 150, which in turn generates predicted transcription 154: "Mr. Hale and Mr. Havisham were overwhelmed by this information." In this example, predicted transcription 154 includes specific phrase 116 b "Havisham" and another phrase 117 "Hale" that substitutes for specific phrase 116 a "Soames" in transcription 114. The other phrase 117 "Hale" is associated with the same category of information 212 as the specific phrase 116 a "Soames," and both the other phrase 117 and the specific phrase 116 a are associated with the same category of information 212 (e.g., character names) in the training dataset 210 used to train model 150.

[0032] The leakage detector 170 compares the predicted transcription 154, which includes the other phrases 117 and the specific phrase 116b, with the ground truth transcription 114, which includes the identified specific phrases 116a, 116b, to determine whether the trained TTS model 150 leaked the specific phrase 116b and other phrases 117 from the training dataset 210 used to train the model 150. Here, the leakage detector 170 may determine that the specific phrase 116b contains an exact match extracted from the modified audio data. The leakage detector 170 may further determine that the other phrase 117, "Hale," does not exactly match the specific phrase 116a, "Soames," but that the other phrase 117 corresponds to a similar phrase because both "Hale" and "Soames" are associated with the same category of information (e.g., character names) in the training dataset 210 (e.g., audiobooks) used to train the ASR model 150. In response to determining that the model 150 has leaked the particular phrase 116b and other phrases 117, the leak detector 170 generates an output 180 indicating that the model 150 has leaked the particular phrase 116b and other phrases 117 from the training dataset 210 used to train the model 150.

[0033] The output 180 from the leak detector 170 may further include or be used to compile a leak report 182 stating, "Warning: The trained ASR model is leaking data from the training dataset." The warning may occur when a threshold number of specific or similar phrases are extracted from the audio data 112 characterizing multiple test utterances. In the illustrated example, the leak report 182 further reports a tally of one or more previous outputs 180 containing the specific leaked phrase 116 as "85% accurate phrase extracted" and a tally of one or more previous outputs 180 containing other leaked phrases 117 as "79% similar phrase extracted." As described above, the leak report 182 is displayed on the screen 105 of the user device 102 to notify the user 104 of the user device 102 whether a leak has been detected. The leak detector 170 may further provide model updates 184 to the model 150 to improve the security / privacy of the model 150 and prevent further information leaks from the training dataset 210 used to train the model 150. The model 150 may automatically update the model 150 using the model updates 184 and / or the model updates 184 may include user instructions / suggestions for updating the model 150 to prevent data leaks from the training dataset 210.

[0034] 2 , the phrase extraction mechanism 108 may include a phrase identifier 200 that enables the user 102 to select specific categories of information 212 in a training dataset 210 that the phrase identifier 200 will identify in the utterance 106. In some examples, the user 102 provides a category selection input 202 that corresponds to the categories of information 212 in the training dataset 210 used to train the model 150. In other examples, the user 102 identifies specific phrases 116 in the transcription 114 to be obfuscated in the modified audio data 300. For example, the transcription 114 may be displayed on the screen 105, and the user 102 may provide an indication of a user input to identify specific phrases 116 to obfuscate in the modified audio data 300.

[0035] Each information category 212 in the training dataset 210 may include one or more example phrases 214, 214a-n, corresponding to the information category 212. When the user 102 provides a category selection input 202 to the phrase identifier 200, the phrase identifier 200 can process the transcription 114 to identify example phrases 214 that correspond to the information category 212 present in the transcription 114. The phrase identifier 200 outputs these identified example phrases 214 as identified specific phrases 116 in the transcription 114. In other words, the user 102 inputs the category selection input 202 of the phrase identifier 200 to identify specific phrases 116 in the utterance 106 that belong to the category 212 associated with the category selection input 202. For example, the information category 212 may correspond to the name of a character in an audiobook used to train the ASR model 150. Here, example phrases 214 may include a list of all character names that appear in training dataset 210 (eg, Soames, Havisham, Hale, etc.).

[0036] As shown in FIG. 2 , a category selection input 202 and a corresponding ground truth transcription 114 are provided to a phrase identifier 200, and in response, the phrase identifier 200 determines whether the ground truth transcription 114 of the utterance 106 contains any specific phrases 116 associated with the information category 212 specified by the category selection input 202. In this example, the category selection input 202 specifies a specific name information category 212a (e.g., "Find a character name"). The name information category 212a includes example phrases 214a-n, which may include "Soames," "Hale," and "Havisham." The phrase identifier 200 processes the information category 212a containing the example phrases 214a-n and the transcription 114 to automatically determine that the transcription 114 contains the specific phrases 116a, 116b, "Soames" and "Havisham," which belong to the specific information category 212a entered as the category selection input 202. The phrase identifier 200 then provides the transcription 114 to the audio-text aligner 120 for correction. In other words, the audio-text aligner 120 and data augmenter 130 correct the audio data 112 in response to the phrase identifier 200 identifying that a particular phrase 116a, 116b is included in the transcription 114 associated with the name-specific information category 212a. This example is illustrative only and is not intended to limit the scope of the information categories 212. The information categories 212 may include any category that characterizes the training dataset 210, such as names, addresses, dates, zip codes, patient diagnoses, account numbers, phone numbers, etc.

[0037] 4 is a flowchart of an example configuration of operations for an ASR model phrase extraction method 400. The method 400 includes, at operation 402, obtaining audio data 112 characterizing an utterance 106 and a corresponding ground truth transcription 114 of the utterance 106. At operation 404, the method 400 includes modifying the audio data 112 to obfuscate particular phrases 116 spoken in the utterance 106.

[0038] At operation 406, the method 400 also includes processing the modified audio data 300 with the trained ASR model 150 to generate a predicted transcription 154 of the utterance 106. The method 400 further includes, at operation 408, determining whether the predicted transcription 154 includes the particular phrase 116 by comparing the predicted transcription 154 of the utterance 106 to a ground truth transcription 114 of the utterance 106. At operation 410, the method 400 also includes generating an output 180 indicating that the trained ASR model 150 leaked the particular phrase 116 from the training dataset 210 used to train the ASR model 150 when the predicted transcription 154 includes the particular phrase 116.

[0039] 5 is a schematic diagram of an exemplary computing device 500 that can be used to implement the systems and methods described herein. Computing device 500 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The components shown here, their connections and relationships, and their functions are for illustrative purposes only and are not intended to limit the implementation of the embodiments described and / or claimed in this document.

[0040] Computing device 500 includes a processor 510, memory 520, a storage device 530, a high-speed interface / controller 540 that connects to memory 520 and a high-speed expansion port 550, and a low-speed interface / controller 560 that connects to a low-speed bus 570 and storage device 530. Each of components 510, 520, 530, 540, 550, and 560 are interconnected using various buses and may reside on a common motherboard or exist in other manners as desired. Processor 510 (e.g., data processing hardware 111 of FIG. 1 ) processes instructions for execution within computing device 500, including instructions stored in memory 520 or storage device 530, and can display graphical information for a graphical user interface (GUI) on an external input / output device, such as a display 580 connected to high-speed interface 540. In other implementations, multiple processors and / or multiple buses may be used as needed, along with multiple memories and memory types. Also, multiple computing devices 500 may be connected, each performing some of the required operations (eg, as a bank of servers, a group of blade servers, or a multi-processor system).

[0041] The memory 520 (e.g., memory hardware 113 of FIG. 1 ) non-transitoryly stores information within the computing device 500. The memory 520 may be a computer-readable medium, a volatile memory unit(s), or a non-volatile memory unit(s). The non-transitory memory 520 may be a physical device used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program state information) for use by the computing device 500. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), as well as disks or tapes.

[0042] The storage device 530 can provide mass storage for the computing device 500. In some implementations, the storage device 530 is a computer-readable medium. In various different implementations, the storage device 530 can be a floppy disk device, a hard disk device, an optical disk device, or an array of devices including a tape device, a flash memory or other similar solid-state memory device, or a storage area network or other configuration of devices. In additional implementations, a computer program product is tangibly embodied on an information carrier. The computer program product also includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer-readable or machine-readable medium, such as the memory 520, the storage device 530, or memory on the processor 510.

[0043] The high-speed controller 540 manages more bandwidth-intensive operations of the computing device 500, while the low-speed controller 560 manages less bandwidth-intensive operations. This assignment of duties is merely exemplary. In some implementations, the high-speed controller 540 is coupled to memory 520, a display 580 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 550 that can accept various expansion cards (not shown). In some implementations, the low-speed controller 560 is coupled to a storage device 530 and a low-speed expansion port 590. The low-speed expansion port 590 may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet, etc.) and may connect, via a network adapter or the like, to one or more input / output devices such as a keyboard, pointing device, scanner, or network devices such as a switch or router.

[0044] The computing device 500 can be implemented in many different forms, as shown in the figure: for example, it can be implemented as a standard server 500a, or multiple times in a group of such servers 500a, as a laptop computer 500b, or as part of a rack server system 500c.

[0045] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry and / or integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementation in one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, which may be specialized or general-purpose, coupled to receive data and instructions from the storage system, and to transmit data and instructions to the storage system.

[0046] A software application (i.e., a software resource) may refer to computer software that causes a computing device to perform tasks. In some examples, a software application may be referred to as an "application," "app," or "program." Exemplary applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0047] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language, and / or assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic circuit (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0048] The processes and logic flows described herein may be executed by one or more programmable processors, also referred to as data processing hardware, which execute one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be executed by special-purpose logic circuitry, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). Processors suitable for executing computer programs include, by way of example, both general-purpose and special-purpose processors, and any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is operatively coupled to receive data from them, transfer data to them, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks, such as internal or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0049] To interact with a user, one or more aspects of the present disclosure can be implemented in a computer that has a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen, for displaying information to the user, and optionally a keyboard and pointing device, such as a mouse or trackball, that the user can use to provide input to the computer. Other types of devices can also be used to interact with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, verbal, or tactile input. Additionally, a computer can interact with a user by sending and receiving documents to a device used by the user, for example, by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0050] Numerous embodiments have been described. It is, of course, understood that various modifications can be made without departing from the spirit and scope of the disclosure. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A computer-implemented method (400), when executed by data processing hardware (510), causing the data processing hardware (510) to: obtaining audio data (112) characterizing an utterance (106) and a corresponding ground truth transcription (114) of the utterance (106); modifying the audio data (112) to obfuscate specific phrases (116) spoken in the utterance (106); processing the modified audio data (300) using a trained automatic speech recognition (ASR) model (150) to generate a predictive transcription (154) of the utterance (106); determining whether the predicted transcription (154) of the utterance (106) includes the particular phrase (116) associated with a specific category (212) of information by comparing the predicted transcription (154) of the utterance (106) with the ground truth transcription (114) of the utterance (106), the specific category (212) of information including a name, an address, a date, a zip code, a patient diagnosis, an account number, or a telephone number; and performing operations including, when the predicted transcription (154) includes the particular phrase (116), generating an output (180) indicating that the trained ASR model (150) has leaked the particular phrase (116) from a training dataset (210) used to train the ASR model (150); The computer-implemented method (400).

2. 2. The computer-implemented method of claim 1, wherein the operations further include, when the predicted transcription includes another phrase in place of the particular phrase from the ground truth transcription that is associated with the same information category as the particular phrase, generating an output indicating that the trained ASR model has leaked the other phrase from the training dataset used to train the ASR model.

3. 3. The computer-implemented method of claim 1, wherein the operations further comprise: generating an output indicating that the trained ASR model has not leaked any information from the training dataset used to train the ASR model when the predicted transcription does not include the particular phrase or another phrase in place of the particular phrase from the ground truth transcription that is associated with the same category of information as the particular phrase.

4. The computer-implemented method (400) of any one of claims 1 to 3, wherein the audio data (112) comprises an audio waveform.

5. The computer-implemented method (400) of claim 4, wherein the audio waveform corresponds to a human voice.

6. The computer-implemented method (400) of claim 4 , wherein the audio waveform corresponds to synthesized speech.

7. Modifying the audio data (112) comprises: identifying, based on the ground truth transcription, segments (122) of the audio data (112) that match the particular phrases (116) in the ground truth transcription (114); and performing data augmentation on the identified segments (122) of the audio data (112) to obfuscate the specific phrases (116) spoken in the utterance (106); The computer-implemented method (400) of any one of claims 1 to 6, comprising:

8. 8. The computer-implemented method of claim 7, wherein performing data augmentation on the identified segment of the audio data comprises adding noise to the identified segment of the audio data.

9. 8. The computer-implemented method of claim 7, wherein performing data augmentation on the identified segment of the audio data comprises replacing the identified segment of the audio data with noise.

10. The operation is processing the ground truth transcription (114) of the utterance (106) to identify any phrases (214) contained in the ground truth transcription (114) that are associated with specific categories of information (212); modifying the audio data (112) in response to identifying that the particular phrase (116) included in the ground truth transcription (114) is associated with the specific category of information (212); A computer-implemented method (400) according to any one of claims 1 to 9.

11. A system comprising: Data processing hardware (510), and memory hardware (520) in communication with the data processing hardware (510), the memory hardware (520), when executed on the data processing hardware (510), to the data processing hardware (510): obtaining audio data (112) characterizing an utterance (106) and a corresponding ground truth transcription (114) of the utterance (106); modifying the audio data (112) to obfuscate specific phrases (116) spoken in the utterance (106); processing the modified audio data (300) using a trained automatic speech recognition (ASR) model (150) to generate a predictive transcription (154) of the utterance (106); determining whether the predicted transcription (154) of the utterance (106) includes the particular phrase (116) associated with a specific category (212) of information by comparing the predicted transcription (154) of the utterance (106) with the ground truth transcription (114) of the utterance (106), the specific category (212) of information including a name, an address, a date, a zip code, a patient diagnosis, an account number, or a telephone number; and When the predicted transcription (154) includes the particular phrase (116), generating an output (180) indicating that the trained ASR model (150) has leaked the particular phrase (116) from a training dataset (210) used to train the ASR model (150); the memory hardware (520) storing instructions for performing operations including The system.

12. 12. The system of claim 11, wherein the operations further include, when the predicted transcription includes another phrase (117) in place of the particular phrase (116) from the ground truth transcription (114) that is associated with the same information category (212) as the particular phrase (116), generating an output (180) indicating that the trained ASR model (150) has leaked the another phrase (117) from the training dataset (210) used to train the ASR model (150).

13. 13. The system of claim 11 or 12, wherein the operations further include generating an output indicating that the trained ASR model has not leaked any information from the training dataset used to train the ASR model when the predicted transcription does not include the particular phrase or another phrase in place of the particular phrase from the ground truth transcription that is associated with the same category of information as the particular phrase.

14. The system of any one of claims 11 to 13, wherein the audio data (112) comprises an audio waveform.

15. The system of claim 14 , wherein the audio waveform corresponds to a human voice.

16. The system of claim 14 , wherein the audio waveform corresponds to synthesized speech.

17. Modifying the audio data (112) comprises: identifying, based on the ground truth transcription, segments (122) of the audio data (112) that match the particular phrases (116) in the ground truth transcription (114); and 17. The system of claim 11, further comprising performing data augmentation on the identified segments (122) of the audio data (112) to obfuscate the particular phrases (116) spoken in the utterance (106).

18. 20. The system of claim 17, wherein performing data enhancement on the identified segment of the audio data includes adding noise to the identified segment of the audio data.

19. 20. The system of claim 17, wherein performing data augmentation on the identified segment (122) of the audio data (112) comprises replacing the identified segment (122) of the audio data (112) with noise.

20. The operation is processing the ground truth transcription (114) of the utterance (106) to identify any phrases (214) contained in the ground truth transcription (114) that are associated with specific categories of information (212); modifying the audio data (112) in response to identifying that the particular phrase (116) included in the ground truth transcription (114) is associated with the specific category of information (212); A system according to any one of claims 11 to 19.

Images